AI Observability by OpenObserve

AI Observability by OpenObserve extends the open-source OpenObserve platform to cover AI workloads. It records each model call as an OpenTelemetry span, so developers can follow agent behaviour with the same instrumentation they already use elsewhere. It is aimed at engineering teams running LLM-based applications and agents.

What AI Observability by OpenObserve offers

• Prompt-level tracing: every prompt, response, tool call and error is captured, down to the individual step in an agent run.
• Cost tracking: token counts and computed cost are attached to each span, so expensive steps can be sorted and found.
• Latency and error monitoring: shows where time goes, including delays outside the model such as database waits.
• Evaluations: quality scoring on the recorded runs, at trace or session level.
• Framework support: accepts traces from LangChain, LlamaIndex and any OpenTelemetry-instrumented code without rewriting instrumentation.

Who it serves

Teams that want AI telemetry stored alongside their logs, metrics and infrastructure data.

Key facts

• Deployment: self-hosted, in the customer's own cloud, or hosted
• Data handling: redaction at ingest and field-level masking
• Base platform: open-source OpenObserve