AI Observability

Moneat captures LLM calls, traces multi-step agent executions, and tracks token usage and costs.

• Trace LLM calls - Input/output, model, latency, and token counts per generation
• Agent tracing - Multi-step agent executions as trace waterfalls
• Cost tracking - Spend per model and provider over time
• Error monitoring - LLM failures, timeouts, and rate limits
• Model analytics - Performance, cost, and error rates across models

Ingestion sources

Moneat stores LLM generations in one AI observability model. Today, data can arrive through the direct LLM ingestion API or through Sentry-compatible SDK transactions that contain ai.* spans.

Direct ingestion API

If you are not using a Sentry-compatible SDK, send LLM generation events directly to Moneat.

Endpoint

POST /api/{projectId}/llm/

Authenticate using either the X-Sentry-Auth header or the sentry_key query parameter with your service public key. The endpoint accepts gzip-compressed payloads via the Content-Encoding: gzip header.

Payload format

Send a JSON object containing a generations array. Each generation represents a single LLM call:

{
  "generations": [
    {
      "trace_id": "abc123",
      "span_id": "span1",
      "parent_span_id": "",
      "name": "chat_completion",
      "model": "gpt-4o",
      "provider": "openai",
      "type": "chat",
      "input": [{"role": "user", "content": "Hello"}],
      "output": {"role": "assistant", "content": "Hi there!"},
      "input_tokens": 10,
      "output_tokens": 5,
      "cost_usd": 0.0003,
      "duration_ms": 450,
      "status": "success",
      "timestamp": "2026-02-17T10:00:00Z",
      "tags": {"user_id": "u123"},
      "metadata": {}
    }
  ]
}

Sentry-compatible SDKs

If your application uses a Sentry-compatible SDK with AI integrations, point your DSN to your Moneat service to start sending LLM observability data.

1. 1
   Install an SDK integration

   Install the Sentry-compatible integration for your LLM framework, such as OpenAI Agents, Vercel AI, LangChain, or another supported integration.
2. 2
   Use your Moneat DSN

   Configure the SDK with your Moneat service DSN instead of a Sentry DSN. You can find the DSN in service settings.
3. 3
   Send ai.* spans

   Moneat detects ai.* spans from SDK transactions and records them as LLM generations.

https://<public_key>@<your-moneat-host>/api/<project_id>

Generation fields

Supported generation types

The type field categorizes each generation. Moneat supports: chat, completion, embedding, tool_call, agent, chain, retriever.

The model and provider fields accept any string value.

Dashboard features

Overview dashboard

The AI overview page shows key metrics: total generations, total tokens, total cost, average latency, and error rate. Time-series charts show LLM call volume and breakdowns by model.

Generations browser

Browse all individual LLM calls with filters for model, provider, type, status, and time range. Each generation shows its timestamp, model, token usage, cost, and latency.

Trace detail

For multi-step agent executions, click a trace to view the full trace waterfall with parent/child relationships. Click any span to see its full details.

Cost tracking

Track your LLM spend across models and providers. Set the cost_usd field on each generation. When using Sentry-compatible SDKs with AI integrations, cost data is typically populated automatically. The cost dashboard shows total spend over time with breakdowns by model and provider.

Data retention

LLM generation data is stored in ClickHouse with plan-specific retention. Check Billing & Plans for current limits.

Next steps

• Getting Started - Set up your account and first service
• Sentry-Compatible SDKs - Setup for Sentry-compatible instrumentation
• Error Monitoring - Error capture and issue grouping
• Billing & Plans - Usage, budgets, and billing