Export over OpenTelemetry

asz push sends what the storage root holds to an OpenTelemetry logs receiver: every landed file and every round, as OTLP logs over gRPC, or over HTTP with a protobuf body. The SkyWalking OAP accepts both, on its gRPC port and on its REST port, and so does an OpenTelemetry Collector.

# asz.yaml
export:
  otlp:
    protocol: grpc              # the default; http posts to the receiver's /v1/logs instead
    endpoint: 127.0.0.1:11800   # the OAP's gRPC port; with http, http://127.0.0.1:12800

./bin/asz push            # send everything not yet sent, then exit
./bin/asz collect         # land, parse and send, once every collector interval

One log record per file

A file is sent whole: one log record whose body is the file’s bytes, unchanged. A receiver stores it as it was landed, checks its digest at once, and has nothing to put back together. Every {seq, row, block} reference in a round still means what it meant, because a row is a line of the body it names.

This is the second design. The first sent one record per line, and it was measured on one conversation of 306 files: 39,683 records, attributes 16% on top of the body even after trimming, and a receiver that had to track which lines had arrived before it could verify anything. A landed file is cut at a budget and a round is cut at the same budget, so a whole file is a small record, and the same conversation is 306 records.

Landed files and rounds are both write-once, so each is sent once. push.state in the storage root lists what was sent, with the digest each file had; a file is recorded only after the request carrying it succeeded, so a failed request leaves it for the next pass.

push.state also names the receivers the files went to, with one endpoint line each: grpc://host:port, grpcs://host:port with tls: true, or the HTTP base URL. A rejected line names each file of a request a receiver took while it rejected some of its records, as Transport describes. Every save writes all three kinds, so neither new line is lost at the next request. An older asz passes over both, and drops them when it saves the file.

schema 1
updated_at 2026-09-11T10:00:00Z
endpoint grpc://127.0.0.1:11800
pushed <session-id>/streams/main/transcript-20260911T095959.000000000Z-000001.sd 9f2c…
rejected <session-id>/streams/main/transcript-20260911T095959.000000000Z-000001.sd

Removing a scenario session depends on these lines. A session goes only when push.state names exactly one receiver, the one the pipeline sends to, and no file of the session has a rejected line. A push.state that lists files and names no receiver was written before these lines existed, or saved by an older asz. It then names the receiver unknown, and no scenario session on that root is removed after that.

Transport

The request is the ExportLogsServiceRequest of the OpenTelemetry protocol, built from the protocol’s own Go definitions and sent by the gRPC client of the official module, or posted over HTTP with a protobuf body. Nothing about the encoding is the project’s own, so any OTLP logs receiver reads it.

headers travel with every request on both transports, as gRPC metadata or as HTTP headers, which is where an authorization token goes. A receiver that answers with a partial success, saying it rejected some records, has taken the request, and the protocol says not to send it again: the count is reported on the pass line as rejected, and the files are marked sent. Each file of that request also gets a rejected line in push.state, because the answer does not say which records were rejected. A scenario session with such a file is never removed.

Over HTTP, a request counts as taken only when the answer has a 2xx status and a body that is empty or protobuf: application/x-protobuf, as OTLP names it, or application/protobuf, whatever the parameters. A 2xx answer with a body of any other type is an error. Nothing in the request is recorded as sent, so it goes again on the next pass:

otlp: http://127.0.0.1:12800/v1/logs answered 200 OK with text/html, not an OTLP response; nothing is recorded as sent

A redirect is not followed. It is an error like any other answer outside 2xx, and a receiver that moved is configured again. Before this rule, an HTML page answered with 200, and a redirect to a login page, both left files recorded as sent that no receiver stored. The rule is the same for logs and metrics. Over gRPC, a call that returns without an error always carries the receiver’s answer.

What every record carries

The resource, which names the service a record belongs to:

The scope is github.com/apache/skywalking-ai-sessionizer with the same version. Each record then says what its file is, so a receiver can route it, index it and verify it without decoding the body:

The record’s time is chosen so a receiver can bound a read by a range it already holds. A landed file is stamped with its last record time, and a file whose records carry no time, such as a child’s meta file, with the latest record time of the session as known at push, which is always inside the session’s range and cannot go stale. A round is stamped with the session’s last activity as of that round, so a receiver’s newest row per conversation is the head. Every record also carries the time it was observed.

Every scenario in the test suite is pushed to a receiver over both transports and checked against the two tables above, in both formats, so a change to the wire that this page does not describe fails the build. See Scenarios.

Checking what a receiver gets

An OpenTelemetry Collector with a file exporter writes back what it decoded, which is the easiest way to see the records before pointing at a backend:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
        max_recv_msg_size_mib: 32   # the default is 4 MiB, below the 8 MiB batches; see Size
      http:
        endpoint: 0.0.0.0:4318      # the default limit is 20 MiB, enough
exporters:
  file:
    path: /out/logs.json
service:
  pipelines:
    logs:
      receivers: [otlp]
      exporters: [file]

Point export.otlp.endpoint at 127.0.0.1:4317, or with protocol: http at http://127.0.0.1:4318, run asz push, and read logs.json: one JSON line per request, with the resource, the scope and the records as the Collector understood them. make e2e-collector does exactly this with a generated session and a Collector container, once over each transport, then checks every record against the root and rebuilds the root from what the Collector wrote; CI runs it on every change. Writing each record’s body to asz.file under a new root gives a root that asz verify and asz view read like the original.

Metrics

asz push sends metrics as well, when an adapter produces them, under the name Claude Code’s own OpenTelemetry exporter uses, so a receiver holds one metric name whichever produced the points. What the local adapter derives is a reconstructed subset of the exporter’s family, not a copy of it, and this table says exactly which part:

The right-hand column is held to a capture of what Claude Code 2.1.260 sent to asz’s receiver from one short session, kept under internal/metrics/testdata with every identifying value replaced, by a test that fails when the exporter sends a label or a metric this table does not account for. go run ./tools/otlpdump -redact FILE.pb prints any spooled request the same way.

With metrics: true on the claude-code-local adapter, the collector derives the points from the landed files by the assembler’s own rule: the usage of a call is its last fragment’s in line order, never a sum, and only a call that finished counts. A main transcript repeats the final usage on every fragment. A child’s carries streaming partials on all but the last. A call has finished when one of its fragments reports a stop reason. A call that never does is left out, because its usage block is a streaming stub, not a count. That is about 10% of all calls in a corpus of 2,970 files and 365,825 records written by Claude Code 2.1.220 to 2.1.251. It is 14.5% of the calls on child streams and 0.02% of those on main transcripts, as Provider calls reports.

A call cut at a landed file boundary is read through following transcript files of its stream when the file’s last record is a fragment of that call. There is no one-file limit. Metadata and change files do not interrupt the reading; the first unrelated transcript record does. A tool result carries no call id, and in the same corpus a call’s own tool results sit between its fragments on 24% of multi-fragment calls. Such a result ends the read-ahead. An unfinished child call remains eligible when its final fragment arrives later. On a main transcript every fragment repeats the final usage, so the call counts in full, in the minute of the last fragment read.

Under a watching collector, a file whose last record is a fragment of a call not yet counted waits a short grace for the next file of its stream, then is derived with what it has. When a continuation is found, its last file starts the grace. A deferred session is retried even while other sessions keep changing. An unfinished call is not marked counted when the grace expires. A file that ends with a tool result does not wait. A single pass, the backfill over history that already exists, derives with what is there and never waits. A call is never counted twice, however many files its records reach, and a record the runtime re-emitted before a context reset is the same call again.

Points are summed per minute and attribute set, and the windows of one series never overlap: a point takes its minute unless the series already has a point at or past it, as when two children ran in the same minute or a child’s file landed later, in which case it follows the series’ last point. No point of a series is ever thrown away for another. The first derivation over a root with history is bounded by metrics_lookback, 24 hours unless set, and by the newest request the receiver adapter landed, so switching the flag on sends neither a year of tokens nor what the runtime’s exporter already sent. A session the first pass did not finish keeps that look-back across retries and restarts, whether a file waited for its grace or an error stopped the session before its later files. Once every file of the session is derived, a later new file is derived whole. Before a request enters the spool, an immutable receipt under the session’s metrics/ records its exact bytes and the calls it counted. If saving progress fails, the retry uses that receipt even when more source fragments have arrived, so its accounting still matches the request already written. The retry applies every such receipt before it derives any file again, and a receipt only moves a series forward. So a file that waited for its grace in the failed pass takes a window after the ones already in the spool, and no two windows of a series overlap.

The other source of the same family is the runtime’s exporter itself: the claude-code-otlp adapter receives what Claude Code sends and lands each metrics request in the same spool, bytes as received. The receiver does not check whether it has landed a request before. A request the exporter sends twice lands twice, and asz push sends both copies. One root sends one source: metrics may be on for the local adapter or for the receiver, and the configuration refuses both.

The points wait in the storage root’s _metrics/ spool, one write-once file per landed file with points or per request received, and go out in order under the same budget and the same once-only rule as the files. export.otlp.logs and export.otlp.metrics switch the two things a push sends, the files and rounds as logs and the spool as metrics, so a receiver that takes one and not the other is sent what it takes. On the way out the resource is normalised to asz’s identity, the service, the layer, the sender, so the OAP holds one service for the runtime. A receiver that answers with a partial success has taken the request, and the protocol says not to send it again: the rejected records or points are counted on the pass line as rejected, the file is marked sent, and push.state gets a rejected line for it.

Rate

A first push sends the whole history of the storage root, which can be hundreds of megabytes, as fast as the receiver takes it. export.otlp.max_bytes_per_minute caps what goes on the wire: a pass waits before a request until a minute’s budget, refilled continuously and never holding more than a minute’s worth, covers the request’s encoded size. The budget starts full, so a pass that sends a few new files never waits, and a request larger than a minute’s budget waits for a full one and leaves it empty. Zero, the default, is no limit. asz push still sends everything before it exits, and the pass line’s paused= field says how long it waited.

A pass goes session by session, the session landed first going first: its files, then its rounds. A receiver rebuilds a session once it holds both, so during a long first push the sessions become complete one after another rather than all at the end.

A receiver that answers 429 over HTTP or ResourceExhausted over gRPC is asking the sender to slow down. The pass stops there, leaves the rest for the next pass, and empties the budget, so the next request waits a whole minute’s worth when a rate is set. In watch mode a Retry-After longer than the interval is honored.

Size

A request carries at most export.otlp.batch_bytes of file bytes, 8 MiB by default, which keeps a request under the 10 MiB the OAP’s HTTP server accepts. A file larger than that is sent alone, in a request of its own. A landed file is cut at max_delta_bytes, 2 MiB by default, and a round is cut at parse.max_round_bytes, also 2 MiB, so a request normally carries several files. The exception on both sides is a single unit larger than the budget: a source record is landed whole, and a round covering one landed file is published whole. The largest source record in the measured corpus is 4.5 MB.

The receiver’s limit must cover the largest single request. The OAP accepts 50 MB over gRPC and 10 MiB over HTTP by default. An OpenTelemetry Collector accepts 4 MiB over gRPC and 20 MiB over HTTP unless its receiver is configured otherwise, so its gRPC receiver needs max_recv_msg_size_mib raised, as the example above does.