Skip to content

opentelemetry-instrumentation-botocore: capture Bedrock prompt cache token usage#4615

Open
vinigrazzioli-96 wants to merge 3 commits into
open-telemetry:mainfrom
vinigrazzioli-96:botocore-bedrock-cache-tokens
Open

opentelemetry-instrumentation-botocore: capture Bedrock prompt cache token usage#4615
vinigrazzioli-96 wants to merge 3 commits into
open-telemetry:mainfrom
vinigrazzioli-96:botocore-bedrock-cache-tokens

Conversation

@vinigrazzioli-96
Copy link
Copy Markdown

Description

Amazon Bedrock's Converse / ConverseStream APIs return cacheReadInputTokens
and cacheWriteInputTokens in the response usage when prompt caching is
used. The botocore Bedrock instrumentation currently reads only inputTokens
/ outputTokens, so the cache token counts are dropped.

This PR maps them to the OTel GenAI semantic-convention attributes (already
defined in opentelemetry-semantic-conventions):

  • cacheReadInputTokensgen_ai.usage.cache_read.input_tokens
    Changes:
  • bedrock.py (_converse_on_success): set the cache token span attributes
    for Converse / ConverseStream / InvokeModelWithResponseStream.
  • bedrock_utils.py (ConverseStreamWrapper): accumulate the cache token
    counts from the streaming metadata event into the response usage.

Out of scope (possible follow-up): the non-streaming InvokeModel Claude
path uses the native Anthropic usage format (cache_read_input_tokens /
cache_creation_input_tokens, snake_case) and would need a separate change.

Fixes #4614

Type of change

  • New feature (non-breaking change which adds functionality)

How Has This Been Tested?

  • Added test_converse_stream_accumulates_cache_tokens, a unit test that
    feeds a metadata event carrying cache token usage to ConverseStreamWrapper
    and asserts the counts are accumulated.
  • tox -e py311-test-instrumentation-botocore-1-wrapt1 — 134 passed
  • tox -e lint-instrumentation-botocore — 10.00/10

Checklist

  • Followed the style guidelines of this project
  • Changelog has been updated
  • Unit tests have been added
  • Documentation has been updated (N/A — internal attribute change)

@vinigrazzioli-96 vinigrazzioli-96 requested a review from a team as a code owner May 21, 2026 14:31
@linux-foundation-easycla
Copy link
Copy Markdown

linux-foundation-easycla Bot commented May 21, 2026

CLA Signed
The committers listed above are authorized under a signed CLA.

  • ✅ login: vinigrazzioli-96 / name: Vinicius Moscon (55d8c17)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

Amazon Bedrock (botocore) instrumentation does not capture prompt cache token usage

1 participant