Open-source observability for your LLM application
OpenLLMetry is a set of extensions built on top of OpenTelemetry that gives you complete observability over your LLM application. Because it uses OpenTelemetry under the hood, it can be connected to your existing observability solutions - Datadog, Honeycomb, and others.
It's built and maintained by Traceloop under the Apache 2.0 license.
The repo contains standard OpenTelemetry instrumentations for LLM providers and Vector DBs, as well as a Traceloop SDK that makes it easy to get started with OpenLLMetry, while still outputting standard OpenTelemetry data that can be connected to your observability stack. If you already have OpenTelemetry instrumented, you can just add any of our instrumentations directly.
The easiest way to get started is to use our SDK. For a complete guide, go to our docs.
Install the SDK:
gem install traceloop-sdkThen, to start instrumenting your code, just add this line to your code:
require "traceloop/sdk"
traceloop = Traceloop::SDK::Traceloop.newThat's it. You're now tracing your code with OpenLLMetry!
Now, you need to decide where to export the traces to.
You can customize your service name by providing a name parameter:
require "traceloop/sdk"
# Without name parameter (uses OTEL_SERVICE_NAME as-is)
traceloop = Traceloop::SDK::Traceloop.new
# Service name: value of OTEL_SERVICE_NAME, or "unknown_service:ruby"
# With name parameter (combines name with OTEL_ENVIRONMENT)
traceloop = Traceloop::SDK::Traceloop.new(name: "worker")
# Service name: "worker-production" (if OTEL_ENVIRONMENT="production")
# Service name: "worker-unknown" (if OTEL_ENVIRONMENT not set)You can create multiple Traceloop instances with different service names in the same application:
traceloop_api = Traceloop::SDK::Traceloop.new(name: "api")
traceloop_worker = Traceloop::SDK::Traceloop.new(name: "worker")
traceloop_scheduler = Traceloop::SDK::Traceloop.new(name: "scheduler")
# Each instance traces with its own service name (assuming OTEL_ENVIRONMENT="production"):
# - "api-production"
# - "worker-production"
# - "scheduler-production"Control your service naming using standard OpenTelemetry environment variables:
# Used when no name parameter is provided
export OTEL_SERVICE_NAME="my-app"
# Combined with name parameter: "worker-production"
export OTEL_ENVIRONMENT="production"Defaults:
OTEL_SERVICE_NAMEdefaults to"unknown_service:ruby"OTEL_ENVIRONMENTdefaults to"unknown"
When shutting down your application, ensure spans are properly flushed:
traceloop = Traceloop::SDK::Traceloop.new
# ... use traceloop ...
traceloop.shutdown # Flush remaining spans before exit- Traceloop
- Dynatrace
- Datadog
- New Relic
- Honeycomb
- Grafana Tempo
- HyperDX
- SigNoz
- OpenTelemetry Collector
See our docs for instructions on connecting to each one.
OpenLLMetry is in early-alpha exploratory stage, and we're still figuring out what to instrument. As opposed to other languages, there aren't many official LLM libraries (yet?), so for now you'll have to manually log prompts:
require "openai"
client = OpenAI::Client.new
traceloop.llm_call() do |tracer|
tracer.log_prompt(model="gpt-3.5-turbo", user_prompt="Tell me a joke about OpenTelemetry")
response = client.chat(
parameters: {
model: "gpt-3.5-turbo",
messages: [{ role: "user", content: "Tell me a joke about OpenTelemetry" }]
})
tracer.log_response(response)
endWhether it's big or small, we love contributions ❤️ Check out our guide to see how to get started.
Not sure where to get started? You can:
- Book a free pairing session with one of our teammates!
- Join our Slack, and ask us any questions there.
- Slack (For live discussion with the community and the Traceloop team)
- GitHub Discussions (For help with building and deeper conversations about features)
- GitHub Issues (For any bugs and errors you encounter using OpenLLMetry)
- Twitter (Get news fast)
To @patrickdebois, who suggested the great name we're now using for this repo!