W&B Weave is an observability and evaluation platform for building reliable LLM applications. Weave helps you understand what your AI application is doing, measure how well it performs, and systematically improve it over time. Building LLM applications is fundamentally different from traditional software development. LLM outputs are non-deterministic, making debugging harder. Quality is subjective and context-dependent. Small prompt changes can cause unexpected behavior changes. Traditional testing approaches fall short.Documentation Index
Fetch the complete documentation index at: https://wb-21fd5541-weave-agents.mintlify.app/llms.txt
Use this file to discover all available pages before exploring further.
The main threads of Weave
Weave provides the following core functionality:- Visibility into agent sessions and multi-turn conversations, or into individual function calls and outputs in application code.
- Systematic evaluation to measure performance against curated test cases.
- Version tracking for prompts, models, and data so you can understand what changed.
- Experimentation with different prompt and model comparisons.
- Feedback collection to capture human judgments and annotations.
- Monitoring in production using guardrails and scorers for LLM safety and quality.
Agentic tracing
Weave provides agentic observability for the full lifecycle of agent conversations including: sessions, multi-turn conversations between a user and an agent, LLM calls, and tool executions. If you’re building an agent, follow the agent tracing quickstart or learn to use the Weave SDK to trace your agents. If you’re using a supported third-party agent framework, such as Claude Code, Weave instruments it automatically with no additional code. See Integrations for all supported frameworks.Application Tracing
If you want to trace individual function calls, application code, or custom logic, use Weave Ops and Calls. Add one line to any function to track inputs, outputs, cost, token count, and latency.- Track end-to-end how data flows through your LLM application.
- See source documents used to produce the LLM feedback.
- Drill down into specific prompts and how answers are produced.
Evaluations
Systematically benchmark your LLM application’s performance to gain confidence when deploying to production.- Easily track which versions of model/prompt resulted in what performance.
- Define metrics to evaluate responses using one or more scoring functions.
- Compare two or more different evaluations over multiple metrics. Contrast specific samples for their performance.
Version everything
Weave tracks versions of your prompts, datasets, and model configurations. When something breaks, you can see exactly what changed. When something works, you can reproduce it. Learn about versioningExperiment with prompts and models
Bring your API keys and quickly test prompts and compare responses from various commercial models using the Playground. Experiment in the Weave PlaygroundCollect feedback
Capture human feedback, annotations, and corrections from production use. Use this data to build better test cases and improve your application. Collect feedbackMonitor production
Score production traffic with the same scorers you use in evaluation. Set up guardrails to catch issues before they reach users. Set up guardrails and monitorsGet started using Weave
Weave provides SDKs for Python and TypeScript. Both SDKs support tracing, evaluation, datasets, and the core Weave features. Some advanced features like class-based Models and Scorers are currently not available for the Weave TypeScript SDK. To get started using Weave:- Create a Weights & Biases account at https://wandb.ai/site and get your API key from https://wandb.ai/authorize
- Install Weave:
- In your script, import Weave and initialize a project.
[YOUR-TEAM] with your W&B team name and [YOUR-PROJECT] with your W&B project name.