The Semantic Layer That Took Anthropic From 21% to 95% Accuracy

Anthropic's own data science team saw AI analytics accuracy jump from 21% to 95%, and the fix was not a smarter model. It was a semantic layer, a translation layer that sits between your data warehouse tables and anything that reads from them. The single biggest lever was defining each metric once for the whole organization, so no team and no AI agent has to guess what a number means.

What A Semantic Layer Actually Is

A semantic layer is a translation layer between raw warehouse tables and everything that queries them. The frames in the video show the architecture directly: source tables and models on the left, a "Semantic Layer" box in the middle, and consumers on the right, including dashboards, APIs, and AI agents.

The point is that none of those consumers talk to the raw tables directly. They go through the semantic layer, which holds the canonical definitions of metrics and entities. A dashboard asking for "revenue" and an AI agent asking for "revenue" both resolve to the exact same definition instead of two separate interpretations of two separate SQL queries.

Define A Metric Once, Everywhere

The core mechanic is defining a metric one time across the entire org. Without a semantic layer, every team and every AI agent interprets a metric on its own. One analyst's "active user" is another team's "monthly active user," and the numbers drift.

Defining the metric once in the semantic layer produces three effects, which the video lays out as a bullet list on screen:

Benefit	What it means
Shared definition	Everyone querying the data works from the same definition of each metric
Single point of change	When a definition changes, you update it in one place and every consumer inherits the update
No guessing for agents	An AI agent references a single source of truth instead of inventing its own interpretation

The third point is the one the video calls out as most important. An AI agent that does not have to guess a metric definition is an agent that stops producing plausible-but-wrong answers.

Why This Moves AI Accuracy So Much

The 21% to 95% figure comes from Anthropic's internal work running analytics on Claude. The reported approach was blunt: their team gave the model access to thousands of SQL files, dashboards, transforms, and notebooks, then wrote a rule into the model's skills that before guessing an answer or writing any SQL, it checks the semantic layer first.

That reframes the accuracy problem. Text-to-SQL failures often are not reasoning failures. They are definition failures, where the model writes syntactically correct SQL against the wrong interpretation of a metric. Removing the guesswork by forcing every data question through a single canonical source is what closed most of the gap. The takeaway for anyone building analytics agents is that governance and definitions can outrank raw model capability.

One caveat worth keeping in view: 95% is a strong jump from 21%, but a 5% error rate is still meaningful for business-critical decisions. The semantic layer is the foundation, not the finish line. Anthropic's production setup paired it with validation and monitoring on top.

Key Takeaways

• A semantic layer sits between warehouse tables and consumers (dashboards, APIs, AI agents) and holds canonical metric definitions.
• Defining each metric once gives you a shared definition, a single point of change, and no guessing for AI agents.
• Anthropic's internal analytics accuracy went from 21% to 95% by forcing the model to check the semantic layer before writing SQL.
• Many AI analytics errors are definition failures, not reasoning failures, which is why the semantic layer helps so much.
• 95% accuracy is a foundation, not a stopping point, for decisions that carry real business risk.

Resources

• How Anthropic's AI Accuracy Went from 21% to 95% -- AtScale blog post by Dave Mariani, shown on screen in the video, walking through Anthropic's semantic-layer result.
• Anthropic Reports Claude Now Handles 95% of Internal Analytics Queries -- InfoQ coverage of Anthropic's internal self-service analytics and the accuracy figures.