Agent Observability Is the New Ops Layer: Four Tools and Five Things to Log

If you cannot see what your agent is doing in real time, you do not actually know what it is doing, and that is the whole problem. In part 24 of his AI Agents for Businesses series, Doby Lanete argues that agent observability is the new ops layer and names four tools that handle it well: Langfuse, Galileo, Braintrust, and Phoenix. He then lists the five things worth logging on every action so a black box becomes something you can debug and improve. The video is a straight talking-head format with website overlays for each tool and a numbered on-screen list for the logging checklist.

Why Observability Becomes the Ops Layer

The framing in the video is blunt. An agent you cannot inspect is "a pile of shit," in the creator's words, because you are flying blind on what it decided and why. Traditional apps have logs, metrics, and traces. Agents add tool calls, model calls, and branching decisions on top of that, so the surface area you need to watch is larger. The claim that observability is "the new ops layer" tracks with where the market is moving. Cisco announced its intent to acquire Galileo in April 2026, and the tool's own site now reads "now part of Cisco," which is visible in one of the frames. When infrastructure vendors buy agent observability companies, it signals the category has moved from nice-to-have to operational baseline.

The Four Tools

The video presents four options, each with a one-line positioning statement.

Langfuse is the open-source pick. The creator calls it easy to self-host and good for startups. It is MIT-licensed, and the self-hosted build is the same codebase as the cloud version, so teams can run it inside a VPC or on-premises when data cannot leave their infrastructure.

Galileo is positioned for production-scale agents with strong hallucination detection. Its homepage, shown on screen, reads "Don't just monitor AI failures. Stop them," and its hallucination workflow is built around prebuilt metrics like correctness and context adherence plus runtime guardrails.

Braintrust is described as having a strong evaluation framework and being popular with engineering teams. It connects development, testing, and deployment, lets teams run evals in code or in the UI, and gates releases through CI/CD. Note that the website overlay in the video briefly shows a talent-marketplace page rather than the eval platform at braintrust.dev, so treat the on-screen visual with a grain of salt and go to the correct site.

Phoenix is the second open-source option, from Arize. The creator highlights its tracing. Phoenix uses OpenTelemetry-based instrumentation and ships tracing, evaluation, dataset management, and a prompt playground, with a local-first mode so you can debug without shipping data to a SaaS backend. One frame shows an unrelated "smart cameras" Phoenix page, another overlay mismatch, so verify you are looking at phoenix.arize.com.

The Five Things to Log

The on-screen list pairs with the narration to give a concrete checklist. Log every tool call and what came back. Log every model call with token count and cost. Log every decision branch so you can see why the agent acted versus escalated. Log the outcome. And log cost per task. The video's argument is that these five turn a black box into something you can manage, improve, and defend. Without them you are guessing. With them you can debug a bad run, attribute cost, and build an agent that is safe enough to actually put in front of a business.

Pick One Before It Goes Live

The practical instruction is to pick one tool, wire it up, and do it before the agent goes into production. The four options are not meant to be layered together. Two are open source (Langfuse, Phoenix) and two lean commercial and production-scale (Galileo, Braintrust), so the choice tends to fall out of whether you want to self-host and how much eval and guardrail tooling you need on day one.

Key Takeaways

Agent observability is treated as the new ops layer, not an optional add-on. Cisco acquiring Galileo is one market signal that backs this framing.
Four named tools: Langfuse (open source, self-host, startups), Galileo (production scale, hallucination detection), Braintrust (evaluation framework, engineering teams), Phoenix from Arize (open source, strong tracing).
Log five things on every action: every tool call, every model call, every decision branch, the outcome, and cost per task.
Logging token count and cost per model call and per task is what makes cost attribution possible.
Logging decision branches is what lets you see why an agent acted versus escalated.
Pick one tool and wire it up before the agent goes live, rather than bolting observability on after failures appear.
Two of the on-screen website overlays (Braintrust and Phoenix) show the wrong company, so confirm the correct product sites before evaluating.