Browser Agent Observability (Part 1): Action Trace

Why Observability Is a Hard Requirement

Your agent ran 50 steps in production — navigation, form filling, search, extraction. Then it errored. Or worse: no error, but wrong results.

Where did it go wrong?

Traditional logging fails here. HTTP logs show requests were sent and responses received, but not what the agent was looking at. Application logs show which function was called, but not why.

AI browser agent observability needs far more:

What each step did — the agent's decision
Page state at execution — what the agent saw
Reasoning behind decisions — why the agent chose that action
Action results — what changed on the page

Without this, debugging AI browser automation is finding a needle in a dark room with a flashlight.

Minimum Step Log

class StepLog:
    def __init__(self, step_number, action, reasoning, page_state):
        self.step_number = step_number
        self.action = action
        self.reasoning = reasoning
        self.timestamp = datetime.now()
        self.page_state = page_state
        self.result = None
        self.error = None
        self.duration_ms = 0

Three Recording Levels and Their Costs

Level	Per-Step Size	1000-Step Task	Best For
Action log only	~100 bytes	~100 KB	Low-cost monitoring, high-volume
+ DOM snapshot	~10 KB	~10 MB	Debugging common errors
+ Full screenshot	~500 KB	~500 MB	Complex scenario replay

Implementation

class AgentTracer:
    async def record_step(self, agent, action_result):
        step = {
            "step": len(self.steps) + 1,
            "timestamp": datetime.now().isoformat(),
            "action": action_result["action"],
            "reasoning": action_result.get("reasoning", ""),
            "url": agent.page.url,
            "title": await agent.page.title(),
            "dom_snapshot": await self.get_dom_snapshot(agent),
        }
        self.steps.append(step)
 
    async def get_dom_snapshot(self, agent):
        return await agent.page.evaluate("""
            () => {
                const els = document.querySelectorAll(
                    'button, a, input, select, textarea, [role="button"]'
                );
                return Array.from(els).slice(0, 50).map(el => ({
                    tag: el.tagName,
                    text: el.textContent?.trim().slice(0, 100),
                    visible: el.offsetParent !== null
                }));
            }
        """)

Replay

The trace should support:

Step forward/backward — see the page state at each step
Full LLM reasoning — why the agent chose that action
Expected vs actual results — did the action actually take effect?

class TraceReplay:
    def find_deviations(self):
        """Steps where click didn't change the page"""
        deviations = []
        for i, step in enumerate(self.steps):
            if step.get("action_type") == "click":
                if i + 1 < len(self.steps):
                    next_step = self.steps[i + 1]
                    if (step["url"] == next_step["url"] and
                        step["dom_snapshot"] == next_step["dom_snapshot"]):
                        deviations.append(i)
        return deviations

find_deviations() catches the classic Validator hallucination — the agent thinks it clicked something, but nothing actually happened.

Anomaly Detection

class AnomalyDetector:
    def detect_loops(self):
        actions = [(s["url"], s["action"].get("selector"))
                   for s in self.trace.steps]
        for i in range(len(actions) - 3):
            if actions[i] == actions[i+1] == actions[i+2]:
                return {"loop_detected": True, "start_step": i}
        return {"loop_detected": False}

Summary

An AI browser agent without action trace data is driving blind. Minimum observability requirements:

Per-step recording — action, reasoning, page state
Replayable — backtrack through each step's context on failure
Automated detection — loops, cost anomalies, ineffective actions from trace analysis

AI Browser Agent Observability (Part 1): The Action Trace