Obscura Rust Library — Browser/Page/Element Complete API
Embedded Rust library API: Browser/Page/Element/CookieStore, interception channel, event callbacks, build notes. Git dependency integration.
Engineering Blog
Category: Technical Guide
99 posts in this category.
2026 Proxy IP Quality Review: 16Yun Products vs Industry Platforms
16Yun tunnel proxy, API proxy, dedicated proxy, scheduled proxy, overseas residential proxy vs 3 industry platforms — 72-hour benchmark data.
Obscura Request Interception — Block APIs, Mock Responses, Inject Scripts
Three interception layers: CDP, Rust, and network. Block ads, mock API responses, inject preload scripts — control every request your page makes.
Obscura Production Deployment — Docker, Systemd, Workers, and Tuning
Docker distroless image, Systemd service unit, multi-process worker scaling, V8 heap tuning, WebSocket reverse proxy — Obscura production deployment.
Obscura MCP Server — Give Your AI Agent a Browser
30+ MCP tools from navigation to form filling to data extraction. Configure Claude Desktop and your AI agent can drive a browser for any web task.
Obscura CDP Compatibility — Zero-Change Migration for Puppeteer and Playwright
Core CDP domains, direct Puppeteer/Playwright connections, full lifecycle mapping — switching from Headless Chrome to Obscura means changing one line of connection code.
Obscura CLI Guide — Seven Output Formats and Concurrent Scraping
Seven output formats, concurrent workers, JS injection, SPA wait strategies — one Obscura CLI command equals a full headless browser.
What is Obscura — An Open-Source Headless Browser Engine
30 MB memory, 85 ms page load, built-in anti-detection — Obscura is the Rust alternative to Headless Chrome for AI agents and scrapers.
AI Agent Anti-Blocking in Practice: Proxy Configuration and Engine-Level Evasion
When AI agents move from lab to production, anti-blocking isn't optional — it's survival. This article covers proxy selection, GeoIP alignment, IP rotation strategies, fingerprint consistency, error troubleshooting, and a production deployment checklist.
2026 AI Browser Agent Comparison: Token Efficiency, Anti-Detection, and Architecture Selection
As of mid-2026, AI browser agents have diverged into four distinct architectural paradigms. This article delivers a comprehensive comparison across six dimensions, with a decision tree for choosing the right tool.
Lightpanda Snapshot Crashes, Nanobrowser Hallucinates Sent Emails, Steel Fingerprint Retries All Fail — 7 Bugs That Broke My Production Pipeline
Every tool in this space looks great in a demo. Running them in production is where you find the gap between marketing claims and engineering maturity. Seven bugs I actually hit, with root causes and workarounds for each.
AWS Agentic Form Filling: Episodic Memory and Semantic Element Discovery on Bedrock
The AWS airline check-in AI Agent demonstrates enterprise cognitive orchestration: Bedrock AgentCore manages episodic memory, Playwright handles browser interaction, Sentence Transformers power semantic element discovery.
Node.js HTTP Client Tunnel Proxy: axios / got / http Comparison
axios, got, and native http module — three approaches to tunnel proxy integration.
Python mitmproxy Tunnel Proxy: Forward to Crawler Proxy via Upstream Mode
mitmproxy upstream mode + addon script for Crawler Proxy integration.
Agent-E: DOM Distillation and Hierarchical Orchestration for Reliable Web Navigation
Agent-E doesn't just control a browser — it understands pages through DOM distillation (filtering content by task type), mmid injection (custom attribute element targeting), and hierarchical orchestration (batch operations via execution planner).
Golang Tunnel Proxy: net/http and resty Four IP Control Scenarios
Go net/http and resty four tunnel scenarios with native HTTPS Proxy-Tunnel support.
Camoufox Anti-Detect Browser: C++-Level Fingerprint Spoofing Beyond JS Shims
When JS shims are increasingly detectable by modern WAFs, Camoufox takes a more radical approach — modifying Firefox source code at the C++ engine level, completing all fingerprint spoofing before JavaScript executes.
Python Selenium Chrome Tunnel Proxy: Browser Automation with Crawler Proxy
Selenium Chrome + Crawler Proxy: Chrome extension-based proxy auth integration.
Node.js Puppeteer Tunnel Proxy: Chrome Automation with Crawler Proxy
Puppeteer + Crawler Proxy: Chrome automation proxy integration.
Lightpanda Engine: A Headless Browser Built from Scratch in Zig, 10× Faster, 10× Less Memory
Lightpanda is not a Chrome fork or patch — it's a headless browser engine written from scratch in Zig. No GUI rendering pipeline, optimized JS engine, CDP protocol compatible. 10× faster, 10× less memory.
Node.js Playwright Tunnel Proxy: Browser Automation with Crawler Proxy
Playwright + Crawler Proxy: browser automation proxy integration.
Agent-Browser Under the Hood: Rust Daemon, A11y Refs, and Token Optimization
agent-browser replaces Node.js cold starts with a native Rust daemon, replaces brittle CSS selectors with A11y snapshot + stable element ref IDs, compressing DOM from 5000 tokens to 200-400.
Python Scrapy Tunnel Proxy: Four IP Control Scenarios
Scrapy middleware configuration for four proxy scenarios, including HTTP Proxy-Tunnel and HTTPS limitation notes.
Python aiohttp Tunnel Proxy: Async Four IP Control Scenarios
aiohttp async four scenarios with proxy_headers for HTTPS Proxy-Tunnel support.
Steel Browser Deep Dive: Enterprise-Grade Cloud Browser API for AI Agents
Steel Browser is an open-source cloud browser API with built-in session management, proxy rotation, stealth configuration, and CAPTCHA solving. Compatible with Puppeteer, Playwright, and Selenium.
Python httpx Tunnel Proxy: Four IP Control Scenarios with Async HTTP
httpx four tunnel proxy scenarios with httpx.Proxy for HTTPS Proxy-Tunnel support.
Browy in Action: Zero-Cost AI Browser Agents via GitHub Copilot
If you already have a GitHub Copilot subscription, Browy makes every browser automation task cost zero in inference. Side panel for daily work, DevTools CLI for power users, 41 built-in tools.
Python requests Tunnel Proxy: Four IP Control Scenarios for Crawler Proxy
requests full four-scenario implementation with custom HTTPAdapter for HTTPS Proxy-Tunnel.
Nanobrowser 101: Multi-Agent Web Automation Without the $200/mo Subscription
A free alternative to OpenAI Operator. Install the Chrome extension, add your API key, control your browser with natural language. Built-in Planner/Navigator/Validator multi-agent system.
Python urllib Tunnel Proxy: Standard Library Four Scenarios
Python standard library urllib implements four tunnel proxy ...
agent-browser + 16Yun Proxies: Enterprise AI Scraping Deployment
Session isolation, security policies, proxy selection, and production deployment checklist for agent-browser.
agent-browser Network Control: Request Interception & Page Diagnostics
network route to intercept requests, network har to record performance, react tree for component inspection, vitals for metrics.
Python feapder & ScrapySplash Tunnel Proxy: Framework Extensions
feapder AirSpider and Scrapy+Splash integrating 16Yun Crawle...
agent-browser Batch Automation & AI Chat
batch for multi-step execution in one call, chat for natural language browser control, concurrent sessions.
Python pyppeteer & Selenium Firefox Tunnel Proxy: Browser Automation
pyppeteer and Selenium Firefox integrating 16Yun Crawler Pro...
Playwright to AI Agent Migration (Part 2): Dual-Mode API Design and Gradual Rollout
No downtime needed. Same API, can route to either scripts or AI. Config switch control, comparison validation, gradual traffic increase.
PHP Tunnel Proxy: curl / Guzzle / Stream Three Implementations
PHP curl, Guzzle, and Stream Context integrating 16Yun Crawl...
agent-browser Session Management: Auth Persistence & State Reuse
--profile to reuse Chrome login state, --session-name for auto persistence, state save/load for explicit management.
Playwright to AI Agent Migration (Part 1): The Assessment Matrix
Stable paths stay as scripts. Unstable paths move to AI. But how do you define 'stable'? Three dimensions: operation stability, change frequency, exception cost.
C++ Tunnel Proxy: libcurl and CPR Two Implementations
C++ libcurl and CPR libraries integrating 16Yun Crawler Prox...
agent-browser Snapshot & Screenshots: Make Your AI Understand the Page
snapshot -i for interactive elements only, screenshot --annotate with numbered labels, diff for change detection.
agent-browser CI/CD Testing: Version Locking, Headless Differences, and Flaky Tests
One Chrome auto-update, CI turns all red. Headless A11y element order differs from headed. Two parallel tests hijack each other's sessions.
C# Tunnel Proxy: HttpClient and HttpWebRequest
C# HttpClient and HttpWebRequest integrating 16Yun Crawler P...
agent-browser 101: AI-First Browser Automation CLI
npm install -g agent-browser, one command to open a browser, snapshot to understand page structure, click/fill to interact.
AI Browser Automation Cost Analysis (Part 2): Proxy and Compute Optimization
Tokens first, proxy second, compute third. Proxy optimization doesn't save much money but reduces latency. Compute optimization matters at scale.
Trafilatura + CloakBrowser: The Complete Full-Text Scraping Pipeline
CloakBrowser renders JS/SPA pages → Trafilatura extracts clean text. Solve the 'JS-rendered content can't be extracted' problem.
Rust Tunnel Proxy: reqwest and isahc
Rust reqwest and isahc HTTP clients integrating 16Yun Crawle...
AI Browser Automation Cost Analysis (Part 1): Tokens, Proxy, and Compute
One extraction task: tokens $0.05, proxy traffic $0.001, server $0.002. Tokens dominate. Proxy and compute are usually too cheap to optimize.
Ruby Tunnel Proxy: Faraday and HTTParty
Ruby Faraday and HTTParty integrating 16Yun Crawler Proxy....
Trafilatura Advanced: Custom Extraction, Language Detection & Performance
Advanced Trafilatura: custom element exclusion, language detection, offline batch processing, and incremental updates.
Chrome Cluster on K8s: Memory Config, /dev/shm, and Graceful Shutdown
Browsers aren't stateless microservices. 200-500MB per instance, /dev/shm config, WebSocket long connections — running browser clusters on K8s needs a completely different design.
Swift Tunnel Proxy: Alamofire and URLSession
Swift Alamofire and URLSession integrating 16Yun Crawler Pro...
Batch Scraping: Trafilatura + 16Yun Proxies in Production
From single-page extraction to million-scale batch pipelines: concurrency control, proxy rotation, error handling, and storage.
Nanobrowser Security (Part 2): Behavior Sandbox and Permission Control
Input filtering isn't perfect. When attackers bypass it, the behavior sandbox is the last line of defense — limiting what the agent can do when compromised.
Perl LWP Tunnel Proxy
Perl LWP::UserAgent integrating 16Yun Crawler Proxy....
Precision Extraction: Content, Metadata, and Tables — Trafilatura Deep Dive
Deep dive into Trafilatura's extraction engine with benchmark data, metadata fields, and tuning strategies.
Nanobrowser Security (Part 1): Prompt Injection Context Isolation and Input Filtering
A single hidden white-text line can make your agent execute attacker instructions. Prompt injection isn't XSS or SQL injection — it lives at the AI reasoning layer.
Smart Crawling: Discover and Scrape Full Websites with Trafilatura
Sitemap discovery → Feed tracking → URL management → bulk extraction — a complete full-site scraping workflow.
Node.js Advanced Tunnel Proxy: undici / superagent / https
Node.js undici, superagent, and native https module integrat...
2026 — Enterprise Proxy Service Stability Assurance
Enterprise proxy stability assurance: SLA design, fault classification, response mechanisms, and continuous improvement.
AI Browser Agent Observability (Part 2): Monitoring and Alerting — HTTP 200 Doesn't Mean Success
The dashboard was all green — 99.9% HTTP 200 rate, 200ms P95 latency, zero 5xx errors. But the agent had looped on the same page 10 times, extracting nothing. Traditional monitoring couldn't see this.
Node.js Legacy Tunnel Proxy: phantomjs / casperjs / request
Deprecated but still widely deployed Node.js tools integrati...
Trafilatura 101: From HTML to Structured Text in 3 Lines
pip install and 3 lines of code to extract article text, title, author, and publication date from any URL.
AI Browser Agent Observability (Part 1): The Action Trace
Agent ran 50 steps in the browser, errored somewhere in the middle, and you only see the final error. No action trace, no DOM replay, no decision chain.
Proxy IP + Fingerprint Browser: The Ultimate Anti-Ban Combo
Fingerprints solve who you are. Proxies solve where you are. Both must work together to bypass modern anti-bot systems.
Playwright + AI Agent Hybrid Architecture (Part 3): When AI Fails — Degradation, Rollout, and Rollback
LLM can't fill a form in 15 minutes. Selector failure rate exceeds threshold. Token consumption is 3x over budget. AI failure isn't 'if' — it's 'when.'
Self-Hosted Fingerprint Browser Cluster: Open-Source Multilogin Alternative
One Docker command to deploy a fingerprint browser cluster — an open-source, self-hosted Multilogin alternative.
Playwright + AI Agent Hybrid Architecture (Part 2): A Bridge Layer for Scripts and AI
Not 'Task A uses scripts, Task B uses AI.' Within the same task: steps 1-3 use scripts, step 4 errors out, AI takes over transparently, hands back after recovery.
Human Behavior Simulation: Bypass Behavioral Detection with humanize=True
One flag to enable human-like mouse, keyboard, and scroll patterns that bypass behavioral detection.
Playwright + AI Agent Hybrid Architecture (Part 1): Where's the Line Between Scripts and AI
Calling page.goto via AI costs 20x more than a script. Login via AI is 3-5x slower. Not every browser operation needs a language model.
Fingerprint Management & Session Persistence: Make Your Scraper Look Like a Returning Visitor
Fixed fingerprint seeds, persistent cookies/sessions, and incognito bypass for maximum trust scores.
Chrome Browser Multi-Instance Isolation (Part 3): State Sync, Locking, and Migration
Agent A recorded its operation state. Agent B doesn't know. Agent A's session needs to migrate to another node, but cookies and localStorage are still on the original browser.
Nanobrowser Source Code: The Multi-Agent Scraping Loop — How Executor Drives Planner + Navigator
The Executor's core is a for loop: Planner runs every N steps to guide direction, Navigator executes one operation per step. Planner says done → stop. Navigator says done → Planner validates. Consecutive failures hit the limit → abandon.
CloakBrowser Hands-On: Bypass Cloudflare Turnstile in 30 Seconds
Install, run, and bypass Cloudflare Turnstile with CloakBrowser in 3 lines of code.
Chrome Browser Multi-Instance Isolation (Part 2): Profiles, Containers, and CDP Targets Compared
Profile isolation works but is fiddly. Containerization is thorough but expensive. CDP Target is lightweight but fragile. Three approaches compared by cost and suitability.
Nanobrowser Source Code: Clickable Element Detection and Hash Deduplication
getClickableElements walks the DOM tree to find interactive elements. hashDomElement uses three-layer hashing (branch path + attributes + XPath) for unique fingerprints. A 30-line iterative stack replaces recursive DOM traversal.
Nanobrowser Source Code: Markdown Extraction and Readability — Two Data Pipelines for Scraping
getMarkdownContent uses turndown to convert DOM to Markdown. getReadabilityContent uses Mozilla Readability to extract article body. Both are under 40 lines in service.ts but the design tradeoffs matter for scraper developers.
Fingerprint Browser 101: Why Your Scraper Keeps Getting Blocked by Bot Detection
Understand browser fingerprinting and anti-detection, and see why C++-level patching is fundamentally different.
Chrome Browser Multi-Instance Isolation (Part 1): Session Hijack, Config Conflicts, Memory Contamination
Two agents running simultaneously. One is filling a checkout form. The other takes over its tab. This isn't hypothetical — agent-browser Issue #326 describes exactly this scenario.
Browser Anti-Detection (Part 3): Why Consistency Beats Aggressive Rotation
An IP looks reasonable. A browser profile looks normal. Code runs without errors. But put the three together and they don't align. Detection systems find inconsistency.
Browser Anti-Detection (Part 2): GPU Pipelines, Keystroke Entropy, and Client Hints
navigator.webdriver was defeated in 2018. 2026 anti-detection looks at mouse movement entropy, keystroke dwell time variance, sub-pixel Canvas rendering offsets.
Browser Anti-Detection (Part 1): TLS Handshake and HTTP/2 Frames — You're Identified Before JS Runs
Camoufox's C++ engine-level spoofing covers the JS layer, but it can't touch the transport layer. The TLS handshake identifies you as script or human before the first HTTP request.
Chrome Headless at Scale (Part 3): Why K8s HPA Doesn't Work for Browser Clusters
CPU-based HPA is nearly useless for browser instances. Startup CPU spikes don't mean it's serving. Memory doesn't release after close(). Browsers aren't stateless services.
Chrome Headless at Scale (Part 2): CDP Connection Pool Exhaustion and Tab Accumulation
Tab 1 loads fine. Tabs 2 and 3 both time out. 120 seconds isn't enough. 1,600,000ms isn't enough either. The problem isn't page load speed — it's the connection pool.
Chrome Headless at Scale (Part 1): Orphan Processes and a 24GB Memory Leak
Unkillable Chrome processes, accumulating zombies, 24GB virtual memory leaks over 21 hours — scaling isn't just running 1,000 copies of the same instance.
Issue Playbook: Crawler logs are cut by day in Proxy Routing and
Single-problem, engineering-grade Scrapy tutorial.
Issue Playbook: Missing the Scrapy entry in Wikipedia in m in Proxy
Single-problem, engineering-grade Scrapy tutorial.
Issue Playbook: S3FilesStore can use a lot of memory in Proxy Routing
Single-problem, engineering-grade Scrapy tutorial.
Issue Playbook: Scrapy needs to have sane defaults that do in Proxy
Single-problem, engineering-grade Scrapy tutorial.
Issue Playbook: Translating the docs in Proxy Routing and Health Scoring
Single-problem, engineering-grade Scrapy tutorial.
2025 — AI's New Demands on Proxy Stability
AI data collection demands far more from proxy stability: high concurrency, extreme success rates, and consistent low latency.
2024 — Proactive Monitoring Drives Proxy Stability
Proactive monitoring vs reactive reporting: how observability systems took proxy stability to the next level in 2024.
2023 — How API Scheduling Affects Proxy Stability
API scheduling impact on proxy stability: why P95 response time matters more than averages.
2022 — Proxy Type Stability Tiers
Residential, data center, static, dynamic — different proxy types have different stability characteristics. Choose the type that fits your scenario.
2021 — Global Proxy Nodes: Balancing Latency and Stability
Global node deployment in 2021: how to balance latency and stability in cross-border proxy scenarios.
2020 — Single Points of Failure Are the Biggest Enemy of Stability
Intelligent load balancing and auto-failover in 2020: why single points of failure matter and how to design for high availability.
2019 — From 95% to 99% Availability in Production
Going from 95% to 99% availability meant more than a 4% improvement. It required architectural redundancy, automated failover, and proactive monitoring.
HTTP vs SOCKS5 — How Protocol Choice Affects Proxy Stability in 2018
HTTP and SOCKS5 proxies behave differently under real-world conditions. In 2018 we studied both protocols extensively to understand their stability characteristics.
Building a Stable HTTP Proxy Service in 2017
The early proxy market in 2017: what connectivity rate means, why proxies fail, and why we chose the harder path.