What it catches, what it misses, and what it costs.

Headline

Lab, exact ground truth: 46 scenarios and 594 attempts on macOS, 43 and 567 on Linux. The baselines are mitmproxy 12.2.3 (12.3.0 in the Linux image), run with the same proxy and CA variables whatleaks sets, and HTTP Toolkit server 1.27.3, driven the way its desktop app drives it: its "existing terminal" setup, which sets proxy and CA variables plus its own Node, Python, Ruby and Java hooks.

Payload: read with a byte-identical body. Body cut: request seen, body missing or cut off. Host only: not decrypted, but the destination named. Best possible: the share of attempts at the best outcome whatleaks's rules allow (see how the lab works).

• Every miss left is a program whatleaks loads no hook into. On the 43 scenarios the old build also ran, whatleaks missed 0 of 564 attempts on macOS. All 20 misses on macOS and all 17 on Linux are in rows added to measure that limit: Go programs that bypass the proxy. No other tool here names those either.
• whatleaks reached the best possible outcome in every scenario where another tool did. In no scenario did a baseline read a payload whatleaks only named. The one row where a baseline went past it is the 9 MiB upload: mitmproxy kept the whole body, and whatleaks caps buffering at 8 MiB by design.
• No false attributions. In every run, nothing the app did not send was credited to it.
• Identity detection: all 8 planted identifiers were found in every run, and all 3 planted in a WebSocket message. None of the 4 decoys was flagged.
• Cost: the lowest median latency of the three proxies on keep-alive requests and on new TLS connections. Large uploads were roughly even with mitmproxy. It uses more CPU than mitmproxy, mostly a fixed second per run.

How the lab works

A lab server runs on the machine's LAN address with its own CA: HTTP/1.1, WebSocket, HTTP/2, raw TCP and UDP. It is reached as an sslip.io name, so clients make real non-loopback connections that use DNS.

Each scenario is a real client: Node 20, 22 and 26, Bun, curl, Python (urllib, requests, httpx, aiohttp, Apple's build), Ruby, Go or Java. The client writes every attempt to a ledger before making it. That ledger is the ground truth. The lab also logs what actually arrived. Each tool runs every scenario through its real interface, and its output is scored per attempt:

"Best possible" is the best outcome whatleaks can reach without breaking its own rules: payload for clients that use the proxy or that its hooks route to it; host for clients that refuse the CA or ignore the proxy, because whatleaks never touches the system trust store or proxy settings and never gets around pinning; request for the 9 MiB upload, which is over the buffering cap by design. Every column is scored against the same expectation, including the old build's.

Baselines. mitmproxy gets the same environment variables whatleaks sets, its own CA and the lab CA upstream, with default settings. HTTP Toolkit's official server release runs with a private config folder; each scenario gets a fresh session and its existing-terminal variables. Both are stopped 10 s after the last client exits, as a person would stop them. Runs per scenario: 3 for proxy-aware clients and 10 for timing-dependent ones, each row with a 95% Wilson interval in the result files.

Every scenario (macOS)

Share of attempts at the best possible outcome. Linux differences are below.

Where whatleaks still misses

20 attempts on macOS, 17 on Linux. Only in programs whatleaks loads no hook into (Go here; also Rust, C and other native code), and only when their connection bypasses the proxy:

• A TCP connection reset within about 10 ms: Go was caught 10% and 30% of the time in two macOS runs, 20% and 30% in two Linux runs. The polling watcher must see it while it's open, and a reset leaves no closed-socket record behind.
• UDP sent with sendto() on an unconnected socket: 0%. The socket never has a remote address for any watcher to read.

The same connections from Node, Python or Ruby are reported by whatleaks's runtime hooks as they open: 100% in the lab. A Go connection the server closes at once is usually still named (90% and 100% in two macOS runs, 100% on Linux), because the client keeps its socket a moment longer.

Seeing every connection from every program would take a kernel-level hook (eBPF, a network extension). whatleaks avoids that, because it needs neither root nor installation. The lab's server is on the same machine, so these connections close within a few milliseconds; over the internet a TLS request takes several round trips, and the "reset after 50 ms" rows are 100% on both platforms.

Where the baselines miss

• mitmproxy sees only clients that read proxy variables and trust its CA. It misses Node 20, app-made Node agents, Java, Apple's Python and the system Ruby (refused, and not named), aiohttp, node:http2, and every direct connection.
• HTTP Toolkit's hooks cover more clients than mitmproxy: Node 20, app-made agents, Java, Apple's Python, the system Ruby and aiohttp. With v1.27.3 on Node 26, the built-in fetch and WebSocket went around it; that covers the Node 26 rows, the 9 MiB upload, the helpers and the identity rows, which all use fetch. Node 22's fetch was caught. It has no watcher for direct connections.

Real tools

Each tool ran four ways: with its debug mode, which prints events and sends nothing (the ground truth); twice with defaults; once with DO_NOT_TRACK=1; and once with its documented opt-out. Then it ran under mitmproxy and under HTTP Toolkit, and while phonehome, a per-process connection watcher, watched. These runs sent each tool's real telemetry.

• Turborepo. The 7 missing events never leave the process: turbo exits about 20 ms after starting its last request. mitmproxy connects upstream before it answers the client's CONNECT, which takes longer than turbo waits, so it sees no turbo event at all. whatleaks answers at once. HTTP Toolkit also caught the 10.
• Astro. DEBUG=astro:telemetry printed nothing on 7.4.1: the debug hook is installed by a logger module that astro build does not load. The ground truth there is the agreement between whatleaks and mitmproxy: 1 event. whatleaks also flagged that the event carries a hash of the project folder path.
• DO_NOT_TRACK. The results match each tool's documentation, as checked from its source: Next.js and Astro document no DO_NOT_TRACK support and still send with it set; Turborepo and Wrangler honour it.
• phonehome samples connections once a second. It named the telemetry destination for 3 of 5 tools and missed the short-lived ones. It never labelled one as telemetry and cannot show contents.

Cost

One Node https client against the lab, run directly, under whatleaks, under mitmproxy and under HTTP Toolkit, at load average 2.0.

• Upload throughput varies from run to run. Over three runs mitmproxy measured 160 to 239 MiB/s and whatleaks 187 and 211. Treat uploads as roughly even.
• CPU: whatleaks's figure is its total minus the bare client's. About 1 s of it is a fixed cost per run: loading mockttp, creating a CA, and the watcher's process listings during the 1 s wait after the app exits.
• Memory: its RSS is the largest process it waited for, so it can include the app. It keeps every body it buffers (up to 8 MiB) for analysis.
• Startup and exit, one-request app, median of 10: bare 92 ms, under whatleaks 1964 ms. 1 s of that is the grace period that catches late senders (--grace); the rest is loading mockttp (about 0.5 s), a fresh CA (about 0.1 s) and the loopback-only safety probe.

Linux

The same lab runs inside Docker (a Debian image with the runtimes) against the same build. Bun, Apple's Python and the system Ruby are macOS-only rows. The rows where Linux differs from macOS:

On Linux a closed socket has no owner in /proc, so the polling watcher names a direct connection only if it sees it open. The Node rows are reported by the preload: 100%.

No tools installed. A check in node:22-slim, which has no ps, lsof, netstat or curl, against real hosts:

The benchmark also found and fixed 20 gaps along the way, from Java ignoring proxy variables to helpers sending 1.5 s after exit. Each is listed with its commit in the method document.

Threats to validity

• It's our benchmark. The whatleaks author built the lab and chose the scenarios. Every script and result file is public, so it can be rerun and challenged.
• One machine. A working desktop with a VPN, Wi-Fi that dropped several times, and three LAN address changes during the runs. The runner waits out outages and reruns any scenario that overlapped one. Perf numbers carry the load average they ran under.
• Linux runs in a VM. Docker on macOS runs a linuxkit VM, not bare-metal Linux. The /proc watcher and the runtimes are real Linux; timing may differ.
• Same-machine lab. Round trips take microseconds, which makes direct connections shorter than over the internet: a hard case for the polling watcher, not an easy one.
• Small samples. 3 or 10 runs per scenario. Rows between 0 and 100% have wide intervals, given in the result files.
• Baselines run on defaults. An expert could tune mitmproxy or HTTP Toolkit. For mitmproxy, the one setting that plausibly matters here, connection_strategy=lazy, was measured and did not change the turbo result. HTTP Toolkit was driven by its terminal script, not its desktop app's launchers for specific apps.
• Real tools change. The versions are pinned above; newer releases may behave differently.
• Debug output is not always ground truth. Debug modes show what a tool intends to send: Turborepo's shows 17 events and 10 cross the wire, and Astro's printed nothing.

Reproduce

$ npm ci && npm run build
$ ./bench/setup.sh        # Go client, Python venv, mitmproxy, HTTP Toolkit server (all under /tmp/wlb)
$ npx tsx bench/run-lab.ts --tag after --tools whatleaks,mitmproxy,httptoolkit   # about 2 h; --resume continues
$ npx tsx bench/summarize.ts
$ bench/linux/lab.sh --tag linux --tools whatleaks,mitmproxy,httptoolkit
$ npx tsx bench/run-perf.ts --tag after
$ npx tsx bench/run-real.ts --tag after   # sends real telemetry; needs the scratch apps in bench/real/tools.ts
$ docker run --rm -v "$PWD":/w:ro -w /tmp node:22-slim node /w/bench/linux/check.mjs

The provenance of every result file is in its meta: build, date, machine, load and reruns. Both lab platforms and the cost runs used commit 22d85d3; the real-tool runs used 7ee2ac6, and the later changes don't affect traffic that goes through the proxy.