We build and operate China delivery infrastructure, so we needed our own measurements rather than someone else’s. Standing up a headless browser in Shanghai took an afternoon. Getting numbers we were willing to publish took considerably longer — because most of what a performance measurement assumes about networks stops being true at the Chinese border, and not in the obvious way. Everyone expects higher latency. Almost nobody expects the failure arithmetic underneath their tooling to quietly stop working.
This is the list of what went wrong. We are publishing the failures rather than the fixes, for a reason we will come back to at the end.
1. Cancelled requests are not failed requests
What happened. Our resource accounting counted every request the browser did not complete as a failure. One of the first sites we measured returned a full page in 0.4 seconds — and a resource loss rate of 100%.
Why it was wrong. net::ERR_ABORTED means the browser cancelled the request: a navigation replaced the page, a preload went unused, a redirect fired. It is not a network event at all. Neither is ERR_BLOCKED_BY_ORB, which is a browser security policy. Lumping them in with real network failures inflates every loss figure you publish.
Why it is easy to miss. Nothing errors out. You get a number, the number is plausible for China, and confirmation bias does the rest. We only caught it because 0.4 seconds and 100% loss cannot both be true.
Which direction it errs. Against us — this one made the internet look worse than it is. It is the only one of the eight that did.
2. The wall mostly drops packets. It does not refuse them.
What happened. After fixing (1), our count of unreachable third-party hosts read zero — while the same page load showed the embedded video and the form’s verification widget both failing.
Why it was wrong. A blocked request frequently produces no event whatsoever. No connection reset, no timeout error, no loadingFailed. The request is simply issued and never answered. Anything that counts error events — including the failure counters in browser devtools — will report that nothing was blocked.
Why it is easy to miss. This is the single most important difference between measuring China and measuring anywhere else, and it is invisible if you have only ever measured networks that fail loudly. In an ordinary network, a host that cannot serve you says so.
Which direction it errs. In our favour — it makes the unoptimised baseline look far healthier than it is.
3. Fixing (1) is what made (2) visible
These are not eight independent items on a checklist. Correcting the error classification is what surfaced the silent-failure problem, because until the noise from cancelled requests was removed, the zero was hiding in plain sight.
This pattern repeated. Several of the errors below only became visible after an earlier one was fixed, and one of them actively conceals another. That is the actual shape of this work: not a list of gotchas you can work through in an afternoon, but a set of interacting failure modes where the order you find them in is mostly luck.
4. A deleted directory that kept serving
What happened. A redeploy produced a beautiful result: two of our three delivery paths suddenly completed loading in about 200 milliseconds, while the third still timed out.
Why it was wrong. They were serving a 153-byte error page. The deploy script removed and recreated a directory that a running container had mounted; the container kept pointing at the deleted original and served nothing. The 200-millisecond “page load” was an error document.
Why it is easy to miss. The measurement completed successfully. Status codes were fine at the layer we were checking. It produced exactly the result we were hoping for, which is precisely when you should be most suspicious. We caught it because a page containing an embedded video cannot load in 200 milliseconds, anywhere, ever.
Which direction it errs. In our favour, dramatically. This one would have been a published lie.
5. IP geolocation is not where the datacenter is
What happened. Three independent IP geolocation services agreed our probe target was in Beijing. The cloud provider’s own instance metadata said it was in Zhangjiakou — a different city, roughly 200km away, in a different province.
Why it matters. Every figure you publish carries its measurement location. “Measured from Beijing” is a factual claim, and if a reader checks it against the authoritative source and it is wrong, everything else you wrote is now suspect too.
Which direction it errs. Neutral, but fatal to credibility.
6. Where you measure from decides who wins
What happened. We compared a nearshore delivery path against an in-country one. Measured from a neutral third-party network, the in-country path was 1.19× faster on time-to-first-byte. Measured from a probe sitting on the same cloud provider as the in-country target, the same comparison came out at 2.82×.
Why it was wrong. Traffic between two machines on one provider’s network rides that provider’s internal backbone and never touches the public Chinese internet that actual users traverse. Both endpoints of our comparison happened to be on the same provider, so a probe on that provider handed one side a home-field advantage and simultaneously penalised the other.
Why it is easy to miss. Nothing about the probe looks wrong. It is in the right country, on a real network, returning stable numbers. The distortion lives entirely in the routing.
Which direction it errs. In our favour — it more than doubled our own headline differentiator.
7. “Reachable” and “fast” are different questions
What happened. We tested whether various third-party hosts were reachable from inside China by requesting them and checking the status code. Several came back in about 100 milliseconds and we recorded them as available.
Why it was wrong. Those were tiny responses. When we requested a real asset from one of the same hosts, it arrived at 14.9 KB/s — a throughput figure roughly three orders of magnitude below what the reachability check implied. The host answers. It just does not deliver.
Why it is easy to miss. Every reachability tool, including ours, reports a binary. The binary was correct. It was also useless.
Which direction it errs. In our favour — the unoptimised path looks functional when it is not.
8. A single measurement is not a measurement
What happened. We measured the same asset over the same path twice, ten minutes apart. First sample: 449ms to first byte, 5.5 MB/s. Second sample: 3,135ms to first byte, 43 KB/s. Same file, same route, same probe, same afternoon.
Why it was wrong. Cross-border throughput in China varies by more than two orders of magnitude across the day. Practitioners will tell you the evening is when things degrade — reachable intermittently, then slow. Any single-sample number is an artefact of the moment it was taken, and reporting a daily average erases exactly the pattern that matters most to anyone operating a site there.
Which direction it errs. Whichever direction you happened to sample in. That is the problem.
What this list is actually about
Seven of these eight errors made our own delivery product look better than it is. We fixed them because a measurement that flatters you is worth less than no measurement at all — the first procurement team that asks “measured how?” will find what we found, and everything else we have published goes with it.
But there is a second reason to publish the list rather than the fixes.
None of these are exotic. Every one is the kind of thing you find by running the same measurement enough times, in enough conditions, that the contradictions start showing up: a page that loads in 0.4 seconds and loses 100% of its resources; zero blocked hosts on a page whose video will not play; a 200-millisecond load of a page containing an embedded video. The errors announce themselves only if you already know what the numbers should look like.
That knowledge is not a technique. It is the residue of having done it wrong repeatedly, which is why we can describe the failure modes precisely and why a first attempt will still hit most of them in a different order.
If you are measuring your own site from inside China, the single most useful thing on this page is item 2: check whether your tooling counts requests that were never answered at all, as distinct from requests that failed. Most tooling does not, and the difference between those two numbers is usually the entire story.
What we are measuring next
A 24-hour sampling run across five third-party resource categories, direct against delivered-through-our-network, broken down by hour of China local time rather than averaged — specifically to characterise the evening degradation described in item 8. After that, the same reference site measured from a Chinese office broadband line rather than a datacenter, because a datacenter’s international transit is materially better than what a real user has, and every figure we have published so far is therefore a floor rather than a worst case.
Both results will be published here, including the parts that do not favour us.
If you want to see what your own site does from inside mainland China right now, our China firewall test and China speed test both run from probes inside the mainland. And if you are wondering whether being served from inside China is itself the fix, we measured that too: it is not.
