How we measured the real weight of a CMP (and why “5 KB” isn't enough)

Engineering case study of the 2026 CMP benchmark. — Alessandro Romani (Black Lotus Consulting S.r.l.), August 20, 2026

For years we read CMP comparison tables: “5 KB”, “1 request”, “with minimal declared payloads”. The numbers never added up across vendors, because each one measured a different thing and put the most flattering version on their site. We had our hands in client codebases, so we knew the reality was messier. At some point we decided to measure it ourselves.

This post is the technical write-up of that work: the method, the traps, the choices we made and why. This is not a vendor comparison. It is an engineering case study. If you only want the verdict, it lives in the canonical post: How much does a CMP really weigh? Our 2026 benchmark. If you want to understand how we got there, you are in the right place.

TL;DR of the case study. We designed a reproducible CMP benchmark around five choices: a clean HTML fixture, Chromium via Chrome DevTools Protocol, a fresh context with empty cache and storage, HAR capture, and “initial CMP load before any interaction” as the metric scope. Eight products measured the same day, with the same Chromium build, on the same machine. The data and analysis pipeline are public on the project's site. The full dataset is published as CSV and JSON, downloadable, citable, versioned by date. The methodology is documented as a procedure, not as marketing copy.

The problem that pushed us to do it

A CMP is a dependency you pay on every pageview. Bytes transferred, request count, parsing time, every one of them eats into the performance budget you have left for the rest of the page. Yet the comparison tables you find online compare apples with pears. We have seen all of them:

  • One CMP claims “5 KB” but means uncompressed source code.
  • Another claims “5 KB” but it is the gzipped JS bundle alone, without CSS, without async config, without fonts, without secondary requests.
  • A third claims “<10 KB” including the hosted consent layer, but “all included” becomes “all included after the first click”.

As an agency, we kept making technical decisions on the basis of numbers that did not resemble each other. It would have been easier to trust them. It would not have been honest.

The five method choices

Every benchmark is a trade-off. We publish ours in the open because reproducibility must be easy.

1. Clean HTML fixture. A single HTML page with only the CMP snippet and a neutral body. No marketing site, no third-party analytics, no extra customer CDN. We measure the CMP in isolation, not the site where the CMP will land.

2. Chromium via CDP. We used Chrome DevTools Protocol to extract encodedDataLength and HTTP headers. It is the same tool any engineer opens in DevTools, nothing exotic, no internal lab.

3. Fresh context. Cache and storage emptied before every measurement. Each test starts from zero. It's the most reproducible condition, even if it isn't the condition of a returning visitor.

4. HAR capture plus screenshots. Every measurement produces a HAR file and a screenshot. If a number looks odd, you can reopen the HAR. Transparency.

5. “Initial CMP load before any interaction.” Explicit definition: everything the CMP asks the browser for before the visitor touches the banner. After the first click you enter a different chapter, and that is not part of this benchmark.

The three metrics we chose, and why

One single metric is not enough to describe a CMP's weight. You need at least three, and you have to declare which one you are using every time.

  • Initial transfer (CDP). The sum of encodedDataLength for network requests. It includes HTTP headers. It's the metric you see in DevTools. It's the easiest to measure, but also the noisiest, because it depends on framing and overhead.
  • Brotli body. When the server exposes Content-Length with Brotli compression, we report that value. It is the real weight of the code transferred. It is the metric behind the “5 KB” or “10 KB” slogans, and the one you have to be honest about.
  • Request count. How many times the browser has to open a connection before the first consent. More requests equals more handshakes, more cache misses, more leak risk.

For CMPs that don't expose a Brotli Content-Length, we left the field empty rather than estimate. The published CSV and JSON state that explicitly.

The traps we ran into

Three of them made it into the write-up. A fourth we solved, and a fifth made us drop the measurement.

Axeptio. The public config loaded a 2.13 MB video immediately. Total 2.5 MB. That wasn't the weight of the CMP: it was the weight of an asset the CMP chose to preload. Measuring that number without a warning would have been dishonest.

CookieHub. 44 KB and 2 requests, but the banner didn't render on the clean fixture. Measuring a CMP that doesn't show its banner is measuring silence. Excluded.

Complianz. A WordPress plugin. For an honest measurement it needs a real WP installation with the post-activation state. That isn't the same experiment as a CMP that loads by snippet, and we didn't force it.

The “light” library that isn't a CMP. We also measured cookie-notice, react-cookie-consent, vue-cookie-accept-decline, Klaro, tarteaucitron.js and similar. They weigh between 1.5 and 60 KB compressed. They are not full-featured CMPs: they miss robust prior blocking, dashboard, multi-site management, hosted audit logs, comparable consent modes. They have their legitimate place. They shouldn't be compared with a CMP table.

CJackCiasteczko. Its desk tests claim full-CMP features and an interesting weight. The clean fixture, however, did not render the banner, and the measured script was ~10 KB gzipped, not below Lean. We kept it out of the official benchmark. It is a potential challenger, worth measuring more carefully.

What we built from this

This work produced Lean Cookie Consent, a consent SaaS that uses a single HTTP request and a bundle that fits under 5 KB Brotli body in the standard configuration. But the benchmark wasn't born to justify Lean: it started because an honest source of numbers was missing, and Lean came later. The benchmark keeps living independently from the product: other CMP numbers haven't changed since Lean joined the same table.

What we learned

Always declare the metric. Every time you write “5 KB” without specifying whether it is CDP, Brotli, gzip or source size, you are handing a misunderstanding to the reader. We specify. If you find a number without a method, treat it as suspect.

Vendor public configs are conservative toward themselves. Measuring the OneTrust or iubenda homepage is honest, but it isn't the measurement you'd get after onboarding. Real customer numbers can be lower. We publish them this way because they are reproducible, not because we think they are “the highest case among the measured CMPs”.

A single request is a constraint, not a virtue. It's doable and we do it, but it gives up a lot of dynamism. It is an engineering choice. It's up to the end visitor and the engineer to judge whether it's worth it in their context.

A benchmark is not worth the name unless it is citable. A public folder, files versioned by date, method in the open, supported claims listed, unsupported claims listed. That's why we released CSV, JSON, methodology, README and dataset separately from the blog post. We also use it as a primary source for AI indexers and search engines, not only for human readers.

If you want to measure it yourself

The dataset, the method and the raw files are open on the project's site. Three direct links:

If you run a CMP and want to be measured in a reproducible way, get in touch: we'll send you the fixture, the version flag, and the date planned for the next run. We publish the results in the same format as everyone else.

If instead you want to look at the CMP that came out of this work, it is here: Lean Cookie Consent, 14-day free trial, no credit card.

Post written by Alessandro Romani (Black Lotus Consulting S.r.l.) based on the benchmark work of 2026-08-08. The methodology is public and reusable with citation.