Web agency measurement errors, from the missing denominator to the copied benchmark

Web design agencies report on websites, campaigns and their own results, and buyers use those numbers when choosing an agency from a directory like this one. The seven errors below appear in agency proposals, case studies and client reports. Each comes with an example from agency work.

Mistake 1: A percentage without an eligible population

Publishing a rate before defining the eligible population is among the most common errors. A completion percentage can look exact while leaving readers unable to tell who qualified or what counted. Define the event, eligible group, numerator, denominator, unit and reporting period before calculating or comparing the result. A case study that claims an enquiry uplift should say what counted as an enquiry (a form, a call click or a booking), which visitors were eligible and which weeks were compared.

Capture the timezone, exclusions, query version, evidence owner and uncertainty with the result. If a correction is needed, preserve a traceable record of the earlier value and the revision. This is basic quality assurance: readers need the source record and its limits, rather than an unexplained replacement number.

Mistake 2: One score for evidence that answers different questions

Treating delivery acceptance, accessibility, technical reliability, incidents, user outcomes and commercial value as interchangeable creates a misleading all-purpose score. They answer different questions, so a composite label can hide which evidence needs attention. Separate the categories and identify the one relevant to the decision at hand. An agency might merge launch-day acceptance, an accessibility audit result, page speed and sales into one project success score, which tells a buyer nothing about which part went wrong.

For a non-transactional website, digital analytics may not show whether a person can finish a task. Direct task research can add user evidence; audience research and a quality checklist support different checks. Keep accessibility requirements and accessibility standards distinct from a commercial outcome rate, even when all appear in one report.

Mistake 3: Measures chosen before anyone names the decision

A dashboard can fill up with activity measures that do not help anyone choose what to do next. Name the task, decision owner and response before selecting measures. Keep a measure only when its result could alter the next action, and state what decision the evidence is meant to support. For a ten-page brochure site, the decision might be whether the monthly care plan is worth keeping, so track form completions and calls, not page views.

The GDS performance framework connects measurement with purpose, hypotheses and available sources. Treat it as a prompt, not a ready-made private-sector template. The agency strategy framework guide can help order the decision, but the owner still needs to explain what evidence supports a choice and what response follows.

Mistake 4: Giving allocated credit the language of cause

Last-touch or weighted allocation describes how credit was distributed under chosen settings. In Google Analytics 4 (GA4), for example, the attribution model and lookback window are settings that change which channel receives credit for the same enquiry. It does not show what would have happened without the change. Calling allocated credit a cause claims more than the method establishes. Label the result as allocation, and choose an evaluation approach that fits the decision. A report that credits a redesign with every enquiry from organic search is allocating credit, not proving the redesign caused it.

Name the attribution rule and settings, then keep the contribution claim within those limits. The attribution methods guide helps compare approaches, but naming a model does not prove cause. Where the method cannot support a causal conclusion, report credit as allocated rather than caused.

Mistake 5: Treating an imported benchmark as a target

A benchmark is not a target just because it carries the same label as your measure. Geography, service, cohort, period or denominator may differ, leaving the underlying records incomparable. Check provenance and fit before drawing a comparison, and do not present an imported threshold as a private-sector norm. For example, an agency might copy a published online-shop conversion rate as the target for a ten-page service site. The two sites sell different things to different visitors, so the target would be meaningless.

Government benchmarking guidance for major projects concerns government projects; its comparator discipline illustrates why scope and data quality need checking. Use the benchmark research protocol to inspect definitions, source and denominator. Compare records only when their scope and eligible population match. Any benchmark rate should name its publisher, year, definition and population; if it cannot, do not use it.

Mistake 6: Trusting a chart while its feed is failing

A steady-looking chart can conceal a failed feed, missing coverage or a changed definition. Treating its appearance as proof of completeness hides material limits. Report extraction status, coverage gaps, correction history and the effect of any definition change alongside the measured value. A dashboard that shows flat traffic after a template update may simply have lost its GA4 tag. Compare it with Google Search Console clicks and impressions for the same pages and dates before trusting either.

Record the source, method and uncertainty so readers can judge statistical quality. The ONS quality and methodology information guidance is a reference for presenting methodological detail. Distinguish observed values, revisions and gaps, and explain when a definition change affects how a result should be read.

Mistake 7: Gathering personal detail without a decision need

Collecting personal detail simply because a system can capture it creates exposure without showing that the detail improves a decision. Limit collection to its stated purpose, then check privacy, security, access controls and data-protection duties separately. Convenient capture alone is not a reason to retain granular paths. Session recordings and form-field tracking on a client site can capture names and email addresses, so collect only what the redesign decision needs.

In the UK, the UK GDPR and the Data Protection Act 2018 apply to personal data, regulated by the Information Commissioner's Office (ICO). The Privacy and Electronic Communications Regulations (PECR) add consent rules for analytics cookies and similar tools. Review the collection purpose and privacy information before gathering data. Public competitor information is not a reason to collect more personal detail. Keep competitor research within the stated need, and check whether each field contributes to the decision rather than being retained by default.

Reporting checklist