Measurement
Measuring a web design agency for a UK buyer: metric dictionary, evidence layers, legal duties and a worked example
How a UK buyer can measure a web design agency: two worked dictionary entries, six evidence layers, five contract reports and the laws and registers to check.
Measure an agency engagement by starting with the buyer decision, not a dashboard template. Define what could happen, what can enter the count, how the value is calculated and what action follows. If the source cannot answer the question, report the gap instead of turning a proxy into a result.
What to take away
- Start with the buyer decision and the chain of assumptions, not a dashboard.
- Give every measure a dictionary entry with formula, population, source, owner and action.
- Keep delivery, accessibility, reliability, security, user-task and commercial evidence in separate layers.
- Leave unknown fields blank. Do not fill them with a proxy.
- Treat rule-based credit and causal effect as different questions.
Write one metric completely
Take a buyer who must decide whether the information journey on an agency-built website is ready for wider release. The metric is task completion: the share of eligible attempts that reach the declared end state without help from a facilitator.
The entry below uses illustrative figures only. They are not an England benchmark.
| Field | Completed entry (illustrative) |
|---|---|
| Purpose | Decide if the revised journey is ready for wider release |
| Event | Participant reaches the confirmation screen of the agency-built journey without help |
| Eligible population | 120 attempts on one task, one build, one recruitment frame |
| Formula | 78 completions / 120 attempts = 0.650 |
| Unit | 65.0 per cent, reported with the counts 78 of 120 |
| Time | A named test month, UK time |
| Boundaries | Withdrawals and technical failures excluded, reasons recorded |
| Quality | Small sample, no confidence interval claimed |
| Action | Release holds below a buyer-set threshold or while a critical accessibility finding is open |
Suppose the buyer sets the threshold at 70 per cent for this example. The 65.0 per cent result then holds the release. The buyer chooses a real threshold from user needs and risk, never from an agency proposal.
A second entry covers reliability. The figures are illustrative arithmetic, not a benchmark.
| Field | Completed entry (illustrative) |
|---|---|
| Purpose | Decide whether hosting and support meet the contract |
| Event | A minute in which the live site passes the monitoring check |
| Eligible population | 43,200 minutes in a 30-day month |
| Formula | (43,200 minus 90 failed minutes) / 43,200 = 0.9979 |
| Unit | 99.79 per cent |
| Boundaries | Agreed maintenance windows excluded and listed |
| Action | Service review if below the contract level |
Begin with the decision and the theory
A plan names the service problem, the users, the commissioned work, the decision owner and the strategy framework options, including repair or no change. Then it writes the chain of assumptions from the work to the expected outcome and marks which links can be observed.
Keep these questions apart, because one figure cannot answer them all:
- Did the agency deliver the agreed item?
- Did that version pass its acceptance checks?
- What accessibility findings appeared in a defined test?
- Did the live service work when users needed it?
- Did a security incident or personal-data breach occur?
- Could eligible users finish the declared task?
- What value and cost were recorded, and who gets the credit?
A finished milestone says nothing about accessible use. Revenue after launch is not automatically revenue caused by the redesign.
Keep a mandatory metric dictionary
Every measure needs the same minimum fields. If a field is unknown, leave it blank. If it does not apply, write the reason.
Field / Required entry
- Purpose
- The decision and who uses the result
- Event
- The observable condition that is counted
- Eligible population
- People, attempts or time units that could count
- Formula
- Numerator and denominator
- Unit
- Count, percentage, elapsed time or currency
- Time
- Period, timezone, clock start and stop
- Boundaries
- Inclusions, exclusions and duplicate rules
- Provenance
- Source system, extraction route, query version
- Accountability
- Data owner, analytical owner, decision owner
- Action
- Threshold, response, stop condition, next review
Do not copy a threshold from an agency proposal, a case study or a public dashboard until you have shown that its population and definition fit your case. The usual measurement mistakes start there.
Ask the agency for five reports
A buyer cannot measure what the agency never records. Put these reports in the contract, each tied to a dictionary entry:
- An acceptance logevery deliverable, its version, the check applied and the decision.
- An accessibility test report for each template, with method, tool, findings and retest.
- A service log for the live sitedowntime, incidents and monitoring gaps.
- A task-completion test on the main journeys, with the full counts.
- A cost ledger that matches the quote, with changes approved in writing.
Separate the evidence layers
Delivery evidence links scope, version, due item and the acceptance decision. Show rejected, conditional and undecided items, not just finished tasks.
For accessibility, record the tested journey, method, tool, finding, owner and retest. The GOV.UK Service Manual tells public service teams to meet WCAG 2.2 level AA as a minimum, and it adds that testing only with automated tools will miss some issues. A private buyer has its own duties but can borrow the method.
Reliability needs a defined service event and eligible time. State planned downtime, monitoring gaps and dependencies. A technical alert is not a legal breach decision, and a risk scenario should say who decides which is which.
Measure user tasks across the whole journey, not clicks on one page. Say whether the unit is an attempt, a person or a session. Commercial figures need finance-owned definitions covering currency, VAT, period and refunds, with gross value and net contribution kept separate.
Name the legal duties and registers to check
These instruments apply across the UK, so a buyer in England uses the same ones. Each is a gate for the dictionary, not a score.
| Check | Rule or register | What to confirm |
|---|---|---|
| Accessibility, public sector buyers | Public Sector Bodies (Websites and Mobile Applications) (No. 2) Accessibility Regulations 2018, in force for public sector bodies from 23 September 2018 | The build meets WCAG 2.2 AA unless a legal reason applies |
| Equality | Equality Act 2010 | The Act protects people from discrimination in the workplace and wider society, so service terms follow it |
| Personal data | UK GDPR Article 5, regulated by the ICO | The seven principles are met, and the data protection fee is paid unless exempt |
| Security | Cyber Essentials | Standard technical controls are in place against common online threats |
| Agency identity | Companies House | Registered address and date of incorporation match the contract |
Govern data before you read the chart
Write a data inventory: purpose, event, fields, people, systems, recipients, retention and access. Analytics is not neutral because it is called analytics.
Collect only what the decision needs. The ICO's personal data breaches guide states that a reportable breach must go to the regulator within 72 hours of becoming aware of it, where feasible. Each breach needs a record, whether or not it is reported.
Under the data minimisation principle you should identify the minimum amount of personal data needed for your purpose and hold no more. Treat consent loss, ad blockers, deletion and offline events as boundaries of the measure. A cleaner funnel may simply be missing the people who never entered it.
Show source health on the dashboard
Build panels around decision owners, not one engagement score. Show these beside every value:
- the definition and query version;
- the extraction time and expected refresh;
- coverage, missing data and the reconciliation result;
- any break in the series;
- the uncertainty or limitation;
- the threshold owner and response.
When a feed fails, freeze the panel. Do not carry the last figure forward as if it were fresh.
Separate credit from cause
Rule-based attribution methods show where recorded credit goes under a declared rule. Last touch gives all credit to the final recorded interaction. A weighted rule shares it by set weights. Neither one tells you what would have happened without the work.
HM Treasury's Magenta Book is guidance on designing an evaluation, written for government. Its ideas help a buyer ask causal questions, but it is not a scoring system for agencies. Stop causal claims when the comparison or baseline fails, and report what moved, with its uncertainty.
Benchmark only comparable definitions
A benchmark needs a clear source set and the same definitions of service, geography, buyer type, period, unit and exclusions. Keep public services, private firms and other UK nations in separate rows. Review the benchmark research method before using a figure.
If comparable records exist, pool events, not percentages:
pooled share = sum of eligible numerators / sum of eligible denominators
Never average percentages whose denominators mean different things.
This dictionary uses no external benchmark, so compare results with your own earlier tests.
Quality-assure the reporting chain
Trace a sample from source event to dashboard and back. Recalculate the formula independently, reconcile totals and test the duplicate rules. The quality checklist lists gates to use.
Pause reporting when a denominator cannot be rebuilt, a source changed without a bridge or a major segment disappeared. Correct past outputs visibly. Do not overwrite history.
Before you act
- Name the decision, its owner and the alternatives.
- Write the assumption chain from the work to the outcome.
- Complete every dictionary field or leave it blank.
- Name the data, analytical and decision owners.
- Separate operational alerts from legal breach decisions.
- Reconcile commercial figures under finance definitions.
Common questions
Why can one figure not answer every measurement question?
Delivery, accessibility, reliability, security, task completion, value, credit and cause each need their own definition, population, owner and rule. A single score hides which one failed, so keep them in separate layers.
What should a metric dictionary record?
Purpose, event, population, formula, unit, time rules, boundaries, source, owners and action. Unknown fields stay blank, and a field marked not applicable needs a reason.
When should reporting stop?
Stop when the denominator cannot be rebuilt, a source changed without a bridge, a privacy question is unresolved or nobody can explain the response to a result.
In this guide
- Web design agencies attribution methods for buyersWeb design agencies attribution methods help buyers assess directory listings, separate credit from causal evidence and ask agencies for reproducible reports.
- Web agency measurement errors, from the missing denominator to the copied benchmarkAvoid seven web agency measurement errors through a dated, non-ranked method covering definitions, denominators, privacy, causality and benchmarks.
- Web design agencies benchmark research: a method protocolUse web design agencies benchmark research with strict comparator rules, metric specifications and rejection criteria before calculating any figure.
