Why the Office for National Statistics changing its pricing methodology is so important

After years of trials, the Office for National Statistics last week announced a major upgrade in how it measures the grocery prices that feed into official inflation figures.

Starting this month, it is replacing 25,000 monthly price points, previously checked in store by price collectors, with 300 million price points from over a billion products sold. The data will be fed directly from supermarket tills and represents half of total grocery sales.

The new dataset will feed into the consumer price indices CPI and CPIH, and the retail prices index (RPI) from March.

Mike Hardie, ONS deputy director for prices transformation, calls it a “step-change in our measurement of inflation”.

So, why is it so significant? And how might the change ripple into wider official inflation figures and the economy?

For a start, the vastly greater number of data points means less reliance on extrapolation. Until now, ONS field collectors have gathered prices once a month (twice for volatile categories) from 141 UK locations. Only 153 food and non-alcoholic drinks items were in the official inflation basket. Assumptions had to be made. For example, carrots were taken to “represent a range of vegetables in the traditional collection”, according to ONS, Using the new ‘scanner data’, “we also include prices for parsnips, beetroot, ginger, radishes, celeriac, horseradish, swedes and fennel”.

“We include nearly all products within the scope of the category, rather than a targeted sample of those products judged to be most popular, as in the traditional collection.”

The data will also be more robust on individual product types. “Rather than assuming the change in the price of one type of apple in a supermarket reflects all apples, for the shops supplying us with scanner data, we will be able to see how the price of every apple changes, as well as knowing exactly how much of each type of apple is bought, so can adjust the figures for changing buying habits,” says Hardie.

Loyalty pricing

Perhaps even more significant is the inclusion for the first time of loyalty pricing and multibuy promotions.

“This means the price charged at the till, not the price shown on the shelf, will feed into our inflation statistics,” Hardie says.

Clive Black, head of research at Shore Capital Stockbrokers, says it’s about time. “It makes eminent sense to remove the at times farcical ONS-derived baskets and use third-party suppliers where more accurate and relevant till data is to hand.

“What folks actually pay is more important than theoretical prices, and loyalty programmes are clearly an important component of modern-day shopping which should be included.

“The breadth of till scanning should make for a fuller, more representative and therefore accurate, measure of actual food prices, whether on promotion or loyalty scheme or not.”

Since Tesco launched Clubcard Prices in September 2020 loyalty pricing has become endemic: today Sainsbury’s, Morrisons, Co-op, Waitrose, Lidl, Boots and Holland & Barrett all now offer discounts to members, and some 97% of shoppers are signed up to at least one supermarket loyalty scheme.

Evidence collected by the CMA in its 2024 investigation of loyalty pricing suggested they could save 17%-25% buying loyalty priced products at the five supermarkets examined: Tesco, Sainsbury’s, Waitrose, Co-op and Morrisons.

Like Black, Retail Economics head of commercial content Nicholas Found believes that “for too long, nuances have been lost in a traditional sampling approach which often failed to capture the reality of modern shopping behaviours”.

The Grocer’s own weekly price comparison survey, the Grocer 33, was updated to include loyalty price promotions two years ago. With most Tesco and Sainsbury’s shoppers being loyalty members, the effect on a read of inflation is dramatic. This week’s basket, for instance, costs 2.9% more on average year on year without loyalty promotions. But that drops to 2.5% when loyalty price cuts are factored in, and further still to 0.9% when multibuy deals, including for loyalty members, are pro-rated. (Digital subscribers can toggle between ‘loyalty display’ and ‘display’ pricing to include or exclude loyalty prices.)

Found says including loyalty and promotions is also useful for retailers, who “will now see the impact of their pricing strategies reflected more accurately in national inflation statistics”.

Some extrapolation is still needed. With only half the groceries market covered by scanner data, the other 50% will use the old method. “I’d view it as an enhancement to the official inflation picture, but not a substitute for richer commercial insight,” Found cautions.

Nonetheless, the impact is expected to be borne out in headline official inflation figures. The ONS estimates the CPIH and RPI would have been 0.02 percentage points lower over five years using its scanner data. And that’s an average – at times the discrepancy would have been greater. In April, the ONS thought using scanner data in 2023 would have reduced RPI by 0.11 percentage points. It now proffers a more cautious 0.07 point reduction.

A spokeswoman says: “Since the April 2025 analysis was published, we have been carrying out a parallel run of our scanner data systems. The intention was to further quality assure our estimates before going live. As a result of the parallel run, we have made further improvements to the way the data is ­processed, which has resulted in this change.”

The discrepancy for food and non-alcoholic drinks, which make up around 12%-15% of the overall inflation measures, will of course be bigger, but this figure has not been published.

But this is not a pedants’ corner for stats nerds. Around £619bn (24.5%) of UK government debt is index‑linked, meaning interest payments rise and fall with RPI. In June 2025, a 1.7% rise in RPI between March and April led to £10.9bn in additional interest costs. Anything that lowers RPI is thus politically and fiscally significant.

Other measures of retail inflation are available. They have included loyalty pricing and bigger datasets for a while, and they consistently produce lower figures for food and drink inflation. The ONS measured food and non-alcoholic beverage inflation at 4.5% in the year to December 2025, and alcohol and tobacco at 5.2%. That fed into an overall CPI figure of 3.4%. Worldpanel, which includes loyalty pricing, had grocery inflation at 4.3% in the four-week period ending 28 December.

New Worldpanel data this week showed grocery inflation easing to 4% in the four weeks to 25 January, its lowest level in nine months. That being so, some argue the greatest damage of ignoring realities of how consumers shop has been done.

But others will say it’s not too late to stop confusing fennel, parsnips and radishes for carrots.