The battle for brains: how supermarkets are seeking the best in data talent

Every card payment, app swipe, loyalty point, scanned voucher, unopened marketing email and e-commerce site click adds a data point to supermarkets’ huge stores. These so-called data lakes are only growing, as the companies draw in data from new sources like meal pics posted on Instagram, the movements of shoppers using self-scanners through a store, and angry tweets about the brand. There’s also data to be drawn from supply chain, logistics, IT, payroll and finance.

The real value of the data is in the insights it can provide. And the importance of those insights to the supermarkets - and the predictions they enable - can’t be overstated.

Sainsbury’s put “data and insights” at the very centre of its business model in this year’s annual report. “Our ambition is to know our customers better than anyone else so we can anticipate and meet their needs,” says Helen Hunter, Sainsbury’s chief data and analytics officer.

M&S is “placing data front and centre of our decision-making across the business” the company says.

For Asda “data is at the heart of everything we do”. Waitrose, likewise, is “putting analytics at the heart of everything we do”.

“We see a big opportunity ahead of us to better capitalise and generate actionable insights on the data we create as a business,” says Jeremy Pee, M&S’s first chief digital and data officer, who was appointed in December last year. “Being able to understand and translate this into actions that could benefit the business is critical to our transformation.”

Making sense of the oceans of data points is difficult. Turning rows of numbers into action is even trickier. To do so you need both data-savvy staff and specialist data scientists, who can use the latest tools and, preferably, understand the sector.

Such candidates are in short supply. Now, having pinned their strategies on their ability to utilise data, the supermarkets are scrambling to secure the best ­talent and even create their own. Having the best data talent is, according to Tesco’s data science director Robert Armstrong, “without doubt a competitive advantage”.

Rising demand

Data scientist was famously dubbed by the Harvard Business Review as “the sexiest job of the 21st century”.

Someone working in the role will typically have a PhD, know the relevant technical languages and have used tools like Python, R, SQL, pandas, matplotlib and sklearn. They work closely with engineers to put machine learning into products and business processes.

At Sainsbury’s, for example, they “choose and tune the right algorithms” which help “colleagues focus on our customers and support our customers in doing their shop,” says Hunter. At Tesco they “take on and make a difference on the most intellectually challenging and vital problems,” says Armstrong.

Unfortunately for the supermarkets, grocery is not the only sector seeking them.

The number of UK job adverts calling specifically for data scientists and advanced analysts has increased by 231% over the past five years, to 27,033, according to a Royal Society-commissioned analysis published in May. Vacancies for data analysts numbered 114,327, up 43% since 2013. The figures probably “underestimate” the true demand, the report said.

“There’s not enough people trained to work with the tools, and if you can find them, they don’t know how to run a supermarket”

An MHR Analytics-commissioned survey of UK business leaders in 2018 found more than three quarters were planning to hire a data specialist this year. A Deloitte skills study from last year found data scientists and analysts to be “the most difficult roles to recruit and retain”.

“The reality of recruiting is that the talent just doesn’t exist,” says Dan Zell, managing partner of data skills education provider Decoded, which works with M&S and the Netherlands’ largest supermarket chain, Albert Heijn.

“There’s just not enough people out there who are already trained to work with the tools, and if you can find them, then they didn’t know how to run a supermarket,” Zell adds.

As well as trying to attract data scientists, supermarkets are also seeking data analysts.

“There’s data scientists and engineers doing the really big, meaty, knotty problems, and then there are data analysts who are leveraging the tools that are already out there and applying them to the data that they have at their disposal,” explains Zell.

“There’s a shortage across all of those roles, that’s the thing,” he adds.

The shortage is beginning to bite. A July Forrester survey of global grocery retail executives commissioned by Dunnhumby found more than a quarter reported a lack of data skills within their companies was holding them back.

“Firms also lack reporting tools and technology, but even with these, they struggle to create actionable insights that deliver clear recommendations, due to a lack of in-house data science and analytics skills,” the report states.

Most wanted

A typical supermarket data scientist job posting based on active vacancies

About the job: Solve complex business problems at pace using a variety of data science techniques; tell engaging data stories to stakeholders and fine-tune algorithms in a commercially relevant way

Qualifications: A quantitative advanced degree, preferably a PhD in statistics, maths, science, computer science or engineering

Technical knowledge: Proficient with Python and R programming languages; tools like pandas, matplotlib, sklearn; and software including SQL Server, Hadoop and Teradata

Experience: Tuning algorithms and quantifying performance; delivering projects with Agile or Kanban methodologies; design of experiments; familiarity with software engineering techniques and cloud computing

Personality: Organised, pragmatic and a lateral thinker with an inquisitive mind. Willing to challenge the status quo and be a team player

Salary: £52,000-£60,000 [UK average, indeed.co.uk analysis]

Amazonians

Meanwhile, the threat of online, data-first businesses like Amazon and Ocado looms large.

“Grocers are saying - what the hell are Amazon doing and how can we begin to compete? Amazon can do what they do because of the quality of their data and predictive analytics. They’re resetting the benchmark,” Zell says.

In response to the skills squeeze, the mults have been redoubling their recruitment efforts to attract data talent.

In November, Tesco’s recruitment team pitched a stall at technology job fair Silicon Milkroundabout, hoping to convince attending data scientists to join the company’s ranks. In recent years it has forged partnerships with universities, sponsoring PhDs and inviting students to work with Tesco’s data team on technical challenges.

“We believe it is important to build partnerships with academia to further the development of new technical approaches as well as generate talent for the future. For Tesco, this means we will have great people supporting the evolution of our business and we will continue to meet the changing needs of customers,” says Armstrong.

The supermarket is also seeking candidates overseas, building data teams in Poland and India.

“We have learnt it’s important to look across the global market and identify talent hotspots to invest in,” Armstrong adds.

In September, Sainsbury’s launched its ‘Digital, Tech, Data’ recruitment initiative, with Hunter hitting the stage at several technology events to tout the opportunity to work on “the UK’s pre-eminent data set”. It has been running ads across social media for the data team with the tagline ‘Curiosity. Meet big business’.

“Finding data scientists who are commercial and pragmatic can prove challenging, but Sainsbury’s is a multi-brand, multichannel business serving 26 million customers a week so there’s a huge opportunity to directly impact how the nation shops,” Hunter says. For people who know how to wrangle data, “there is no better place to be than Sainsbury’s” she adds.

That this work will be touched by millions of people is a significant drawcard for supermarkets compared with other industries skilled candidates could head into.

“Our data scientists can make an impact in the millions of pounds, for thousands of colleagues and millions of shoppers,” Hunter says.

Armstrong says that Tesco’s data scientists “develop living and breathing technology products that improve all areas of our business - not simply one-off pieces of analysis that sit on a shelf gathering dust”. Their work, he adds, “reaches millions of people, businesses and communities every day”.

To reach more potential candidates, and improve the diversity of their data teams, M&S and Sainsbury’s have also partnered with data professionals network Women in Data. M&S said the ­partnership would help drive positive change and “help us to attract the very best talent in the field”.

Faced with a highly competitive job market, some supermarkets are also looking inward.

As well as “bringing in new and exciting talent” to M&S’s data capability - hiring the company’s first head of data science, Mehdi Hosseini, in October - there has also been an internal skills drive. Last year, M&S announced plans to upskill existing staff to work with the company’s data. A course aimed at management is ongoing to “create a new raft of data-skilled leaders” while non-managerial staff can enlist on an 18-month, in-work Data Fellowship programme.

“We’re upskilling our people to ensure we can drive change and the way we handle and capitalise on the data we create as a business. This allows us to retain the business expertise we already have in house and arm it with cutting-edge data skills,” the company says.

In March, Asda said it would be making a data science training programme available to hundreds of workers. Twenty-five data analyst apprentices are currently being trained.

“This is vital to make sure we can continue to keep up with the changes we see in retail,” Kate Fisher, Asda’s senior director for talent and capability said at the time.

Outdated tools

The data recruitment and training drive is ongoing for the supermarket majors. But simply having people with the right skills doesn’t make a business data-driven. The new talent needs to be able to access the data and have the software to use it.

According to IGD’s Building Digital Capability report, published earlier this year, companies are still struggling with “systems installed a long time ago that don’t speak to each other” and “managing with big data gaps”.

Zell says that far too many supermarkets are hamstrung by their outdated data tools. “I’ll put this bluntly: 98% of the work that happens with data in supermarkets is done on Microsoft Excel. And it’s holding these places to ransom,” he says.

Enacting a data-centric strategy crucially requires a change of culture. For retailers in which “gut instinct” has ruled business and customer strategy decisions for decades, analytical approaches that challenge this way of working are met with “some hesitation”, said 29% of respondents in the Dunnhumby survey.

As Pee puts it, becoming a data business “doesn’t just require an increase in talent and data literacy, but a mindset change across the whole organisation to think and act differently”.

“And we’re only just at the beginning of this change.”

Follow your gut or follow the data?

M&S last year launched what it called “the world’s first retail data academy” working with London-based education provider Decoded.

With more than 1,000 employees to be enrolled on courses, it is also claimed to be the “world’s largest data skills initiative”.

M&S CEO Steve Rowe said it was the company’s “biggest digital investment in our people to date”.

“We need to change their digital behaviours, mindsets and our culture to make the business fit for the digital age,” he said at the time.

A company spokesman says the programme is now working to “help us to move away from gut-feel and allow us to make more information-enriched decisions across the business in real time”.

Leaders get “hands-on” with different datasets and get help to improve their data literacy. Employees learn how to use data tools and how to adopt technologies like machine learning. In classroom and online lessons, they work towards a British Computing Society-accredited course in data analytics. They are also “mandated to apply the learnings in their day jobs” explains Decoded’s Dan Zell.

“The side effect is that you’re building these communities of best practice and data evangelists that go back to their teams. And it creates this positive ripple throughout the organisation,” Zell says. “You’re lighting hundreds of little fires around the business.”

In M&S’s case the training is fully funded by the Apprenticeship Levy. To date 500 leaders have taken the data-literacy course and 185 staff across four cohorts have enrolled.

It has created a new digital and data function with six areas of focus: customer growth, loyalty, digital product management, data science, data and business development. The data science team alone is to expand by some 20 roles in coming months.

“There’s a need for data to permeate and enrich every role and every decision that’s made in an organisation,” Zell says.