Mysupermarket.co.uk was launched with great fanfare last week, with director Johnny Stern grandly proclaiming it would have the same impact on grocery shopping as eBay has had on non food - and save shoppers 20% on their usual weekly shop.

It was a bold statement. So when it went live this week, The Grocer decided to check out the site and find out for ourselves how it works - and how it plans to make money.

The premise is simple. The site allows shoppers to compare the price of their weekly shop at Tesco, Asda, Sainsbury's and Ocado. It first checks whether delivery is possible in the customer's area. Then, as you input your grocery items, it suggests other ways to save money using like-for-like alternatives and, after totalling your bill, it transfers the order to the site of the chosen retailer for payment.

In practice, however, things are not quite so simple. We log on and make our choices, no problem. Tesco is deemed the cheapest at £27.33 versus Sainsbury's £29.07, while Asda does not deliver in the area. No sign of a 20% saving yet, so we investigate the offer of a further £4 saving on the Tesco shop.

The site suggests swapping Tesco Norwegian Jarlsberg (£2.47 for 310g) for Président Emmental (£1.99 for 250g). OK, not the same cheese or weight, but a saving of 48p nonetheless. It also suggests switching a tin of Baxters soup at 74p (but on promotion at two for £1) for a Campbell's at 42p. Fair enough.

We're less convinced by two other suggestions. One is to trade Country Life Unsalted butter at 81p for its salted variety at 71p - surely a switch to another unsalted butter would be more logical? The site also suggests swapping a £4.99 500ml bottle of Almedra Estate Olive Oil for a £6.28 750ml bottle of Carapelli Olive Oil. A spokeswoman says the idea is to offer bulk savings, but does paying an extra £1.29 really constitute a saving?

Another issue is the site's failure to include the cost of delivery or details of delivery slots. Worse is to come. We try to pay but the site fails to link with Tesco's, announcing 'The page cannot be found'. The same message pops up when we try to link to other retailers' sites.

Apparently there were technical problems because of the sheer number of people logging on - 10,000 in the first 12 hours - which have since been resolved. Even so, that's a lot of people potentially put off in one fell swoop. And when we try again the next day, we still experience problems. But teething problems aside, there is still the question of whether the business model is robust. Stern insists that it is. However, the price comparison lacks clarity and the site is based on the premise that consumers shop purely on price. Will they really trade branded for own-label products - or, indeed, switch supermarkets just to save a few quid?

Neither Stern nor his unidentified backers has a background in grocery, but he is confident. "The site could suggest savings of up to 40%, but the average was 20% because shoppers may not choose to take all the alternatives offered. The main point is that shoppers really can save a lot of money."

Stern also believes the site will make money, though he backtracks on his earlier suggestion that shoppers will be charged minimal user fees once the site is established: "The site will be free for users. What we might charge for are any future add-on features, such as comparing healthy eating and dietary advice."

The logic of charging for advice available free elsewhere is debatable, but Stern says the supermarkets have been "positive" and that shopper feedback has been good. Whether shoppers will still be enthusiastic once the novelty factor wears off remains to be seen.