angebots-radar

Why it’s built this way

Four places where it could have gone differently.

• No offers from OCR

  One leaflet was put through OCR as a test. It read the product names well, but between a third and four fifths of the offers extracted that way would have been wrong, and plausibly wrong: the right price on the wrong product, a discount sticker read as the price. That is the worst kind of error for an application that pushes to a phone unasked, because what is pushed gets believed.

• The whole leaflet page to the vision model, not the single tile

  Asking a vision model was rejected at first: running costs and a dependency on someone else’s service in an application that had none until then. The maths changed once it was no longer each of the 189 product tiles going out on its own, but the leaflet’s 17 pages in one go. That takes a minute and a half per leaflet, once a week, and the names come out complete; the hand-written parser had stopped at 88 per cent. Even so, only what passes the counter-checks is taken over: a price the model names has to appear in the text it quotes, and the model is allowed to say “unsure” about itself.

• A closed API stays closed

  One of the big chains explicitly turns away requests to its own offers API. That is a decision by the operator and not a technical hurdle, so it is not worked around with different headers. That chain’s offers are in the collection all the same, through the service they are published on anyway.

• No bought-in service for reading other people’s pages

  At around 600 calls a month, a paid scraping service would have fitted comfortably into the free tier. But what it is good for is reaching pages that fight off scripts. The source used here serves plain HTML, and the number of offers found would have stayed exactly the same. It becomes the right choice only when that source goes away.