/ 3 min
A price forecast that has to grade itself
Research notes from barato.si: how the price-error engine weighs evidence, and why the new forecast publishes its own report card.
barato.si is a daily index of US price drops and probable price errors. Most deal sites rank by the discount the seller claims. I wanted the opposite: rank by what the index has actually seen, and say out loud when it hasn’t seen enough. These are my notes on the two pieces that do the judging.
Evidence before discounts
Every offer the pipeline collects goes into a ledger keyed by product, one daily low per product. When a price shows up, the engine looks for the strongest reference it can find. First the product’s own observed median over at least three earlier days. Then the same model at another store in the same currency. Then a dated reference price someone curated. The seller’s list price comes last.
The order matters because list prices are the easiest number to inflate. A pair of pajama pants “2 for $10” against the seller’s own list price looked like a pricing mistake under the old rules. Under the current ones it scores 39, one point under the bar, because a noname seller’s list claim alone can’t carry an error. When I re-scored the archive with today’s rules, all eight of the old “errors” fell under the line. The site shows them nowhere.
The newest signal looks for the shape of a real mistake: a price that held steady for three or more observed days, then fell 30 % or more below its own low. Sales tend to step down. Mistakes fall through the floor.
An empty errors page is a legitimate result. The threshold doesn’t drop to fill it.
The forecast
The .si in the domain started as a joke about super intelligence, so the site now has to earn it. The page asks one question per product: will a price at least 5 % lower show up within 14 days?
The model is small on purpose. Each product’s history is cut into closed, non-overlapping 14-day windows, and each window either saw a 5 % drop or didn’t. Products with little history borrow strength from their category: a Beta prior fitted by method of moments across all products, updated with the product’s own windows. That’s an empirical-Bayes beta-binomial. If the chance is 0.5 or more it says “wait”, otherwise “buy”. With fewer than two windows of its own it says “learning” and makes no call.
The report card
A forecast nobody checks is decoration. Every prediction is logged with the base rate known at that moment. Fourteen days later it resolves against the ledger: it dropped, it didn’t, or nothing was observed and it’s left out instead of guessed. The page publishes the model’s Brier score next to a naive predictor that always says the base rate, plus calibration buckets that compare predicted chances with what happened.
Right now every product says “learning”. The ledger started in October, so no product has two closed windows yet, and the first predictions resolve on November 19. That’s the honest state, and the page says so.
Scoring a model against a naive baseline, and showing readers how far to trust it, came straight from my applied machine learning and human-centered AI coursework. barato.si is where I get to watch it work on live data.