The machine, honestly
Method
How a messy pile of marketplace listings becomes a price you can stand behind, and where the honest limits of that are. Nothing here is dressed up beyond what the data supports.
Sources arrive
Two marketplaces feed the corpus, Grailed and eBay, pulled by Apify actors on a scheduled cron rather than on demand. The ingestion never runs on the site; the site only reads what the pipeline has already resolved and stored.
Every source enters through an adapter
Each marketplace speaks its own dialect, so each one passes through an adapter that maps its fields into a single canonical shape. eBay prices arrive as strings and are cast at the boundary, rows that fail to cast are rejected rather than coerced to zero, best-offer sales are flagged, and Grailed sold dates are dropped because they are scrape time, not sale time. No source-specific field leaks downstream.
The hard part: resolving a piece across marketplaces
Two sellers on two sites describe the same shoe in different words, with no shared identifier between them. Resolution is text-first: brand aliases, brand-specific model tokens, a garment vocabulary, and season codes combine into a stable canonical key that both listings land on.
Maison Margiela GAT German Army Trainer Replica Sneakers 43 Cream Suede
margiela replica gats size 10 * archive
| brand | maison margiela | alias hit on a normalized token |
| archetype | footwear | from the garment vocabulary |
| model | GAT | brand-specific model token, from the title |
| season | none | no season code present |
maison margiela|footwear|GAT|Both rows, written by different sellers on different sites, land on the same piece. That join, with no shared identifier to lean on, is the whole problem the project solves.
The marts compute the bands
Resolved rows roll up in dbt into per-piece marts: the median and the tenth and ninetieth percentiles, a best-offer-excluded median beside the all-in one, a confidence grade, and a recommended list price. Today that is 531 pieces over 4,280 sold comps across 2 marketplaces, with 14 pieces resolved cleanly on both sides for a cross-marketplace spread.
The model, measured against a plain baseline
A price model was trained and then scored honestly against the simplest baseline, the brand-and-archetype median. The verdict is that it does not beat that baseline in any way that matters, and saying so plainly is the point.
A LightGBM regressor, scored under group cross-validation by piece so it is always judged on pieces it never saw. It lands within about a percent of the plain brand-and-archetype median, which is to say it has no edge yet. The comps mart already recommends off that same median, so nothing is lost by admitting it. The model is a proof of the pipeline, not a price oracle; it earns its keep only once the corpus is broadened and deepened.
Measured on the current corpus under GroupKFold by piece. Printed by the pipeline, not persisted; shown here as a snapshot, not a live figure.
Visual search, and what it can and cannot see
Upload a photo and it is matched against a reference set of 943 listing images of the eighteen grails, embedded once with an off-the-shelf fashion CLIP model and stored as vectors in Postgres. The query image is embedded on a GPU that scales to zero, the nearest references are found by cosine distance, and those roll up to resolved pieces with their real comps. No fine-tuning: this is zero-shot similarity, nothing more.
Measured leave-one-out on the reference set, the nearest match shares the correct brand 95% of the time, the correct model 93%, and the exact piece 86%; widen to the top ten and brand reaches 99%. That ordering is the honest one: brand and type are reliable, the exact model is a hedge, the season is a guess. It is also an upper bound, because query and reference are both resale listing photos, so a real phone photo in bad light does worse, most of all on the soft items the grail set already flags, Dior denim and Undercover graphics. It cannot tell a real from a good replica, and a piece outside the eighteen comes back as no strong match rather than a forced answer.
The credibility rests on not overstating what the data supports.
- Grailed comps are undated, so they set price level, never timing.
- Velocity is eBay-only, because eBay carries the only trusted sold date.
- Days-to-sell is not derivable from sold comps and is never shown.
- Thin pieces fall back to a brand-and-archetype median, and say so.
- Confidence is graded by sample size, next to every price.
- Cross-marketplace spreads appear only where they can be stood behind.
- Visual search is zero-shot and graded leave-one-out; a real phone photo, or a piece outside the eighteen, does worse and is told so.