---
title: "Scraping car auction sites: the real cost, versus buying the data"
description: "A line-by-line cost model for building in-house auction scrapers: proxies, storage, engineering time, and the maintenance tail nobody budgets for. Plus the cases where building it yourself is genuinely the right call."
slug: scraping-car-auction-sites-vs-api
canonical: https://thecarapi.com/blog/scraping-car-auction-sites-vs-api
category: Build vs buy
author: TheCarApi Engineering
published: 2026-02-26
updated: 2026-08-14
tags: [web scraping, build vs buy, proxies, data engineering]
---

# Scraping car auction sites: the real cost, versus buying the data

The build-versus-buy conversation usually stalls because one side is comparing a subscription against a sprint. The honest comparison is against three years of maintenance.

Someone on your team has already built a scraper for one auction site. It took a weekend, it worked, and it produced a CSV that made everyone briefly optimistic. That prototype is the reason this conversation is difficult: it is real evidence that the problem is easy, and it is measuring the wrong thing.

The weekend prototype solves collection for one source, one page layout, one moment in time, with no availability requirement. Production solves collection for seven sources, continuously, while the sources actively change, at a volume where storage and bandwidth stop being rounding errors. This article prices that second thing.

## The cost model, line by line

Figures below are order-of-magnitude for a team collecting roughly a million live listings across seven European and overseas sources with images. Your numbers will differ; the structure of the model will not. Treat the ranges as a framework to fill in with your own quotes, not as a benchmark.

| Line item | Typical monthly range | Why it lands where it does |
| --- | --- | --- |
| Residential / mobile proxies | €800 – €4,000 | Datacentre IPs get blocked quickly on these targets. Residential bandwidth is priced per GB, and image fetching dominates the bill. |
| Anti-bot / CAPTCHA solving | €150 – €900 | Only needed on some sources, but the ones that need it need it constantly. |
| Compute (crawlers, browsers, queues) | €300 – €1,500 | Headless browsers are ~10× the cost of plain HTTP fetches. Which sources need one drives the whole figure. |
| Object storage for images | €100 – €600 | Multiple terabytes once you keep galleries for closed lots. Cheap per TB, easy to forget entirely. |
| CDN / egress for images | €200 – €1,200 | Scales with your traffic, not your crawl. Frequently the line that surprises people at launch. |
| Database and search | €200 – €800 | Faceted search over a million rows with price ranges is not a small instance. |
| Engineering maintenance | **0.5 – 1.5 FTE** | The dominant cost by a wide margin, and the one that never appears in the initial estimate. |

_Infrastructure lands somewhere around €2k–€9k/month. The engineering line is usually larger than everything above it combined._

> **The line that gets left off** — Half an engineer, permanently, is not a rounding error — at European loaded cost that is roughly €40k–€70k a year, every year, forever. It is also half an engineer not building the thing your customers actually pay for.

## Why maintenance never converges

The intuition that scrapers stabilise over time is wrong, and it is wrong for a specific reason: you are not maintaining code against a fixed specification. You are maintaining it against seven independent teams who ship whenever they like and owe you nothing.

### Markup drift

A frontend redesign, an A/B test, or a component library upgrade breaks selectors. The A/B test case is the worst: your extraction succeeds for 70% of requests and silently returns nulls for the other 30%, which looks like sparse data rather than a bug and can run for weeks before anyone notices.

### Anti-bot escalation

Bot mitigation improves continuously and asymmetrically. Every provider upgrade is a step change for you, arrives without notice, and is followed by an urgent unplanned week. You cannot schedule around it and you cannot predict it.

### Semantic drift

The subtlest failure. A source renames a fuel category, changes a currency, starts including VAT where it previously excluded it, or adds a fee to a displayed price. Nothing errors. Your pipeline ingests the new meaning under the old field name, and every downstream number is quietly wrong. This is the class of bug that reaches customers.

> Broken scrapers announce themselves. Semantically drifted scrapers do not, and those are the ones that damage trust in your product.

### Coverage decay

Rate limits force you to prioritise. You start crawling only the first N pages of each source, or refreshing prices only for lots under a certain age. Coverage drops from 99% to 85% over a year without a single incident, and nobody notices until a customer asks why a car they can see on Auto1 is not in your product.

## The legal and contractual dimension

This is genuinely jurisdiction-dependent and this article is not legal advice — but it belongs in the decision, because it is a real risk that rarely makes it into the spreadsheet.

- Most auction platforms' terms of service restrict automated access. Whether those terms bind you, and with what consequence, varies by country and by how you access the site.
- The EU **Database Directive** creates a *sui generis* right over substantial investment in compiling a database, separate from copyright in the individual records.
- Photographs are copyrighted works. Re-hosting a source's images is a distinct question from extracting factual specifications, and the answer is not the same.
- Personal data occasionally appears in listing text — seller names, phone numbers, locations — and GDPR applies to it regardless of how you obtained it.

The practical point is not that collection is forbidden. It is that the risk is real, unquantified, and sits with you. Buying from a provider does not make the underlying questions disappear, but it does move the operational relationship with the sources onto someone whose business depends on maintaining it, and it gives you a contract to point at.

## Time to first useful product

Cost is one axis. Calendar time is usually the one that decides it.

| Milestone | Building in-house | On an API |
| --- | --- | --- |
| First listing in your database | Days | Under an hour |
| One source, reliable, with images | 3 – 6 weeks | Same hour |
| Seven sources, normalized to one schema | 4 – 8 months | Same hour |
| Historical archive worth modelling on | **12+ months of collection** | Available immediately |
| Steady state | Never — see maintenance | Provider's problem |

_The archive row is the one that cannot be bought back with money or headcount. History only accrues in real time._

That last row deserves emphasis. If your product needs price history — comparables, market value estimation, anything trained on past sales — then starting collection today means your first credible model is a year away. There is no way to accelerate it, because the data is generated by the passage of time.

## When building it yourself is the right answer

Not a rhetorical section. There are cases where in-house collection is clearly correct, and pretending otherwise would be dishonest.

- **One source, one narrow slice.** You need Copart Germany, salvage only, one brand. That is a maintainable script, not a platform.
- **Data collection is your product.** If you sell the feed, the pipeline is your moat and outsourcing it makes no sense.
- **You need a field nobody exposes.** A specific inspection-sheet annotation, a seller-level signal — if it is not in any commercial feed, you have no alternative.
- **A source no provider covers.** Regional platforms outside the major seven are often only reachable if you build it.
- **Contractual data residency** requirements that a third-party API cannot satisfy.

> **The hybrid that usually wins** — Buy the commodity — the seven large sources everyone needs, normalized, with images — and spend your engineering budget on the one proprietary source or derived signal that actually differentiates you. Nobody wins a market by having the same Auto1 listings as everyone else, slightly later.

## Making the comparison honestly

If you are running this decision internally, build the comparison over 36 months rather than 12, and include these four things that standard estimates omit:

1. Loaded engineering cost for ongoing maintenance, not just the initial build. Use a real fully-loaded figure, not salary.
2. The opportunity cost of that engineering time — what does not get built.
3. The revenue impact of coverage gaps and stale prices during the months your pipeline is degraded.
4. The value of the archive you will not have for the first year.

Run that model and the answer is usually clear in one direction or the other, which is the point. For a concrete comparison of the two approaches against our own surface, [see the side-by-side](/vs-diy-scraping); for what the data looks like once it arrives, start with [the auction data guide](/blog/car-auction-api-guide).

## Frequently asked questions

### How much does it cost to scrape car auction sites at scale?

Infrastructure for roughly a million live listings across seven European and overseas sources with images typically runs €2,000–€9,000 per month — proxies and image egress dominate. The larger cost is engineering: sustained maintenance is generally 0.5–1.5 full-time engineers, which at European loaded cost exceeds the entire infrastructure bill.

### Is scraping car auction websites legal?

It depends on jurisdiction, on the platform's terms of service, and on what you extract and republish. In the EU the Database Directive's sui generis right, copyright in listing photographs, and GDPR where listings contain personal data are all separate considerations from the terms of service. This is a question for a lawyer familiar with your jurisdiction and business model, not one to settle from a blog post.

### Why do auction scrapers break so often?

Four independent causes: markup changes from redesigns and A/B tests, anti-bot escalation that arrives without notice, semantic drift where a field keeps its name but changes meaning, and gradual coverage decay under rate limits. Only the first announces itself with an error; the other three degrade data quality silently.

### Can I start with scraping and migrate to an API later?

Yes, and it is a reasonable path for validating a market. The one thing that does not migrate is historical data — if your roadmap includes price modelling or comparables, the archive you have not been collecting cannot be bought back after the fact, so factor that into the timing of the switch.

### What is the biggest hidden cost people miss?

Image bandwidth. Teams budget for collecting data and forget that each listing carries 20–60 photographs, that fetching them over residential proxies is billed per gigabyte, and that serving them to users is a second, separate egress bill that scales with traffic rather than with crawl volume.