eurocrap

Look up a Eurostar service by date and train number.

A small Python app (src layout) with one CLI, eurocrap, dispatching to a few modules under src/eurocrap/ (the data sources live under eurocrap.providers.<kind>.<operator>):

• eurocrap.providers.gtfs — GTFS / GTFS-RT providers behind one abstract base (GtfsProvider): the join/lookup logic is shared, each provider only supplies the feeds.

  • .eurostar — the official open data. Pulls the static GTFS timetable and the GTFS-RT TripUpdates feed published by Eurostar on transport.data.gouv.fr, caches them on disk, and joins the scheduled stops for a service with its realtime overlay (per-stop delays, expected times, assigned platform/track, cancellations). No antibot/WAF.
  • .nl — the Dutch national feed (openOV / OVapi: gtfs.ovapi.nl static + gtfs.openov.nl/gtfs-rt/trainUpdates.pb), which carries the Eurostar services running through the Netherlands (Amsterdam ⇄ London / Paris / Marne-la-Vallée, agency IFF:NS_INT). The national stop_times.txt is ~1 GB, so the provider filters the feed to its Eurostar routes while loading, then reuses the same machinery. A different realtime source, so it often shows a different live picture than .eurostar.

  eurocrap gtfs queries all providers and prints each result, collapsing ones that agree; --provider eurostar|nl restricts to one.

• eurocrap.providers.scrape.eurostar — curl_cffi with TLS/JA3 impersonation, no browser. Parses the server-rendered __NEXT_DATA__ blob from the public timetable page (which honours ?date=YYYY-MM-DD), avoiding the WAF-protected JSON API. Returns the service status and the per-stop schedule with realtime times (scheduled vs expected arrival/departure, per-stop delays and cancellations) scraped from the page. Override the impersonation target with EUROSTAR_IMPERSONATE=chrome131 if needed.

• eurocrap.scheduler — an APScheduler job that periodically fetches a configured iCal feed, finds the Eurostar train events in it (extracting each train number from the event details), and schedules a one-shot check one hour before departure that reports the train's situation from both gtfs and scrape. Configured with pydantic-settings (env / .env / eurocrap.toml). Optional scheduler extra.

Install with uv

uv sync                      # gtfs + scrape modules
uv sync --extra scheduler    # + the iCal scheduler

No project checkout? Run a module straight from a one-off env:

uv run --with gtfs-realtime-bindings python -m eurocrap gtfs 9007 2026-06-20
uv run --with curl_cffi --with gtfs-realtime-bindings python -m eurocrap scrape 9007 2026-06-20

Usage

# One CLI, sub-commands (also: `python -m eurocrap ...`). All take <train_number> <day>.
eurocrap gtfs   9381 2026-06-16                      # all GTFS providers; shows diffs
eurocrap gtfs   9381 2026-06-16 --provider nl        # just the Dutch feed
eurocrap gtfs   9117 2026-06-13 --json
eurocrap gtfs   9117 2026-06-13 --refresh            # bypass the on-disk cache
eurocrap scrape 9007 2026-06-20
eurocrap scrape 9117 2026-06-20 --json

# Per-command console scripts also exist:
eurocrap-gtfs 9117 2026-06-13                     # all providers
eurocrap-gtfs 9381 2026-06-16 --provider nl       # Dutch national feed only

scrape derives the route (the origin/destination UIC codes the timetable page needs) from the train number via the static GTFS — the train's first and last scheduled stops — so it takes only the train number and date. The resolved route is shown in the header line (as route_long_name), matching gtfs.

Scheduler (serve)

eurocrap serve            # loop: refresh feed, schedule checks, serve web UI
eurocrap serve --once     # fetch once, list matched train events, exit
eurocrap serve --ical-url https://… --interval 15   # ad-hoc overrides
eurocrap serve --host 0.0.0.0 --port 8080           # web UI bind (default 127.0.0.1:8080)
eurocrap serve --include-past                       # also keep past trips
eurocrap serve --window-days 30                     # only events within 30 days

By default only future trips are kept (parsed, scheduled, and listed); pass --include-past (or set include_past = true) to include past trips too. Only events departing within window_days (default 89) are parsed at all.

Note: the scheduler only looks at trips within the next 89 days (window_days, the GTFS cutoff — just inside Eurostar's rolling ~90-day GTFS window). Eurostar's GTFS is a rolling ~90-day window, so trips further out can't be resolved against the schedule (status, route, per-train detail) anyway — they'd fall back to slow per-event booking lookups and have no GTFS detail page. Trips beyond the window are simply skipped.

While running, it also serves a minimal web UI (bottle):

• / — upcoming trips from the iCal feed: train number (linking to its detail page), departure/arrival times, current status (from GTFS), and — in the status cell — the trains immediately before (↑) and after (↓) on the same route, with their times and status.
• /<day>/<train_number> — full per-stop detail for that train on that day (GTFS schedule + realtime), e.g. /2026-06-15/9393, plus an adjacent trains table (the train before/after on the same corridor) at the bottom.

The "train before / after" is the service that passes the same start station before/after this one and also serves the same end station — i.e. the neighbouring departure on the corridor — looked up from GTFS + GTFS-RT.

On each refresh it scans the feed for Eurostar trips, pulls the train number out of each event (e.g. EUROSTAR 9368 in the description), and schedules a one-shot job 1 h before departure. That job looks the train up via both gtfs and scrape and logs the situation — see the TODO(notification) in scheduler.py:check_train to plug in real alerting. When an event has no train number in its text (some, e.g. eurostar.com entries, carry only a booking reference), the number is recovered from the GTFS by matching the route + departure time and, failing that, from the booking itself: eurocrap.booking does a two-step gateway fetch — GET the customer-dashboard page for the PNR to read the short-lived Bearer token from its __NEXT_DATA__, then POST the bookingBySession GraphQL query to site-api.eurostar.com/gateway (via the stealth curl_cffi client) for the booking's legs. Anything still unresolved is skipped.

A resolved train number is cached on disk ($EUROCRAP_CACHE_DIR/resolved_trains.json) keyed by the event's content, so an unchanged event isn't re-resolved on the next refresh — the GTFS match and (especially) the booking-page scrape only run when an event is new or changed (e.g. rebooked). Live status, however, is recomputed from GTFS every refresh.

Notifications

Get notified when a trip is cancelled / delayed / not scheduled (on first sighting, including the scheduler's first run) or whenever a trip's status changes between refreshes. Two independent channels:

• Email — set notify_email (env EUROCRAP_NOTIFY_EMAIL). Sent through the system's sendmail (the MTA mail uses), no SMTP config needed.
• SMS — set notify_sms_user + notify_sms_apikey (your Free Mobile account ID and API key). Sent via the Free Mobile SMS API.

Enable either, both, or neither. The last status per trip is cached on disk ($EUROCRAP_CACHE_DIR/notify_state.json), so restarts don't re-send unchanged alerts.

Configuration

Configure the feed URL and interval (no flags needed) via any of, in decreasing precedence: environment variables, a .env file, or a TOML config file. Copy .env.example → .env or eurocrap.example.toml → eurocrap.toml:

# eurocrap.toml
ical_url = "https://example.com/calendar.ics"
fetch_interval_minutes = 30

# or via the environment / .env (note the EUROCRAP_ prefix)
EUROCRAP_ICAL_URL=https://example.com/calendar.ics
EUROCRAP_FETCH_INTERVAL_MINUTES=30

Point EUROCRAP_CONFIG at a different TOML path if you don't want ./eurocrap.toml. The fetched feed is cached at $EUROCRAP_CACHE_DIR/ical_feed.ics.

As a library

from eurocrap.providers.gtfs.eurostar import lookup_train

trip = lookup_train("2026-06-13", "9117")  # -> TripResult
for stop in trip.stops:
    print(stop.stop_name, stop.scheduled_departure, stop.expected_departure)

lookup_train(day, train_number) returns a TripResult (route, per-stop schedule, and the realtime overlay). Feeds are cached under $EUROCRAP_CACHE_DIR (default $XDG_CACHE_HOME/eurocrap, i.e. ~/.cache/eurocrap); static feed ~6 h TTL, realtime ~30 s.

Exit codes (scraping commands): 0 match, 1 network error, 2 antibot block, 3 train not found. The gtfs command uses 0/1/3 (no antibot).

Development

uv sync --extra scheduler   # all deps incl. the dev group (ruff, mypy, pytest)
uv run ruff check src tests
uv run mypy
uv run pytest               # the unit-test suite under tests/

The test suite (tests/) is fully offline and deterministic: a synthetic in-memory GTFS zip + GTFS-RT protobuf stand in for the live feeds, and every network call (downloads, scraping, sendmail, SMS, the booking gateway) is monkeypatched. Run it before/after changes as the consolidated regression check.