cs-data

Start here: current state & results → docs/STATUS.md · findings history → docs/LEDGER.md · operations → docs/RUNBOOK.md · agent protocol → CLAUDE.md · script index → scripts/README.md

Two ingest pipelines into one shared SQLite at data/polymarket.db:

• polymarket-cs (this README) — Polymarket CS market trades
• hltv-scraper — HLTV match, map, round, and player data; see src/hltv_scraper/README.md

The hltv pipeline writes only to hltv_* tables and uses a dedicated hltv_alembic_version row, so the two pipelines coexist without fighting each other's migrations.

---

polymarket-cs

Ingest Polymarket Counter-Strike market trades into a local SQL database.

Setup

uv sync
cp .env.example .env

No API keys required — uses Polymarket's public Gamma and Data APIs.

Usage

1. Discover CS markets

uv run polymarket-cs discover

Fetches esports events from the Gamma API, classifies CS markets by tags and keywords, and upserts them into the database.

Add --report to write a summary to data/discovery_report.md.

2. Backfill trades

uv run polymarket-cs backfill

For every known market, paginates the Polymarket Data API to fetch all trades. Idempotent via ON CONFLICT DO NOTHING on tx_hash. After backfill, flags markets with zero trades for investigation.

3. Incremental sync

uv run polymarket-cs sync

Re-fetches trades for all markets, skipping already-ingested ones.

Data model

Table	Primary key	Description
markets	condition_id	CS market metadata (question, slug, exchange type)
outcomes	token_id	Outcome tokens per market (Yes/No or multi-outcome)
trades	id (unique on tx_hash)	Individual trades with price, size, side, trader address
needs_investigation	condition_id	Markets flagged by self-healing (zero trades despite existing)

Key fields on trades

• price: probability (0-1) at which the trade executed
• size: number of outcome tokens traded
• side: BUY or SELL
• trader: proxy wallet address
• outcome: outcome label (e.g. "Yes", "NaVi", "FaZe")

Development

uv run ruff check .
uv run mypy --strict src/polymarket_cs/
uv run pytest

Migrations

uv run alembic revision --autogenerate -m "description"   # polymarket
uv run alembic upgrade head

uv run alembic -c alembic_hltv.ini revision --autogenerate -m "description"
uv run alembic -c alembic_hltv.ini upgrade head            # hltv