Skip to content

Test system overview

Concise reference for developers: how pytest is wired, where shared fixtures live, and how configuration is supplied in tests.

Running tests

Use the project’s package manager:

uv run pytest                    # whole tree (from repo root)
uv run pytest src/listener/tests/test_listen_unit.py
uv run pytest src/tests/test_environment.py -q

pyproject.toml sets asyncio_mode = auto and pytest_env sets APP_ENV=test plus LISTENER_SR_URL_LEAGUE=mlb (so get_listener_environment() works without a repo-root .env).

Why tests see the Postgres container

Pytest only loads conftest.py files on the path from each test file up to the root. src/tests/conftest.py is not an ancestor of src/listener/tests/, so it would not load for listener-only runs.

The repo-root conftest.py fixes that with:

pytest_plugins = ["tests.conftest"]

That registers src/tests/conftest.py as a plugin for every run that uses this rootdir. Session infrastructure (below) therefore runs even when you collect only src/listener/tests/ or src/api/tests/.

Shared plugin: src/tests/conftest.py

Mechanism Role
pytest_configure Celery: task_always_eager and task_eager_propagates so tasks run inline in tests (see Celery). Also mkdirs repo-root data/listener so relative FILESYSTEM_CONTAINER_DATA_DIR tests work on a clean checkout.
container_db_url (session) Starts a Postgres testcontainer (pgvector/pgvector:pg17), sets DATABASE_URL via MonkeyPatch for the session, and points db_sessionmanager at that URL.
container_db (session, autouse) Creates DB, runs init_database (SQLModel metadata create_all), yields, then drops the DB.
container_redis (session, autouse) Starts Redis (redis:7-alpine) and sets REDIS_URL via MonkeyPatch. Isolated from host/compose Redis (do not use Scarlett staging Redis).
db (function) Async SQLAlchemy session bound to a connection inside a transaction that rolls back after each test.
factories (function) Async Factory Boy factories tied to that same AsyncSession (see below).
localstack_container (session) Optional LocalStack + S3; sets AWS env vars for tests that request this fixture.

Anything that needs the real async DB in a test should use the db fixture (or the same patterns as existing API/lib tests). Listener unit tests that only exercise _process_event with mocks may not request db but still pay the session cost of container_db because it is autouse.

Factory Boy and the factories fixture

Async tests use the db fixture, which opens an outer transaction and binds the session with join_transaction_mode="create_savepoint". Commits on that session end a savepoint, not the outer transaction, so everything still rolls back when the test finishes.

Test data for League / Team / Match should be created with the factories fixture so Factory Boy uses that same AsyncSession (async-factory-boy AsyncSQLAlchemyFactory). The bundle is returned by bind_async_sqlmodel_factories(db) in tests.factories (src/tests/factories.py). Do not add rows through a different session or connection, or isolation breaks.

Usage:

  • Request factories and call await factories.league.create(...), await factories.team.create(...), await factories.match.create(...), or await factories.match.create_with_teams(...) for league + two teams + match in one step.

Code reference: src/tests/conftest.py (factories fixture docstring), src/tests/factories.py.

Test layout (high level)

Area Typical path Notes
Shared DB/Celery/LocalStack src/tests/conftest.py Loaded globally via pytest_plugins.
App / plumbing src/tests/ e.g. test_plumbing.py, test_environment.py.
Listener src/listener/tests/ test_listen_unit.py (in-process), test_listen.py (script subprocess), stages/ (mirrors listener/lib/stages/), conftest.py for listener fixtures.
Library src/lib/tests/ Often nested by domain.
API src/api/tests/ May define local conftest.py.

Configuration in tests: get_base_environment / get_listener_environment

Settings classes use Pydantic Settings. Factories wrap the constructors:

  • get_base_environment(**overrides) → BaseEnvironment(**overrides)
  • get_listener_environment(**overrides) → MatchListenerEnvironment(**overrides)

Precedence (highest first): keyword arguments → process environment → env files → field defaults. See Pydantic field value priority.

In tests, prefer explicit overrides for the fields you care about instead of relying on a full .env. Remaining fields still resolve from env/files (e.g. DATABASE_URL once the session container has set it).

Daily CI

Workflow: .github/workflows/staging-daily-pytest.yml (Staging daily pytest) on Scarlett (self-hosted, macOS ARM64). Always checks out staging.

  • Schedule: 0 6 * * * (06:00 UTC = 2am EDT / 1am EST). GitHub cron is UTC-only.
  • Manual: Actions → Staging daily pytest → Run workflow (still tests staging, not the branch you pick).
  • Command: uv run pytest --junitxml=pytest-results.xml from repo root (same default suite as above; -m 'not external' via pyproject.toml). The job mkdirs $GITHUB_WORKSPACE/data/listener and sets placeholder DATABASE_URL / REDIS_URL / SPORTRADAR_API_KEY / AWS keys / FILESYSTEM_CONTAINER_DATA_DIR so import-time BaseEnvironment can construct (no repo-root .env on a clean checkout). Testcontainers still replace DATABASE_URL and REDIS_URL. Do not source .env.staging (that would point tests at staging Postgres/Redis). LISTENER_SR_URL_BASE is set to the staging srsim (https://pr-srsim-staging.jvsassoc.com); requires_srsim tests skip when that host is unreachable. pytest_env supplies LISTENER_SR_URL_LEAGUE.
  • Success: silent. Failure or timeout (90m): Slack root (staging, short SHA, Open Actions run + Fix with AI) plus a thread listing failing tests from junit. Artifact pytest-results. See STAGING.md.
  • Reuses Actions secret SLACK_BOT_TOKEN and variable or secret SLACK_CHANNEL_ID (same as staging release notes).
  • GitHub registers the cron from the default branch workflow file (develop); the job then checks out staging. The YAML must exist on develop before the nightly job fires.
  • Celery — broker, result backend, autodiscovery, and why tests use eager execution.
  • Environment configuration — env files, BaseEnvironment, MatchListenerEnvironment.
  • Root conftest.py — short comment on pytest_plugins.