Environment configuration¶

Configuration is loaded with Pydantic Settings (BaseSettings). Shared settings live in lib/base_environment.py; process modules add their own layers (listener/listener_environment.py, api/api_environment.py, srsim/srsim_environment.py, tasks/celery_environment.py).

`BaseEnvironment`¶

Class: BaseEnvironment in src/lib/base_environment.py.
Env files: repo-root .env (path is resolved relative to that module).
Factory: get_base_environment(**overrides) → BaseEnvironment(**overrides). Overrides take precedence over env (see Pydantic Settings field value priority).

Fields include app_env, database_url, redis_url, AWS credentials, sportradar_api_key, filesystem_container_data_dir, optional OpenAI key, shared sentry_traces_sample_rate (SENTRY_TRACES_SAMPLE_RATE) and sentry_profiles_sample_rate (SENTRY_PROFILES_SAMPLE_RATE), match_event_metrics_enabled (MATCH_EVENT_METRICS_ENABLED), etc. Sample rates are not sensitive. Sentry DSNs are not on BaseEnvironment — each process module owns its own optional sentry_dsn (marked sensitive): Sentry.

`match_event_metrics_enabled`¶

Master switch for slim match_event_metrics rows (Metabase match timelines). Default off. The listener allocates no batch and no flush loop when off. Catch-up, sweep, prune, and staging backfill no-op (skipped). Enable on staging (MATCH_EVENT_METRICS_ENABLED=true on listener + full worker). Rows are type, time, period/phase, and flags only (no event JSON); pruned after 14 days. See Celery, Metabase, and listener system.

Credential-bearing fields are marked json_schema_extra={"sensitive": True}. Use model_dump_for_logs / model_dump_json_for_logs for logs, Sentry contexts, and DB startup_params (first six characters + ***). Plain model_dump keeps full values for runtime use.

`filesystem_container_data_dir`¶

In-container path the app uses for listener recordings and other filesystem data. Distinct from the host bind-mount source used only by Docker Compose.

Role	Name	Typical value
Inside containers / app settings	`filesystem_container_data_dir` / `FILESYSTEM_CONTAINER_DATA_DIR`	`/app/data` (default)
Host bind source (compose)	(hardcoded in compose / local `./data`)	`$HOME/parlay-rewards-data` (staging) or `./data` (local)

Default: /app/data. Usually omit from container .env files when that path is bind-mounted.
Optional override via FILESYSTEM_CONTAINER_DATA_DIR or constructor kwargs (required on the host if /app/data does not exist).
The stored value is always an absolute Path. Relative values resolve against the repository root (parent of src/). ~ is expanded first.
The directory must exist when settings are built; otherwise validation fails (no fallback to <repo>/data).
MatchListenerEnvironment inherits this field from BaseEnvironment.

Used by: listener logging/recordings (live write path), and archive upload from those paths. Replay / admin download / clone load from S3 (AWS_S3_MATCH_ARCHIVE_BUCKET), not from this directory.

S3 buckets and credentials¶

Terraform creates account regional namespace buckets for staging / main (Terraform): prefix {app_env}-general / {app_env}-match-archive, full name {prefix}-{account_id}-{region}-an. Set env vars from terraform output (general_bucket_id, match_archive_bucket_id).

LocalStack keeps short names develop-general / develop-match-archive via docker/localstack/init-s3.sh.

AppEnv	`AWS_S3_DEFAULT_BUCKET`	`AWS_S3_MATCH_ARCHIVE_BUCKET`
`develop` (LocalStack)	`develop-general`	`develop-match-archive`
`staging` / `main` (AWS)	Full account-regional id from Terraform	Full account-regional id from Terraform

Setting	Local / LocalStack	Staging (Scarlett)	Main (ECS/EC2, future)
`AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY`	Required (e.g. `test`)	Set (IAM user keys)	Omit — task role / instance profile
`AWS_ENDPOINT_URL`	LocalStack URL	Omit	Omit
`AWS_DEFAULT_REGION`	e.g. `ca-central-1`	e.g. `ca-central-1`	e.g. `ca-central-1`

S3Helper (src/lib/s3.py) passes explicit keys only when both access-key env vars are set; otherwise it uses the default AWS credential provider chain. No AppEnv-specific branch — staging keys vs main role is entirely env config.

`aws_s3_match_archive_bucket`¶

S3 bucket for archived listener event recordings and logs.

Role	Name
Env / settings	`aws_s3_match_archive_bucket` / `AWS_S3_MATCH_ARCHIVE_BUCKET`

Required for archive upload and for admin download, clone, and srsim loads of archived matches (Match.listener_events_archived). Object keys are {shortcode}.json and {shortcode}.{listener_record_id[:8]}.log at the bucket root.

Staging/main only: develop and test never archive to S3. Listener recordings stay on local disk; enqueue and archive_match_listener_data are no-ops. The 5-minute sweep_pending_listener_archives beat task is disabled in develop.

Also set aws_s3_default_bucket / AWS_S3_DEFAULT_BUCKET to {app_env}-general.

Tests for this behavior live in src/tests/test_environment.py.

`MatchListenerEnvironment`¶

Class: MatchListenerEnvironment, subclass of BaseEnvironment, in src/listener/listener_environment.py.
Env files: same as BaseEnvironment — repo-root .env only.
Factory: get_listener_environment(**overrides) → MatchListenerEnvironment(**overrides). Overrides take precedence over env.

Adds listener-only settings (log level, Sportradar URL pieces, league, timeouts, exit codes, etc.) plus optional sentry_dsn bound to SENTRY_DSN_LISTENER.

Used by: listener/listen.py, listener Celery worker Sentry init, and listener tests.

Other module environments¶

Thin BaseEnvironment subclasses that add only a module-owned sentry_dsn (each with a distinct env alias). Same env-file list and precedence as base.

Class	Factory	Env alias	Used by
`ApiEnvironment`	`get_api_environment`	`SENTRY_DSN_API`; also `SLACK_SIGNING_SECRET`, `SLACK_BOT_TOKEN`, `SLACK_QD_EDITOR_USER_IDS`, `SLACK_QD_EDITOR_API_KEY`, `SLACK_PYTEST_FIX_USER_IDS`, `GITHUB_DISPATCH_TOKEN`, optional `GITHUB_REPOSITORY`; `PLAY_OPEN_CACHE_TTL_SECONDS`, `PLAY_WS_MAX_CONNECTIONS`	FastAPI lifespan, Slack agents, live-play hub
`SrsimEnvironment`	`get_srsim_environment`	`SENTRY_DSN_SRSIM`	SRSim lifespan
`CeleryEnvironment`	`get_celery_environment`	`SENTRY_DSN_CELERY`	General Celery `worker_init`

Push notifications (Expo transport)¶

Variable	Purpose
`EXPO_ACCESS_TOKEN`	Optional Bearer for Expo Push API when Enhanced Security for Push is enabled on the EAS project. Read at send time from the environment (not admin UI).

Push gates and message templates are not env vars — they live in stored_settings (push_notification_settings) and Admin → Push Notifications.

Shared across all modules via BaseEnvironment: SENTRY_TRACES_SAMPLE_RATE (Performance sampling; default 0).

Details: Sentry.

Live play realtime¶

Consumer protocol: Live play client. This section is ops only (same process as the API today; edge routing ready to split).

Knob	Env / where	Staging (≈5–10 users)	Thousands of concurrent
Process	Compose `fastapi` only — no second realtime service	`uvicorn … --workers 1`	Scale API tasks first; optional later: retarget WS rule to a dedicated pool (same image)
Open-questions cache TTL	`PLAY_OPEN_CACHE_TTL_SECONDS` (`ApiEnvironment`)	`2` (default)	Same; invalidate-on-issue/resolve is primary
Max WS connections / process	`PLAY_WS_MAX_CONNECTIONS` (`ApiEnvironment`)	`100` (default; headroom above 5–10)	Size from FD/RSS budget; replicas ≈ `ceil(peak / cap)`
Redis DB	`REDIS_URL` db0 — keys `play:open:shared`, channels `play:match:{id}`	Shared Redis OK	Watch pub/sub + cache memory; do not use RedBeat `/1` or broker `/2`
Edge path routing	Caddy (staging) / ALB (prod)	Separate matcher for `/api/play/stream` with WebSocket upgrade → same API upstream	Retarget that rule to a realtime target group when WS contends with REST; no client URL change
Idle timeout	Proxy / ALB	Modest OK	Raise above silence between questions (minutes). Sticky sessions optional — not required (Redis pub/sub fans out to every instance)

`PLAY_WS_MAX_CONNECTIONS` resource implications¶

This is a per API process gate on concurrent WS /api/play/stream connections (enforced by PlayStreamHub; over-cap clients get WebSocket close 1013). It is not a cluster-wide quota — total capacity ≈ cap × number of API (or realtime) replicas.

Resource	Effect of raising the cap
File descriptors	~1 FD per open socket (plus OS/proxy overhead). Host `ulimit -n` must clear `cap` with margin for REST, Redis, Postgres, and other sockets.
Memory (RSS)	Each connection keeps hub subscribe state and short-lived outbound buffers. Idle play sockets are cheap vs REST payloads, but thousands on one worker still add up; budget roughly low‑KB per connection plus fan-out spikes when many clients get the same `question.opened`.
Event loop / CPU	Sockets are asyncio-idle most of the time; cost shows up on broadcast (issue/resolve) and reconnect storms. A very high cap on one worker can stall REST latency on that same uvicorn process.
Redis	Cap does not open Redis connections per player. Pub/sub is one (or few) subscriber(s) per process; load grows with event rate × local subscribers, not with the numeric cap alone.
Postgres	Cap does not hold a DB session per socket. Auth/snapshot use short-lived sessions; answer `PUT`s stay REST.

Sizing guidance: keep the cap at what one worker can hold comfortably (staging default 100 is fine for tens of users). For thousands of concurrent players, add replicas and keep a moderate per-process cap rather than setting this to the full peak on a single process. Watch API RSS, open FDs, and p99 REST latency under live games when tuning.

Staging Caddy lives in the shared services repo. Add (or keep) a site snippet that matches path /api/play/stream* with reverse_proxy and WebSocket support to the same pr-fastapi upstream as other /api/* routes. Document the rule separately from the default API handle so prod ALB can mirror it (path-pattern → target group).

Precedence and naming¶

Keyword arguments passed to get_base_environment(...) / get_listener_environment(...) / other module factories (or to the settings class directly) override everything below (Pydantic Settings).
Process environment (shell, CI, containers) overrides values from env files.
Env files are read in list order; a later file can override an earlier one for keys supplied only via files.

Pydantic maps field names to env vars in the usual way (e.g. database_url → DATABASE_URL). Module Sentry fields use an explicit validation_alias (e.g. sentry_dsn ← SENTRY_DSN_LISTENER).

Class inheritance¶

Module environments subclass BaseEnvironment; both build on Pydantic’s BaseSettings. Module-only fields live on the subclass; shared fields are inherited and configured the same way (env var names, validators such as filesystem_container_data_dir).

flowchart TB
    BS["BaseSettings (pydantic_settings)"]
    BE["BaseEnvironment<br/>shared app fields"]
    MLE["MatchListenerEnvironment"]
    API["ApiEnvironment"]
    SR["SrsimEnvironment"]
    CE["CeleryEnvironment"]

    BS --> BE
    BE --> MLE
    BE --> API
    BE --> SR
    BE --> CE

`BaseEnvironment` — where each field’s value comes from¶

Highest row wins when the same setting is provided in more than one place. This matches Pydantic Settings field value priority: init kwargs, then environment variables, then env file(s), then defaults.

flowchart TB
    subgraph BE_prec["BaseEnvironment"]
        direction TB
        L1["1 strongest — kwargs / get_base_environment overrides"]
        L2["2 — process environment os.environ"]
        L3["3 — repo-root .env"]
        L4["4 weakest — field defaults in BaseEnvironment"]
    end

    L1 --> L2
    L2 --> L3
    L3 --> L4

Arrows run from higher priority (top) to lower (bottom): each level wins over everything below it.

`MatchListenerEnvironment` — inheritance¶

The listener class inherits all BaseEnvironment fields (including the same env-file list) and adds listener-specific fields. Precedence is identical to BaseEnvironment: kwargs → process env → repo-root .env → defaults.

flowchart TB
    subgraph MLE_prec["MatchListenerEnvironment"]
        direction TB
        M1["1 strongest — kwargs / get_listener_environment overrides"]
        M2["2 — process environment"]
        M3["3 — repo-root .env"]
        M4["4 weakest — field defaults base + listener fields"]
    end

    M1 --> M2
    M2 --> M3
    M3 --> M4

flowchart TB
    MLE_inst["Single MatchListenerEnvironment instance"]

    subgraph inherited["Fields inherited from BaseEnvironment"]
        B1["database_url, redis_url, app_env, filesystem_container_data_dir, …"]
    end

    subgraph own["Fields declared on MatchListenerEnvironment"]
        L1["listener_*, sentry_dsn ← SENTRY_DSN_LISTENER"]
    end

    MLE_inst --- inherited
    MLE_inst --- own

Listener-only fields are set via LISTENER_* env vars (in process env or .env), constructor kwargs, or start-time merge (resolve_listener_environment). BaseEnvironment uses extra="ignore" so those LISTENER_* keys in the shared .env do not break non-listener settings loads.

Listener start merge (Beat / Celery / admin)¶

When a listener is started for a known Match (Celery start_match_listener, or run_match_listener), resolve_listener_environment in src/listener/lib/start.py builds the kwargs passed into Layer A above:

Call overrides (task/CLI kwargs) — strongest among start inputs
Match.listener_start_settings (JSON on the match row)
Auto-inject listener_sr_request_match_uuid from match.sr_id and listener_sr_url_league from match.league.name unless the call overrides already set those keys
Then Pydantic Layer A fills anything still unset (process env → env files → defaults)

Beat-driven starts pass overrides=None, so the Sportradar match id (match.sr_id) and league come from the database and optional tweaks come from listener_start_settings. Celery still addresses the row by internal match id (Match.id). Full sequence diagram and guards: Match listener startup.

`AppEnv`¶

app_env is typed as AppEnv (develop, staging, main, test), typically from APP_ENV. Code branches on it—for example, DatabaseSessionManager defers engine creation when app_env == test so pytest can point at a test database after startup.

Other details¶

load_dotenv() runs at import in lib/db/utils.py, so .env is also merged into os.environ early for anything that reads the process environment directly.
Factories return a new settings instance each call; there is no global singleton. Some modules (e.g. listener helpers) assign get_listener_environment() once at import for that process.

Quick reference¶

Need	Import / call
Shared app config (DB, Redis, AWS, …)	`from lib.base_environment import get_base_environment`
Listener process	`from listener.listener_environment import get_listener_environment`
Listener start with match merge	`from listener.lib.start import resolve_listener_environment` / `run_match_listener`

Environment configuration¶

BaseEnvironment¶

match_event_metrics_enabled¶

filesystem_container_data_dir¶