S3 outputs

Static artifacts are published to records.abstractplay.com (S3 + CloudFront). DynamoDB exports land in abstractplay-db-dump daily (dumpdb); batch jobs read the latest completed export.

Records bucket layout

Game record format

Each record in ALL.json, meta/, player/, and event/ files conforms to the APGameRecord schema documented in Recranks. Records are produced by calling GameFactory(metaGame, state).genRecord(...) in records.ts.

Key header fields used downstream:

• header.game.name — meta game display name (used as map key in summarize)
• header.players[].userid — player ID
• header["date-start"], header["date-end"] — ISO timestamps
• moves — move history (timeout/abandoned detection in summarize)

_summary.json and tier files

The monolith (_summary.json) is typed as StatSummary in src/types/stats/StatSummary.ts. Tier types: StatSummaryTiers.ts.

All JSON objects use Content-Type: application/json. Each tier/slice includes generated (ISO timestamp).

Monolith top-level fields

Full field documentation: Summarize.

mvtimes.json

Produced by records-move-times. Object with:

summarize copies seasonality and aligns weeklyActiveMovers into _summary.json (histograms.activeMovers).

recommendations/cooccur.json

PMI-normalized co-occurrence neighbors per meta-game for the hybrid recommender. Includes optional stars boost (includeStarredBoost). Full schema and algorithm: Recommendation co-occurrence.

Private ops bucket (private-ops-153672715141-us-east-1-an)

Not served via CloudFront. IAM-restricted.

See Recommendation analytics and Summarize.

Dump bucket layout

Exports from dumpdb appear under:

AWSDynamoDB/{export-uid}/manifest-summary.json
AWSDynamoDB/{export-uid}/data/*.ion.gz

Batch functions locate the newest manifest-summary.json, extract the UID, and read all data/*.gz files for that export.

_manifest.json (v2)

Typed in src/utils/recordsManifest.ts. The records bucket manifest is no longer a bare S3 Contents array.

{
  version: 2,
  generated: string,          // ISO timestamp when manifest was built
  summaryFiles: {
    monolith:  { key, lastModified?, size? },
    site:      { key, lastModified?, size? },
    players:   { key, lastModified?, size? },
    ratings:   { key, lastModified?, size? },
    playerSummaryPattern: "player/{userId}-summary.json",
    playerManifest: { key: "_summary-player-manifest.json", ... }
  },
  objects: _Object[]            // full bucket listing (same data as legacy root array)
}

Backward compatibility: legacy consumers expecting a root array should use Array.isArray(data) ? data : data.objects.

records-manifest runs at 04:00 and 07:30 UTC (late pass is after summarize at 06:00 and player-summary-fanout at 06:15). The handler logs a warning if any required _summary*.json tier key is missing from the listing.

CloudFront and caching

Distribution: EM4FVU08T5188 — https://records.abstractplay.com

CloudFront does not run blanket /* invalidations (removed to avoid quota/cost issues). Freshness relies on S3 object headers set by crons.

S3 cache headers (all records-bucket JSON)

After each daily cron overwrite, the next CDN/browser request revalidates with S3 (If-None-Match); changed objects return a new body without invalidation.

Implemented in src/utils/recordsJson.ts (putRecordsJson).

Gzip compression

CloudFront compresses responses only when the origin returns a compressible Content-Type (e.g. application/json). Objects previously uploaded as application/octet-stream were not compressed.

1. Crons — set Content-Type: application/json on upload (above).
2. CloudFront — enable Compress objects automatically on the default cache behavior.

One-time enable (prod credentials):

npm run enable-records-cdn-compress
# preview: npm run enable-records-cdn-compress -- --dry-run

Or in the AWS Console: CloudFront → distribution EM4FVU08T5188 → Behaviors → Edit default → Compress objects automatically: Yes.

Verify after deploy + CF propagation:

curl.exe -sI -H "Accept-Encoding: gzip" https://records.abstractplay.com/_summary-site.json

Expect Content-Encoding: gzip and Content-Length much smaller than the uncompressed object.

Clients should use _manifest.json summaryFiles or tier URLs rather than hard-coding cache assumptions.

Related

• Records pipeline
• Recommendation co-occurrence
• Recommendation analytics
• Recranks schema reference
• Functions reference