From 4a510847b76ac0a4617b845be4cd08e3afb18e42 Mon Sep 17 00:00:00 2001 From: VinPropane Date: Mon, 30 Mar 2026 19:00:59 -0400 Subject: [PATCH] docs: data reload and telemetry operations policy (#6) Add data_and_ops_policy.md: boot-time content load, optional dev reload, E9.M3/E2.M4 for live tuning; telemetry PII rules, sampling, retention principles; baseline events from vision plan with future catalog as field source of truth. Register cross-cutting + intro; E9.M1, E2.M2, contracts, decomposition README, tech_stack (Related + Observability pointer). Made-with: Cursor --- docs/architecture/tech_stack.md | 3 +- docs/decomposition/README.md | 2 +- .../modules/E2_M2_XpAwardAndLevelEngine.md | 2 +- .../modules/E9_M1_TelemetryEventSchema.md | 2 +- docs/decomposition/modules/contracts.md | 1 + .../modules/data_and_ops_policy.md | 59 +++++++++++++++++++ .../modules/module_dependency_register.md | 6 +- 7 files changed, 70 insertions(+), 5 deletions(-) create mode 100644 docs/decomposition/modules/data_and_ops_policy.md diff --git a/docs/architecture/tech_stack.md b/docs/architecture/tech_stack.md index 0432578..18a8487 100644 --- a/docs/architecture/tech_stack.md +++ b/docs/architecture/tech_stack.md @@ -85,7 +85,7 @@ The [`client/`](../../client/) Godot project targets **4.6** (`config/features` ## Observability - **Serilog** and/or **OpenTelemetry** for .NET from early prototype. -- Align event names with `E9.M1 TelemetryEventSchema` in the vision doc; client batches to server or OTLP endpoint as capacity allows. +- Align event names with `E9.M1 TelemetryEventSchema` in the vision doc; client batches to server or OTLP endpoint as capacity allows. **PII, sampling, retention, reload:** [`docs/decomposition/modules/data_and_ops_policy.md`](../decomposition/modules/data_and_ops_policy.md). ## Security (proportionate to phase) @@ -115,3 +115,4 @@ Revisit this document if: - [`docs/decomposition/modules/client_server_authority.md`](../decomposition/modules/client_server_authority.md) — intents vs authoritative state; movement, camera, combat, economy ownership - [`docs/decomposition/modules/pvp_combat_integration.md`](../decomposition/modules/pvp_combat_integration.md) — E5.M1 combat engine × E6.M1 PvP eligibility (single pipeline, deny reasons) - [`docs/decomposition/modules/quest_scope_and_party.md`](../decomposition/modules/quest_scope_and_party.md) — quest state per character, party credit, turn-in, instance deferral +- [`docs/decomposition/modules/data_and_ops_policy.md`](../decomposition/modules/data_and_ops_policy.md) — content boot load vs dev reload, telemetry PII/sampling/retention, baseline events vs vision plan diff --git a/docs/decomposition/README.md b/docs/decomposition/README.md index 0ef09d6..9cbbad1 100644 --- a/docs/decomposition/README.md +++ b/docs/decomposition/README.md @@ -9,7 +9,7 @@ This workspace contains detailed planning artifacts derived from the finalized m ## Folder Layout - `epics/` - One file per epic with implementation slices. -- `modules/` - [Contract definitions](modules/contracts.md), [client vs server authority](modules/client_server_authority.md), [PvP × combat engine](modules/pvp_combat_integration.md), [quest scope × party](modules/quest_scope_and_party.md), [full module dependency register](modules/module_dependency_register.md) (all epic modules), [per-module docs](modules/module_dependency_register.md#per-module-documentation), and interface notes. +- `modules/` - [Contract definitions](modules/contracts.md), [client vs server authority](modules/client_server_authority.md), [PvP × combat engine](modules/pvp_combat_integration.md), [quest scope × party](modules/quest_scope_and_party.md), [data reload × telemetry ops](modules/data_and_ops_policy.md), [full module dependency register](modules/module_dependency_register.md) (all epic modules), [per-module docs](modules/module_dependency_register.md#per-module-documentation), and interface notes. - `milestones/` - Phase-gated implementation tracks and pass/fail checklists. ## Working Rules diff --git a/docs/decomposition/modules/E2_M2_XpAwardAndLevelEngine.md b/docs/decomposition/modules/E2_M2_XpAwardAndLevelEngine.md index 1fe06ba..a75d636 100644 --- a/docs/decomposition/modules/E2_M2_XpAwardAndLevelEngine.md +++ b/docs/decomposition/modules/E2_M2_XpAwardAndLevelEngine.md @@ -18,7 +18,7 @@ Central server-authoritative engine for classless skill progression: applying XP - Apply `XpGrantEvent` (or equivalent) per skill with validated skill references from [E2.M1](E2_M1_SkillDefinitionRegistry.md). - Resolve experience against `LevelCurve` / threshold tables. - Emit `LevelUpEvent` when thresholds are crossed. -- Support data-driven curves and reload policy agreed for prototype. +- Support data-driven curves; **reload policy:** [Data reload and telemetry operations policy](data_and_ops_policy.md#content-and-gameplay-data-reload) (boot load default; optional dev reload). ## Key contracts diff --git a/docs/decomposition/modules/E9_M1_TelemetryEventSchema.md b/docs/decomposition/modules/E9_M1_TelemetryEventSchema.md index 5127c33..34fc30a 100644 --- a/docs/decomposition/modules/E9_M1_TelemetryEventSchema.md +++ b/docs/decomposition/modules/E9_M1_TelemetryEventSchema.md @@ -11,7 +11,7 @@ ## Purpose -Artifact policy for telemetry (envelope, field naming, versioning) aligns with [contracts.md — contract kinds](contracts.md#contract-kinds) and **Telemetry** row. +Artifact policy for telemetry (envelope, field naming, versioning) aligns with [contracts.md — contract kinds](contracts.md#contract-kinds) and **Telemetry** row. **PII, sampling, retention, baseline list:** [Data reload and telemetry operations policy](data_and_ops_policy.md). Canonical taxonomy and versioning for gameplay and operations events: session, progression, economy, combat, PvP, movement desync, and integrity-adjacent signals. Provides the shared **envelope** so every epic can instrument consistently and breaking changes are explicit. diff --git a/docs/decomposition/modules/contracts.md b/docs/decomposition/modules/contracts.md index cc3f169..d225271 100644 --- a/docs/decomposition/modules/contracts.md +++ b/docs/decomposition/modules/contracts.md @@ -82,6 +82,7 @@ A module’s register **Status** can move to **Ready** when the contracts it **e - [Client vs server authority](client_server_authority.md) — intents vs state, E1/E5 defaults, deferred networking choices - [PvP and the combat engine](pvp_combat_integration.md) — E5.M1 gating with E6.M1, deny reasons - [Quest scope and party rules](quest_scope_and_party.md) — per-character quest state, party credit, turn-in +- [Data reload and telemetry operations policy](data_and_ops_policy.md) — content reload, PII, sampling, retention, baseline events - [Module dependency register](module_dependency_register.md) - [Decomposition README](../README.md) diff --git a/docs/decomposition/modules/data_and_ops_policy.md b/docs/decomposition/modules/data_and_ops_policy.md new file mode 100644 index 0000000..0b112c9 --- /dev/null +++ b/docs/decomposition/modules/data_and_ops_policy.md @@ -0,0 +1,59 @@ +# Data reload and telemetry operations policy + +Prototype-through–early-pre-production rules for **how gameplay data is loaded and refreshed** (including progression curves and content tables) and **how telemetry is collected** (PII, sampling, retention, baseline events). Aligns with [contracts.md](contracts.md), [E9.M1 — TelemetryEventSchema](E9_M1_TelemetryEventSchema.md), and [`neon_sprawl_vision.plan.md`](../../../neon_sprawl_vision.plan.md). + +--- + +## Content and gameplay data reload + +Applies to **JSON/YAML** (or equivalent) loaded by the server: [E2.M1](E2_M1_SkillDefinitionRegistry.md) / [E2.M2](E2_M2_XpAwardAndLevelEngine.md) `LevelCurve` and skill data, items, recipes, quests, zones, etc. + +| Environment / phase | Policy | +|---------------------|--------| +| **Prototype default** | Load at **process start** after **CI validation** (JSON Schema or agreed validators). No requirement for hot reload in the first milestone. | +| **Development** | Optional **file-watcher reload** or **developer-only** admin endpoint to re-read tables; must be **disabled or authenticated** in any shared/staging/prod-like deploy. Document the switch in server config. | +| **Tuning without redeploy** | Prefer **[E9.M3 — LiveBalanceControlPlane](E9_M3_LiveBalanceControlPlane.md)** and **[E2.M4](E2_M4_ProgressionPacingControls.md)** when those exist—not ad-hoc editing production disk files. | + +**E2.M2:** Replace vague “reload policy TBD” with: **boot load** for prototype; optional dev reload as above; production path goes through gated balance patches per tech stack. + +--- + +## Telemetry: privacy and payload rules + +- **Identifiers:** Use **opaque** account/session/character ids in telemetry. Do not send email addresses, real names, billing identifiers, or unhashed hardware fingerprints in the default `ClientEventEnvelope` / event catalog. +- **User-generated content:** **No** free-form chat, player names, or report narrative in high-volume analytics streams unless a **separate**, access-controlled pipeline exists with stricter retention (integrity / moderation workflows may use different storage—see [E9.M4](E9_M4_IntegrityAndAbuseResponse.md), [E8.M4](E8_M4_ChatModerationAndReporting.md)). +- **Geo / device:** Collect only what is needed for stability and fraud signals; document any addition in the event catalog with purpose. +- **Envelope:** `ClientEventEnvelope` (or successor) carries **schema version**, **sampling** metadata, and a **privacy review flag** for events that need explicit approval before shipping to general ingest. + +--- + +## Sampling and volume + +- Follow vision plan **decision-driven instrumentation**: high-frequency signals (e.g. movement samples) are **sampled or bucketed**, not logged raw at full tick rate, unless a short-lived debug cohort is enabled. +- Each **high-volume** event class gets an explicit **default sample rate** in the E9.M1 catalog when defined; ops may raise/lower within agreed bounds. +- **Noise:** Prefer fewer, well-named events over many redundant ones; use `schema_version_mismatch` / ingest errors to catch client drift (see [Epic 9](../epics/epic_09_liveops_integrity.md) Slice 1). + +--- + +## Retention and storage + +- **Retention buckets** (raw vs aggregate, per-region legal requirements) are chosen when **ingest and storage** are provisioned—not fixed in this doc. Record the chosen values in runbooks or infra config. +- **Principle:** Raw events shorter retention; rolled-up KPIs longer. Integrity / incident artifacts may follow a **separate** retention policy under E9.M4. + +--- + +## Baseline event taxonomy: source of truth + +**Product baseline (categories and examples):** [`neon_sprawl_vision.plan.md`](../../../neon_sprawl_vision.plan.md) — section **“Event Instrumentation Baseline”** (session lifecycle, progression, economy, combat, PvP, system health), plus **“KPI and Telemetry Framework”** for what decisions the events must support. + +**Machine-readable schemas:** When implemented, the versioned catalog under the repo layout in [contracts.md](contracts.md) (e.g. `contracts/schemas/telemetry/`) is the **field-level** source of truth for implementers; the vision plan remains the **intent** check. If the two disagree, **fix the catalog** and note the vision update if product meaning changed. + +--- + +## Related docs + +- [E9.M1 — TelemetryEventSchema](E9_M1_TelemetryEventSchema.md) +- [E2.M2 — XpAwardAndLevelEngine](E2_M2_XpAwardAndLevelEngine.md) +- [contracts.md](contracts.md) +- [Module dependency register — cross-cutting: data reload and telemetry](module_dependency_register.md#cross-cutting-data-reload-and-telemetry) +- [`neon_sprawl_vision.plan.md`](../../../neon_sprawl_vision.plan.md) diff --git a/docs/decomposition/modules/module_dependency_register.md b/docs/decomposition/modules/module_dependency_register.md index 5a06862..2e2a830 100644 --- a/docs/decomposition/modules/module_dependency_register.md +++ b/docs/decomposition/modules/module_dependency_register.md @@ -2,7 +2,7 @@ Tracks cross-epic dependencies and contract readiness. Every **Depends On** module ID in this table has its own row (closed graph). -**Contract** means a stable boundary with a defined artifact (Protobuf wire message, JSON Schema content file, telemetry catalog entry, or server-internal type until promoted). See **[What a contract is](contracts.md)** for kinds, repo layout, and versioning. **Authority** (who owns `PositionState`, camera, combat outcomes): **[Client vs server authority](client_server_authority.md)**. **PvP × combat:** **[PvP and the combat engine](pvp_combat_integration.md)**. **Quests × party:** **[Quest scope and party rules](quest_scope_and_party.md)**. +**Contract** means a stable boundary with a defined artifact (Protobuf wire message, JSON Schema content file, telemetry catalog entry, or server-internal type until promoted). See **[What a contract is](contracts.md)** for kinds, repo layout, and versioning. **Authority** (who owns `PositionState`, camera, combat outcomes): **[Client vs server authority](client_server_authority.md)**. **PvP × combat:** **[PvP and the combat engine](pvp_combat_integration.md)**. **Quests × party:** **[Quest scope and party rules](quest_scope_and_party.md)**. **Data reload × telemetry ops:** **[Data reload and telemetry operations policy](data_and_ops_policy.md)**. Fleshed-out scope, contracts, and integration notes live in **per-module documents** (see [Per-module documentation](#per-module-documentation)). @@ -73,6 +73,10 @@ When hostile actions can target **players**, [E5.M1 — CombatRulesEngine](E5_M1 Quest state is **per-character** by default; **party credit** and **leader-only turn-in** use [E8.M1 — PartyAndMatchAssembly](E8_M1_PartyAndMatchAssembly.md) when enabled in `QuestDef`. Solo milestones need no party service. Details: **[Quest scope and party rules](quest_scope_and_party.md)**. +### Cross-cutting: data reload and telemetry + +Gameplay **content and curves** default to **boot load** with optional **dev-only** reload; live tuning defers to **E9.M3 / E2.M4** when available. **Telemetry:** PII rules, sampling, retention principles, and **baseline event taxonomy** (vision plan + future catalog). See **[Data reload and telemetry operations policy](data_and_ops_policy.md)**. + ### Epic 7 — Quest / Faction | Module ID | Module Name | Depends On | Contract Needed | Phase Required | Status |