ADR-0031: Export Format Versioning Strategy

Context

• ADR-0029 establishes a JSON-based export/import mechanism with a format_version field in the envelope, starting at 1.
• As the app evolves (new Room tables, renamed fields, removed entities, restructured relationships), the export schema will change. Without a disciplined versioning strategy, older exports become unimportable and newer exports become unpredictable.
• The app is pre-release and the schema is actively evolving. New tables are planned for phases 8–14 (weekly syntheses, personal visions, habits, projects). Each will add keys to the data object.
• LocusFlow is local-first (ADR-0002) with no server to mediate format upgrades. The app itself must handle every version it has ever produced.
• Users may hold backup files for weeks or months before restoring — the app cannot assume the export was created by the currently installed version.

Constraints:

• No network: the app must carry all transformation logic locally.
• Minimal code complexity: the versioning scheme must scale without requiring a full migration framework comparable to Room's.
• Human-readable exports (JSON) must remain inspectable even after transformations.
• Must support both forward compatibility (old app reading new export where possible) and backward compatibility (new app reading old export).

Decision

We will adopt a sequential version numbering scheme with registered transformations and a compatibility contract.

1. Version numbering

• format_version is a monotonically increasing integer starting at 1.
• Each version corresponds to a specific, documented export schema shape.
• The version is incremented only when the export schema changes — not on every app release.

2. Change classification

Every export schema change is classified as one of three types:

All three types increment format_version by 1. The classification determines how the transformation is implemented, not whether the version bumps.

3. Format version registry

A FormatVersionRegistry (or equivalent constant map) documents each version and what changed:

// data/export/FormatVersionRegistry.kt
object FormatVersionRegistry {
  const val CURRENT_VERSION = 1

  /**
   * Human-readable changelog for each format version.
   * Used for diagnostics and documentation; not evaluated at runtime.
   */
  val changelog: Map<Int, String> = mapOf(
    1 to "Initial export format: inbox_items, processed_items, categories, contexts, " +
            "processed_item_categories, processed_item_contexts, daily_reflections, " +
            "daily_reflection_answers, reflection_questions, settings.",
    // 2 to "Added weekly_syntheses array. Added display_name to settings.",
    // 3 to "Renamed processed_items.type → processed_items.item_type.",
  )
}

4. Compatibility contract

Importing older exports (backward compatibility — always supported)

When the app's current CURRENT_VERSION is higher than the file's format_version, the app applies sequential transformations to upgrade the export data before inserting it:

file v1 → transform_1_to_2() → transform_2_to_3() → … → current version → insert

Each transformation is a pure function that takes a JsonObject at version N and returns a JsonObject at version N+1:

// data/export/transformations/ExportTransformation.kt
fun interface ExportTransformation {
  fun transform(data: JsonObject): JsonObject
}

Transformations are registered in order and applied sequentially:

// data/export/transformations/ExportTransformationChain.kt
object ExportTransformationChain {
  private val transformations: Map<Int, ExportTransformation> = mapOf(
    // key = source version, value = transforms source → source+1
    // 1 to Transform1To2,
    // 2 to Transform2To3,
  )

  /**
   * Upgrades a raw export JsonObject from [fromVersion] to [toVersion].
   * Throws if any intermediate transformation is missing.
   */
  fun upgrade(data: JsonObject, fromVersion: Int, toVersion: Int): JsonObject {
    var current = data
    for (v in fromVersion until toVersion) {
      val transform = transformations[v]
        ?: error("Missing export transformation from v$v to v${v + 1}")
      current = transform.transform(current)
    }
    return current
  }
}

Rules for transformation functions:

• Additive changes: Insert the new key with a sensible default value (empty array, default setting value). No data loss.
• Transformative changes: Map old key/value to new key/value. Preserve data.
• Destructive changes: Remove the key. Document what is lost in the changelog.
• Each transformation is unit-tested with a snapshot of the prior version's format.

Importing newer exports (forward compatibility — best-effort)

When the file's format_version is higher than the app's CURRENT_VERSION:

The 1-version tolerance accommodates the common case where a user exports from a slightly newer build and restores on an older device. The >1 cutoff prevents silent data loss from major schema divergence.

fun validateFormatVersion(fileVersion: Int, appVersion: Int): ImportValidation {
  return when {
    fileVersion == appVersion -> ImportValidation.Exact
    fileVersion < appVersion -> ImportValidation.NeedsUpgrade(fileVersion, appVersion)
    fileVersion == appVersion + 1 -> ImportValidation.LenientForward(fileVersion)
    else -> ImportValidation.Rejected(fileVersion, appVersion)
  }
}

sealed class ImportValidation {
  data object Exact : ImportValidation()
  data class NeedsUpgrade(val from: Int, val to: Int) : ImportValidation()
  data class LenientForward(val fileVersion: Int) : ImportValidation()
  data class Rejected(val fileVersion: Int, val appVersion: Int) : ImportValidation()
}

5. When to bump the version

A version bump is required when any of the following apply to the export JSON schema:

• A new top-level key is added to data (e.g., weekly_syntheses).
• A new key is added to settings (e.g., display_name).
• An existing key in data or settings is renamed or restructured.
• An existing key is removed.
• The type or format of an existing value changes (e.g., string → object, date format change).

A version bump is not required when:

• A new optional column is added to an existing Room entity (and the export simply includes it as a new nullable JSON field within an existing array). This is additive within an existing key and handled by lenient parsing. However, if the field is non-nullable or semantically required, bump the version.
• App logic changes without affecting the export schema.
• The app_version changes (this field is informational, not structural).

6. Testing requirements

Each version transition must have:

• A fixture file: a valid JSON export at version N, stored in src/test/resources/exports/.
• A transformation test: asserts that Transform_N_to_N+1 produces a valid version N+1 export from the fixture.
• A round-trip test: asserts that exporting the current version and re-importing produces identical data.
• An end-to-end chain test: asserts that importing the oldest supported fixture through all transformations produces correct data at the current version.

src/test/resources/exports/ ├── v1-full-backup.json ├── v2-full-backup.json # (when version 2 exists) └── ...

7. Architecture integration

app/ ├── data/ │ └── export/ │ ├── FormatVersionRegistry.kt # Version constants and changelog │ ├── ExportTransformationChain.kt # Orchestrates sequential upgrades │ └── transformations/ │ ├── ExportTransformation.kt # Fun interface │ ├── Transform1To2.kt # (when version 2 exists) │ └── ... └── src/test/ └── resources/exports/ └── v1-full-backup.json # Fixture for version 1

Rationale

• Sequential integer versions over semantic versioning: Export files are single-purpose artifacts, not libraries. A simple incrementing integer is easier to compare, chain, and reason about than major.minor.patch. Semantic versioning's "compatible minor bump" concept is handled by the additive-vs-breaking classification instead.
• Transformation chain over "read any version" parser: Writing a single parser that handles every historical schema leads to tangled conditionals and is hard to test. Sequential transformations isolate each version change into a single, testable function. Each transformation only needs to understand two adjacent versions.
• Pure-function transformations on JsonObject: Operating on raw JSON (via kotlinx.serialization's JsonObject) rather than domain model classes decouples the transformation logic from the current domain model. If the domain model changes, old transformations remain valid because they operate on JSON structure, not Kotlin types.
• Fixture-based testing: Snapshot fixtures guarantee that the transformation chain is tested against real data shapes, not just the developer's assumptions about what version N looked like.
• 1-version forward tolerance: Strict rejection of all newer versions would frustrate users who update one device first and try to restore on another. A 1-version grace window with lenient parsing is a practical compromise.
• Reject >1 version ahead: Attempting to leniently parse exports from far-future versions risks silent data loss or corruption. Rejection with a clear upgrade message is safer.

Consequences

• Positive:

  • Every export ever created by the app can be imported by any future version — no data is abandoned.
  • The transformation chain is composable and testable; each step is isolated.
  • Fixture files serve as living documentation of each export schema version.
  • Forward compatibility (1-version tolerance) reduces friction for multi-device users.
  • The changelog in FormatVersionRegistry provides a quick reference for developers and diagnostics.

• Downsides:

  • Each new version requires writing a transformation function and maintaining a fixture file. This is deliberate overhead that pays off in reliability.
  • Long transformation chains (e.g., v1 → v15) may be slow for very large exports. Mitigation: transformations operate on in-memory JSON; performance is unlikely to be an issue for personal productivity data volumes. If it becomes a problem, skip-transformations (v1 → v5 directly) can be added as an optimization.
  • Destructive changes (removed keys) cause irreversible data loss during transformation. This is inherent and must be documented clearly in the changelog.

• Follow-up work:

  • Create FormatVersionRegistry, ExportTransformation, and ExportTransformationChain classes.
  • Create the initial v1 fixture file from a real export.
  • Update ImportDataUseCase (ADR-0029) to call the transformation chain before inserting data.
  • Add the forward-compatibility validation logic to the import flow.
  • Document the "how to bump the version" checklist in a developer-facing CONTRIBUTING guide.

Alternatives Considered

• Semantic versioning (major.minor.patch) — Rejected: over-engineered for a single export file. The additive/transformative/destructive classification captures intent more directly than minor vs major version semantics.
• Single "read any version" parser with conditionals — Rejected: becomes unmaintainable as versions accumulate. Each new version adds branches to every conditional. Transformation chains scale linearly.
• No forward compatibility (reject all newer versions) — Rejected: too strict for users who may update one device before another. The 1-version grace window is a practical compromise.
• Store multiple format versions in the app and let the user choose — Rejected: users should not need to understand format versions. The app always exports at CURRENT_VERSION; import handles the rest transparently.
• Binary diff/patch approach — Rejected: complex, not human-readable, and unnecessary for JSON files of this size.

Notes

• The format_version in the export envelope is independent of the Room database schema version. Room handles its own migrations via Migration classes. Export transformations handle the JSON representation. The two may evolve at different rates.
• When adding a new Room table in a future phase, the developer checklist is:
  1. Add the table to Room with a migration.
  2. Add the table's array to the export JSON.
  3. Bump format_version in FormatVersionRegistry.
  4. Write Transform_N_to_N+1 that adds the new key with an empty array default.
  5. Create a v(N+1) fixture file.
  6. Write tests for the transformation.
• Transformations should never throw exceptions for missing optional data. Use defaults and log warnings. Only throw for structurally invalid JSON (e.g., missing format_version or data keys).
• This ADR pairs with ADR-0029 and should be read together. ADR-0029 defines what is exported; this ADR defines how the format evolves safely.