From Metadata to Database

Henceforth metadata is the source of truth. The database is a compiled projection. You author JSON5 files that describe your data model, and the runtime compiles those files into PostgreSQL tables, views, indexes, foreign keys, and triggers. This page explains the authoring loop and what happens at each step.

The Authoring Loop

Working with Henceforth metadata follows a three-step cycle:

1. Edit – add, change, or remove JSON5 metadata files
2. Plan – preview the resulting database changes
3. Apply – execute the changes in PostgreSQL

Step 1: Edit Metadata

Metadata lives in a configured directory (e.g., dev/metadata/). To make a change, you edit the JSON5 files directly:

• Add a new field file to add a column
• Change required: true on a scalar field to make it mandatory
• Add a new object file to create a new table
• Add a lookup field file to create a relationship between objects
• Add a rollup field file to create a trigger-maintained aggregate
• Add an object with extends to introduce inheritance

For file naming conventions, see Metadata File Layout.

Step 2: Plan

POST /plan loads the metadata, compiles the desired schema, compares it to the live database, and returns a dry-run result. No DDL is executed.

The response tells you what would change:

{
  "status": "ok",
  "changes": [
    { "action": "createTable", "table": "hf_core__opportunity", "risk": "safe" },
    { "action": "addColumn", "table": "hf_core__opportunity", "column": { "name": "name" }, "risk": "safe" },
    { "action": "createIndex", "table": "hf_core__opportunity", "index": "idx_hf_core__opportunity__account_id", "risk": "blockingOnLargeTable" },
    { "action": "createTrigger", "table": "hf_core__contact", "trigger": "trg_rollup_contact_count", "risk": "safe" }
  ],
  "summary": {
    "tablesCreated": 1,
    "columnsAdded": 3,
    "columnsChanged": 0,
    "columnsDropped": 0,
    "columnsTypeChanged": 0,
    "constraintsChanged": 0,
    "indexesCreated": 1,
    "indexesReplaced": 0,
    "unchanged": 0
  },
  "diagnostics": []
}

Each change includes a risk level: safe, requiresBackfill, blockingOnLargeTable, or destructive.

By default, destructive operations (column drops, unsafe type narrowings) are not included in the plan – they appear as diagnostic warnings instead. To see what a destructive apply would do, pass {"destructive": true} as the request body.

Planning is the safe way to understand the effect of a metadata edit before committing to it. Review the changes and diagnostics before proceeding.

Step 3: Apply

POST /apply performs the same load, compile, and diff as /plan, then executes the generated DDL inside a database transaction. If any statement fails, the entire transaction rolls back.

The response has the same shape as /plan, plus confirmation that the changes were applied. If there are no changes (the database already matches the metadata), the status is "unchanged".

Apply accepts an optional JSON body with options:

• destructive (default false) – allow destructive operations such as dropping orphaned columns or narrowing column types
• vault (default true) – before destructive operations, preserve the affected column data in operations.vault so it can be restored later

Apply is idempotent: it computes a checksum of the desired schema and skips execution when the database already matches. Each successful apply is recorded in operations.migrations for audit and history.

What Gets Created in PostgreSQL

The compiler produces these artifact types:

Artifact	When it is created
Table	One per object. Holds user-defined field columns. Primary key references hf_object (or the parent table for inherited objects).
View	One per object. Joins hf_object with the object table(s) for a unified read surface. For inherited objects, the view joins the full table chain.
Column	One per scalar field (stored). One per same-row formula field (virtual generated). One per lookup field (UUID + FK). One per rollup field (trigger-maintained stored column).
View expression	One per cross-object formula field (correlated subquery in the view).
Index	On indexed scalar fields, on lookup FK columns, and expression indexes on indexed formulas.
Foreign key	On every lookup column: referencing the target table (single-target), or hf_object (multi-target). On inherited object tables: id referencing the parent table.
Trigger + function	On immutable lookup fields (mutable: false), guarding against FK changes. On rollup fields, maintaining aggregate values on insert/update/delete of child records.

How Views Work

Each object gets a view that joins hf_object with the object table:

CREATE OR REPLACE VIEW "hf_core__account__view" AS
  SELECT o."id", o."object_type", o."created_at", o."updated_at",
         t."name", t."industry", t."active", t."contact_count"
  FROM "hf_object" o
  JOIN "hf_core__account" t ON t."id" = o."id";

For inherited objects, the view joins the full table chain from hf_object through each ancestor to the leaf:

CREATE OR REPLACE VIEW "hf_core__individual__view" AS
  SELECT o."id", o."object_type", o."created_at", o."updated_at",
         t0."email",
         t1."first_name", t1."last_name"
  FROM "hf_object" o
  JOIN "hf_core__party" t0 ON t0."id" = o."id"
  JOIN "hf_core__individual" t1 ON t1."id" = o."id";

Cross-object formula fields appear as computed expressions in the view’s SELECT clause (correlated subqueries), not as columns on the object table.

The data API reads from these views, which means every query returns both system fields and user-defined fields (including inherited fields and cross-object formulas) in a single result set. Writes go to the object table(s) and hf_object separately.

Diagnostics

The compiler may emit diagnostics alongside the schema diff. These appear in the diagnostics array of the /plan and /apply responses, even when there are no changes to apply.

Diagnostics surface modeling feedback – for example, a warning that required: true with onParentDelete: "setNull" was auto-corrected to restrict, or an informational note that a new subtype widens existing lookups targeting its base type. For the full diagnostic catalog, see Diagnostics.

What’s Next

With metadata applied and the database schema in place, the data API serves CRUD endpoints against the compiled schema – see Create records through the API and the other data API How-to recipes.

For the internal mechanics of the loader, compiler, and diff engine, see Metadata and Apply Pipeline.

Reference

• Metadata File Layout – directory structure, filename conventions, and parsing rules
• Diagnostics – severity levels, diagnostic catalog, and how diagnostics surface in /plan and /apply responses