Live Database Ingest
Use this skill when the ingest work unit contains raw files under
raw-sources/<connectionId>/live-database/<syncId>/.
Workflow
- Read the table JSON file listed in the work unit.
- Read
connection.json to understand the snapshot metadata.
- Read
foreign-keys.json when the table has a foreign key or when joins are
needed for the semantic-layer source.
- Create or update one semantic-layer source for the table with
sl_write_source.
- Use the physical table name from the raw JSON as the source
table field.
- Preserve database comments as
descriptions.db on tables and columns.
- Add joins only when the foreign key index names both sides.
- Write wiki pages only for durable business meaning that is present in table
or column comments.
- Run
sl_validate for the table source before the work unit completes.
Sample values come from the scan record; do not invent values not present in
relationship-profile.json.
Identifier Verification Protocol
Before writing a wiki page or SL source on any topic:
discover_data({query: "<topic>"}) - see what wikis, SL sources, and raw
tables already exist. Prefer updating existing pages over creating new ones.
Before emitting any schema.table or schema.table.column into a wiki body,
SL source, tables: frontmatter, sl_refs, or emit_unmapped_fallback:
entity_details({connectionId, targets: [{display: "<identifier>"}]}) -
confirm the identifier resolves; inspect native types, FK/PK, and
sampleValues.- For literal values from the source, such as status codes or plan tiers,
check whether they appear in
entity_details sampleValues for the relevant
column. If sampleValues is short or the sample may have missed real values,
run a sql_execution probe with the same warehouse connection id:
sql_execution({connectionId, sql: "SELECT DISTINCT <col> FROM <ref> LIMIT 50"}).
- If the candidate identifier still does not resolve, do one of:
- Use
sql_execution({connectionId, sql: "SELECT 1 FROM <ref> LIMIT 0"}).
If it errors, the identifier is fictional.
- Wrap the identifier in
[unverified - from <rawPath>] in the wiki body,
citing the exact raw path that mentioned it.
- When recording
emit_unmapped_fallback with no_physical_table, include
the failing probe error in clarification.
- Never copy
<schema>.<table> placeholder strings from these instructions
into output.
Source shape
For a raw table with this shape:
{
"name": "orders",
"db": "public",
"columns": [
{ "name": "id", "type": "integer", "nullable": false, "primaryKey": true }
]
}
Write a semantic-layer source with this shape:
name: orders
table: public.orders
grain: id
columns:
- name: id
type: number
Use string, number, time, or boolean for column types. When a database
type is ambiguous, use string.
Boundaries
The raw snapshot is structural evidence. Do not invent measures, segments,
business definitions, or joins that are not present in the snapshot files.