Work with data

A connection, called a datasource in the API, holds what NSTACK needs to reach one of your systems: a connector and its settings for each environment. Admins create most connections in Studio. Your code can do the same, and the pattern is the same: pick a connector, send settings per environment, test them, then list what the system can read.

H="Authorization: Bearer $NSTACK_SESSION"
DATA="$NSTACK_BASE/data/api/v1/organizations/$NSTACK_ORG"

curl -s "$DATA/data-connectors" -H "$H"
curl -s "$DATA/datasources" -H "$H"

const data = `${base}/data/api/v1/organizations/${org}`;
const get = async (url: string) => (await fetch(url, { headers: auth })).json();

const connectors = await get(`${data}/data-connectors`);
const sources = await get(`${data}/datasources`);
console.log(connectors.items.map((c) => c.name), sources.items.length);

data = f"{base}/data/api/v1/organizations/{org}"

connectors = requests.get(f"{data}/data-connectors", headers=auth).json()
sources = requests.get(f"{data}/datasources", headers=auth).json()
print([c["name"] for c in connectors["items"]], len(sources["items"]))

• Settings per environment. DEV, UAT, and PROD each hold their own host, database, and key. A PROD setting needs the permission to manage production, and Studio asks for a typed confirmation.
• Keys go in once. Every read of a connection returns a mask. Tests and runs use the key inside your instance.
• Files for settings. A root certificate or a client key goes through POST …/datasources/{datasource_id}/file with env in the query; the answer is the storage key you put in the settings.
• HTTP APIs. Upload an OpenAPI spec with PATCH …/data-processing/{connector}/catalog/file and the HTTP connector reads its endpoints as a catalog.

A dataset is a query on a connection, or a template applied to it. NSTACK does not copy your tables. When an agent, an app, or an automation reads the dataset, the data service runs its query against your system in that environment and answers the rows. Three calls take a query from draft to saved.

1. Prepare. POST …/data-processing/{connector}/datasets/prepare parses the query and answers its placeholders, such as ${token}.
2. Test. POST …/data-processing/{connector}/datasets/test runs it with env_id, limit up to 1,000, and offset. Every test is a dataset_run on the run record.
3. Save. POST …/datasources/{datasource_id}/datasets creates the dataset. PUT updates it, PATCH {"archive": true} retires it, and a delete stays restorable for 30 days.

An ontology template names an object type and its fields: a household, an account, a claim. Datapoints map a dataset's fields onto a template, so every agent and app reads the same shape no matter which system the rows came from.

Templates your organization creates need the permission to manage the ontology. Platform templates ship with NSTACK and take a platform administrator to change.

One call reads a dataset in an environment. Without limit you get the first 100 rows; page with offset. The answer carries the rows as your system returned them under original, and their mapped form under mapped.

curl -s "$DATA/datasources/$DATASOURCE_ID/datasets/$DATASET_ID/get-data?env_id=$ENV_ID&limit=200&offset=0" -H "$H"

const url = `${data}/datasources/${datasourceId}/datasets/${datasetId}/get-data`
  + `?env_id=${envId}&limit=200&offset=${offset}`;
const page = await (await fetch(url, { headers: auth })).json();
for (const row of page.original.rows) upsertIntoYourStore(row);
if (page.original.row_count === 200) offset += 200;

page = requests.get(f"{data}/datasources/{datasource_id}/datasets/{dataset_id}/get-data",
                    params={"env_id": env_id, "limit": 200, "offset": offset}, headers=auth).json()
for row in page["original"]["rows"]:
    upsert_into_your_store(row)
if page["original"]["row_count"] == 200:
    offset += 200

{
  "connector": "pgsql",
  "datasource_id": "3f9a0b1c-2d3e-4f5a-8b6c-7d8e9f0a1b2c",
  "dataset_id": "6c7d8e9f-0a1b-4c2d-9e3f-4a5b6c7d8e9f",
  "env_id": "9a8b7c6d-5e4f-4a3b-9c2d-1e0f9a8b7c6d",
  "original": { "ok": true, "rows": [{ "id": 1, "status": "open", "updated_at": "2026-09-30T16:04:00Z" }], "row_count": 1, "columns": [{ "…": "…" }], "page": { "…": "…" } },
  "mapped": null
}

• Who else reads it. GET …/usage?ref_type=dataset&ref_id=… lists the agents and automations that use a dataset, and whether that blocks archiving it.
• On the record. A dataset test lands on the run record as a dataset_run; an agent's read shows as a tool:query_<dataset>#n step on the agent's run.

Agents search documents and long text through a knowledge index built per dataset and per environment. You choose the fields, request a sync, and test retrieval with a prompt before an agent depends on it.

An agent version with the dataset assigned searches the index of the run's environment through its built-in search_knowledge tool. Each search is a step on the run record, so you can see what the agent found and where.

Keep DEV pointed at a copy of your data and promote the same dataset definitions to UAT and PROD by environment. The dataset is one object; only its environment settings differ.

• In design Object routes. Reading and writing the records your datasets map to as ontology objects, with object events on nstack.object.v1.
• In design Machine access to REST. Your pipelines reading datasets with a machine credential, with no member session.