Collections

Collections are structured datasets: tables of records that give your workflows durable, long-term memory. Where the Knowledge Base stores documents for AI search, collections store rows of data, for example the latest price per symbol, a daily history of scores, or an append-only event log.

Collections belong to your whole workspace, not to a single project. That is deliberate: one project can run the ingestion workflows that fill a collection while another project reads from it.

Open them from Knowledge Base in the left navigation, then the Collections tab.

The three kinds

Every collection has a kind that decides what a write does:

Kind	Behavior	Use it for
Latest	One record per key, updated in place on every write.	Current state: prices, profiles, latest scores.
Series	A point appended per write, kept over time.	History and charts.
Log	Append-only entries with no key.	Audit trails and events.

Pick Latest when you only care about "what is it now". Pick Series when you need "what was it over time". Pick Log when entries are independent events that never update each other.

Key fields

Latest and series collections need at least one key field. The values of the key fields identify a record. For example, key fields strategyId, symbol mean one row per strategy per instrument:

• In a latest collection, writing a record with the same key values replaces the existing row.
• In a series collection, each key gets its own timeline of points.
• Log collections have no key fields; every write appends a new entry.

In the record browser, composite keys display their parts separated by " / ".

Creating a collection

Creating, deleting, and clearing collections requires the workspace admin role. Other members can browse them.

1. On the Collections tab, click New collection.
2. Enter a Name (for example "Latest quotes"). An Id is suggested from the name; you can edit it. The id must be lowercase letters, digits, hyphens, and underscores. Workflows reference this id, so pick something stable like quotes_latest.
3. Choose the Kind (Latest, Series, or Log).
4. For latest and series, enter the Key fields (comma separated).
5. Optionally add a Description.
6. Optionally set Retention (see below).
7. Click Create collection.

A workspace can hold up to 200 collections.

Retention and the janitor

Retention keeps collections from growing forever:

• Keep raw for N days: raw records expire automatically N days after their timestamp. Leave it empty to keep records indefinitely.
• Roll up to daily (series collections only), with Keep rollups for N days: before raw points age out, a nightly maintenance job (the janitor, which runs in the early morning UTC) summarizes each completed day into one point per key per day. Charts keep long-range history at a fraction of the storage.

Rolled-up records are labeled with a rollup badge (including how many raw points were combined) in the record browser.

You can change retention later: open the collection's record browser and use the retention editor (Raw days, Daily rollup, Rollup days, then Save retention). Retention on platform-managed collections is fixed.

Browsing records

Click a collection card (or its Browse button) to open the record browser. Each card also shows the collection's key fields, retention, when it was last written, and which workflow last wrote to it.

In the record browser you can:

• Filter by key prefix: type the start of a key (for a collection keyed by symbol, typing AA matches AAPL, AAL, and so on) and press Enter or click Apply.
• Rows: choose how many rows to load (25, 50, 100, 250, or 500).
• Read the table: a Key column, a Timestamp column, and up to eight field columns detected from the returned records.

If more records exist than the row count, a notice tells you the listing is truncated; narrow the key prefix or raise the row count. Queries are capped at 1,000 records per request.

If a collection stores customer-submitted task responses, the browser shows a notice that it contains customer-submitted responses and is kept private to this workspace. These collections can never be exposed to the customer portal.

Clearing, deleting, and Nuke all

Three admin actions remove data, with different blast radii:

Action	Where	What it does
Clear records	Eraser icon on a collection card (tooltip: "Nuke records (keeps the collection)")	Deletes every record but keeps the collection itself, including its id, kind, keys, and retention. Works on platform-managed collections too.
Delete	Trash icon on a collection card (not shown for platform-managed collections)	Removes the collection and every record it holds.
Nuke all	Toolbar button at the top of the tab	Deletes every collection you created (with all records) and empties platform-managed collections without removing them.

Each action confirms before running:

• Clear records asks: Nuke every record in "<name>" (<id>)? The collection stays, but all of its records are deleted. This cannot be undone.
• Delete asks: Delete "<name>" (<id>)? Every record it holds is deleted too. This cannot be undone.
• Nuke all requires you to type NUKE into a prompt before anything happens.

danger

None of these actions can be undone. Workflows that write to a cleared collection will simply refill it on their next run; a deleted collection must be recreated first (or recreated automatically by a workflow that writes with the auto-create option).

How workflows read and write collections

Memory nodes on the canvas

The workflow designer's Data & Memory palette group has three nodes:

• Data Sync (source → collection): pulls rows from a connected data source into a collection on a schedule, with no AI model in the path (see Syncing a data source).
• Collection (read / write): binds a specific collection to the flow. In write mode, records produced upstream are stored in it; in read mode, agents connected to it can query it without needing to know the collection id.
• Vault (agent notes): files agent notes into the Knowledge Base, not into a collection.

The dataset tools

Under the hood, agents and steps use two built-in tools:

• dataset.write stores records. In a latest collection repeat writes update rows in place; in a series collection each write appends a point. Records can be bare objects (for example {"symbol": "AAPL", "price": 100}); the key is derived from the collection's key fields. A write can pass ensure to create the collection on first use instead of failing. Writes are batched (up to 1,000 records per request, chunked automatically for larger batches).
• dataset.query reads records, filtered by exact key, key prefix, field comparisons (equals, greater/less than, in a list), and a time range for series data. Results default to 100 records and are capped at 1,000, so agents ask for what they need rather than everything.

There is also dataset.query_bound, which reads a compact evidence view from every collection bound to an agent through Collection nodes in read mode, in one call.

Two guardrails apply to all of this: a workflow can only ever reach its own workspace's collections (the workspace identity comes from the execution context, never from tool input), and malformed records in a batch are skipped and reported individually instead of failing the whole write.

Every successful write also stamps provenance, which is why collection cards show Last written and By workflow.

Platform-managed collections

Some collections are created and managed by Faberiq itself. Their cards show a lock icon with Managed by Faberiq instead of a created date. Current examples are instruments (the tradable instrument universe), signals_latest, and signals_history. They are registered lazily, the first time something writes to them, so a workspace that never uses them carries no empty entries.

You cannot delete a platform-managed collection or change its retention, but you can clear its records, and Nuke all empties it while keeping it registered.

Syncing a data source into a collection

The most common way to fill a collection is the Data Sync node, packaged as the Sync data source to a collection recipe (sync-source-to-collection) in the recipe catalogue. The shape is simple: a Schedule Trigger fires, the Data Sync node calls a table on one of your connected sources, and the rows are stored directly in the target collection. No agent runs, so a cycle costs source API quota only.

In the node you pick:

• The source table on a connected data source (for example daily price history on a market-data connector).
• The target collection.
• For symbol-based sources, which companies to cover, three ways: type tickers directly, tick a universe table such as an index (S&P 500, Nasdaq, Dow) or screener, or point it at a Google Sheets tab that holds your list.
• Optionally a moving window (keep only the last N minutes, hours, or days of source history): older rows are filtered out on fetch and pruned from the collection.
• Optionally a field mapping to reshape source rows before storing them.

Syncs are idempotent: rows are upserted by key and source timestamp, so running the same sync repeatedly refreshes data instead of duplicating it. Transient provider errors (rate limits, timeouts) are retried automatically, and the run summary reports how many symbols returned data, how many rows were written, and anything skipped.

Use one sync workflow per feed so each keeps its own cadence: prices every few minutes, fundamentals daily, earnings weekly. An analysis workflow in any project can then read whatever the syncs write.

Limits at a glance

Limit	Value
Collections per workspace	200
Collection id	1 to 64 characters: lowercase letters, digits, hyphens, underscores
Collection name	Up to 120 characters
Description	Up to 500 characters
Key fields	Up to 8
Records per write request	1,000 (larger batches are chunked automatically)
Records per query	Default 100, maximum 1,000
Retention values	Up to 3,650 days for raw and rollup retention

Related pages

• Knowledge Base: document storage and AI search, the other half of this page.
• Workflows and Agents: building the flows that read and write collections.
• Connectors: connecting the data sources that syncs pull from.