---
title: "Data Sources"
id: "1800"
type: "page"
slug: "sources"
published_at: "2026-09-20T21:51:53+00:00"
modified_at: "2026-09-21T01:36:28+00:00"
url: "https://xedant.com/agents/data/docs/sources"
markdown_url: "https://xedant.com/agents/data/docs/sources.md"
excerpt: "A data source is what you can query: tables with fields from which Analytics Agent…"
---

# Data Sources

[https://xedant.com/agents/data/docs/sources.md](https://xedant.com/agents/data/docs/sources.md)

A data source is what you can query: tables with fields from which Analytics Agent builds reports and the agent checks results. In Data Agent, sources need no manual setup: they appear on their own as soon as a ready database shows up in the data.

## Automatic discovery

Every Lakehouse folder with a SQLite database file at its root becomes a source named `{project}_{source}` — for example, `market_prices`. The source points at the folder’s newest database and opens read-only: scripts write the data, queries never change it. If the database is deleted, the source disappears at the next rescan.

## Data freshness

A rescan happens after every script run finishes in the source’s folder, and also when an unknown name is looked up. So fresh data that a script just collected is available for queries immediately — nothing needs a manual “refresh”.

## Overrides and external databases

Sometimes a discovered source needs changing: point it at a different database, fix its title or settings — or connect a database that does not live in the Lakehouse. JSON files in the overrides folder (inside the `/data` volume) do this. One broken file does not break the registry: it is skipped with a warning, and the other sources keep working.

## Querying the data

Today SQLite sources are queryable. Only read-only queries (SELECT) are allowed: the agent checks data after loading, and you use it in your analysis. The table schema is available right away — you can see which fields and types sit in every table.

## The interface is a read-only dashboard

The Data Sources section lists the sources, each one’s details and its table count. Registration is the agent’s job — sources cannot be created or edited in the interface, and so cannot be accidentally broken either.

## The point in plain words

Connecting a database used to mean configuring connections, access rights and formats. Here, none of that: a script builds a database in the Lakehouse, and the source is ready to use. One principle — “put a database in, get a source” — covers new tasks too.

## Where a source’s material comes from

A source appears where a ready database lies, and a database is built by a script. The script usually takes its material directly — from APIs, files or exports. But when bulk collection is needed — hundreds of competitor pages, dozens of search queries, thousands of short texts — harvesters step in: they search the internet, download pages and process texts with models. HTML pages are saved as clean Markdown text, with the original page kept beside it.

Harvesters are separate services: configured once in the Harvesters section, launched by the agent. That way material is gathered fast and in volume, and the scripts turn it into Lakehouse data — and eventually into the sources reports are built on.

See also: [Harvesters](/agents/data/docs/harvesters)
, [Lakehouse](/agents/data/docs/lakehouse)
, [Runs](/agents/data/docs/runs)
, [Data for Analytics Agent](/agents/data/docs/for-analytics)
.

[← Back to the documentation index](/agents/data/docs)