---
title: "Scripts"
id: "1796"
type: "page"
slug: "scripts"
published_at: "2026-09-20T21:51:53+00:00"
modified_at: "2026-09-21T01:36:28+00:00"
url: "https://xedant.com/agents/data/docs/scripts"
markdown_url: "https://xedant.com/agents/data/docs/scripts.md"
excerpt: "The scripts folder is the heart of Data Agent. It is a plain folder of…"
---

# Scripts

[https://xedant.com/agents/data/docs/scripts.md](https://xedant.com/agents/data/docs/scripts.md)

The scripts folder is the heart of Data Agent. It is a plain folder of plain files on your server: one folder per data source. All scripts are written and maintained by the agent — you read and run them; there is nothing to edit by hand. The interface reads the folder directly, so the agent’s changes are visible instantly.

## Folder structure

Every data source is a folder `scripts/{project}/{source}/` that holds:

- `README.md` — the source’s mandatory description: where the data comes from, what is inside, which fields. It also becomes the folder’s page in the interface;
- `extract.py` — the export script: pulls data from the external service and saves it as Parquet;
- `load.py` — the build script: shapes the exported data into a single SQLite database for queries and visualization;
- optionally `download.py` — downloading files (when the export goes through files), `test.py` — checks, `PLAN.md` — notes.

## The folder’s rules

- Parquet is always the raw data: what the export script pulled is never touched;
- SQLite is the derived database: rebuilt by the `load.py` script on every run;
- run logs are written next to the data in the Lakehouse, not in the scripts folder;
- passwords and keys live only in the `.env` file, never in the code: they reach the process as environment variables on every run;
- a new library is added to `requirements.txt` — the container installs it by itself on startup;
- keys for external services are better set up as harvesters rather than stored in code: the agent then takes a ready-made service, and the secrets live in one place.

## The tree in the interface

The Scripts section shows folders and files as a tree. A folder’s page is its README description, formatted like a document (title, description, notes). A file’s page is the code with syntax highlighting; for Markdown, a finished document. Breadcrumbs at the top always show where you are.

## Search, unread marks, icons

- search across folders and files works as you type — results appear immediately;
- unread changes stay highlighted until you open them;
- every folder can be given an icon — that is the section’s only write besides read marks; the interface never touches the code itself.

## Live updates

File watching is built into the application: the agent fixes a script — the fix appears on screen instantly, without a reload. The same for new folders, descriptions and notes: you always see the current state of the folder.

## Actions from the page

- a permanent link to a folder or file — easy to share;
- the “Ask the agent” button — opens a chat with the file’s path already pasted in: ask for a fix or an explanation;
- for runnable files — a run button and the run history right on the page.

## Distribution and moving

The scripts folder is self-contained: copy it, pack it into an archive, or keep it as its own Git repository — the agent then works right on the repository. Moving to another server is simply moving the folder.

See also: [Harvesters](/agents/data/docs/harvesters)
, [Runs](/agents/data/docs/runs)
, [Scheduler](/agents/data/docs/scheduler)
, [Chat with the Agent](/agents/data/docs/chat)
.

[← Back to the documentation index](/agents/data/docs)