Extraction solves the main problem of deep research: the volume of downloaded material is several times larger than any language model’s context (the text window a model can read at once). Research Agent splits the text into manageable chunks and extracts structured data in batches — hundreds of pages turn into tables.
Chunk API
The downloaded Markdown is split into chunks with a token budget: every piece is guaranteed to fit into the model’s context. A file that is itself larger than the budget is divided into consecutive parts — visibly pieces of one source. Chunks are saved as files with source headers, so you can see which fetched pages each piece is made of.
Extract API
Extraction runs through an LLM (the AI model): each chunk is processed by its own call, which pulls structured records out of it by a given schema. The result formats are TSV (the default) and JSON. Processing is batch: a result table per chunk, with aggregation across all the chunks at once.
A long extraction can be stopped: the job has a stop button, and it asks for confirmation. The records collected by that moment are kept — the work does not have to start from scratch.
Extraction needs at least one enabled LLM service in the Services section — otherwise the operation ends with the status Failed and a services error. This is the pipeline’s only self-recoverable error: enable a service and retry.
Tokens and cost
Every extraction is accounted for: tokens, calls and cost. The Completions section shows the full history of LLM calls, so the economics of a research — 9–27 cents on a cheap model — is transparent down to the cent.
Additional APIs
- Links — extracting URLs by pattern from fetched pages (for example, collecting the Telegram channels of a niche);
- Markdown Clean — Markdown normalization: reports and documents are brought to a single look;
- Telegram Verify — checking the metadata of t.me channels: subscribers and activity by link.