If you are building a backend integration, internal tool, agent runtime, or custom frontend, this is the service you will use most often. It is the user-facing control plane for task definitions (including modular search, crawl, scrape, and API-call steps), execution lifecycle, templates, results, datasets, and stable media access.Documentation Index
Fetch the complete documentation index at: https://webscraping.titannet.io/docs/llms.txt
Use this file to discover all available pages before exploring further.
Rust snippets use ureq and serde_json (
ureq = { version = "2", features = ["json"] }, serde_json = "1"). Go snippets use the standard library only. Tab titles match Authentication and the Quickstart so your language choice stays in sync.What you will build
In this guide, you will build a minimal programmatic integration that can:- Authenticate with a bearer token
- Create a task
- Trigger execution
- Poll execution status
- Export results
Before you begin
| Requirement | Example |
|---|---|
| Task Service base URL | https://api.webscraping.titannet.io (no /api/v1 suffix in env vars below) |
| Bearer token | JWT or API key |
| Required scopes | tasks:read, tasks:write, executions:read, executions:write, data:read |
- cURL
- Go
- TypeScript
- Python
- Rust
How Task Service fits into the platform
Response model
Most JSON endpoints return a success envelope with the typed payload underdata:
success: false and an error object with code and message (see API reference overview and HTTP errors and exceptions).
When building a client, parse and check the envelope first, then read fields from data. A few routes (for example GET …/results/export) stream files instead of JSON envelopes—treat those as raw HTTP bodies.
Main endpoint groups
| Capability | Common endpoint group |
|---|---|
| Task management | /api/v1/tasks |
| Execution control | /api/v1/executions |
| Templates | /api/v1/templates |
| Result access | /api/v1/executions/:id/results |
| Dataset access | /api/v1/tasks/:id/datasets and /api/v1/executions/:id/datasets |
| Media access | /api/v1/executions/:id/media |
Build a minimal client
These helpers assumeTITAN_API_URL is the origin (for example https://api.webscraping.titannet.io) and append /api/v1.
- cURL
- Go
- TypeScript
- Python
- Rust
Create a task programmatically
The table below describes the classic extraction shape most teams start with: known URLs plus an objective and output schema. Titan also supports modular and multi-step tasks whereurls, output_schema, and other fields are expressed per action or inside an execution_plan instead of only at the top level. For those patterns, see Action types overview and the Task Service operations in the API Reference tab.
| Field | Required (classic) | Meaning |
|---|---|---|
name | Yes | Human-readable task label |
urls | Yes | Target URLs |
objective | Yes | What the run should accomplish |
output_schema | Yes | JSON schema describing the structured result |
execution_type | Yes | single or scheduled |
schedule | For scheduled tasks | Cron string such as 0 * * * * |
Example: create a single-run task
- cURL
- Go
- TypeScript
- Python
- Rust
Example: create a scheduled task
- cURL
- Go
- TypeScript
- Python
- Rust
Trigger and monitor execution
| Pattern | Endpoint | Best for |
|---|---|---|
| Run saved task | POST /api/v1/tasks/:id/run | Standard user workflows |
| Trigger execution directly | POST /api/v1/executions | Execution-oriented integrations |
Run a saved task
- cURL
- Go
- TypeScript
- Python
- Rust
ExecutionResponse in data; use data.id as the execution UUID for later GET /api/v1/executions/:id calls.
Poll execution state
- cURL
- Go
- TypeScript
- Python
- Rust
data.status from the envelope on each poll.
Export results
When the execution reaches a terminal status such ascompleted, switch from execution monitoring to result access.
Result metadata (JSON envelope)
- cURL
- Go
- TypeScript
- Python
- Rust
Export as JSON file (raw body, no envelope)
- cURL
- Go
- TypeScript
- Python
- Rust
Download as CSV
- cURL
- Go
- TypeScript
- Python
- Rust
A full client flow
Copy the Build a minimal client snippet for your language, then run the flow below (sametitanRequest / titan_json names where applicable).
- cURL
- Go
- TypeScript
- Python
- Rust
Best practices
| Practice | Why it matters |
|---|---|
| Keep one reusable task per workflow | Produces cleaner execution history |
| Parse the response envelope consistently | Avoids ad hoc response handling |
| Poll executions, then switch to result endpoints | Keeps runtime and result logic separate |
| Use API keys for automation | Avoids browser-session coupling |
| Keep output schemas stable | Makes downstream consumption simpler |
Troubleshooting
| Problem | What to check |
|---|---|
Task creation returns 400 | Invalid body or output schema |
Execution never moves past queued | Worker availability or scheduler health |
| Export endpoints fail | Execution may not be completed yet |
| Media does not render in frontend | Use the correct export mode or media download path |
Next steps
- Create and manage tasks
- Monitor and control executions
- Download results and access media
- HTTP errors and exceptions — non-standard error bodies and rate limits