Skip to main content

Documentation Index

Fetch the complete documentation index at: https://webscraping.titannet.io/docs/llms.txt

Use this file to discover all available pages before exploring further.

This guide shows the shortest useful Titan workflow: authenticate, create a task, run it once, poll the execution, and export the result. It uses the classic pattern—URLs, objective, and output schema—which matches most product monitors and extraction jobs. Titan also supports modular actions and multi-step tasks; see What is Titan Web Scraping? and Action types overview when you need search, crawl, or API steps in the same model. It is written as a practical API walkthrough, not a conceptual tour.
Rust snippets use ureq and serde_json. Tab titles (cURL, Go, TypeScript, Python, Rust) match Authentication and Use the Task Service API so your language choice stays in sync across pages.

What you will do

By the end of this guide, you will:
  1. Configure your API connection
  2. Create a task
  3. Start an execution
  4. Check execution state
  5. Export the result

Before you begin

RequirementNotes
Task Service URLhttps://api.webscraping.titannet.io (set as TITAN_API_URL without /api/v1)
Bearer tokenJWT or API key with tasks:write, executions:write, executions:read, and data:read (see Authentication and API keys)
JSON-capable HTTP clientcurl, Postman, or your own code
export TITAN_API_URL="https://api.webscraping.titannet.io"
export TITAN_TOKEN="your-bearer-token"

Quickstart architecture

The sequence below is schematic: real responses wrap payloads in { "success": true, "data": … } except for file exports, which return raw bytes (see Step 4).

Step 1: create a task

curl -sS -X POST "$TITAN_API_URL/api/v1/tasks" \
  -H "Authorization: Bearer $TITAN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My first Titan task",
    "urls": ["https://example.com/product/123"],
    "objective": "Extract the product name, price, and description",
    "output_schema": {
      "type": "object",
      "properties": {
        "product_name": { "type": "string" },
        "price": { "type": "string" },
        "description": { "type": "string" }
      }
    },
    "execution_type": "single"
  }'
Successful responses use the standard Titan envelope. The task identifier is data.id:
{
  "success": true,
  "data": {
    "id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "name": "My first Titan task",
    "status": "active",
    "execution_type": "single",
    "created_at": "2025-01-01T00:00:00Z"
  }
}

Capture TASK_ID

export TASK_ID="$(
  curl -sS -X POST "$TITAN_API_URL/api/v1/tasks" \
    -H "Authorization: Bearer $TITAN_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
      "name": "My first Titan task",
      "urls": ["https://example.com/product/123"],
      "objective": "Extract the product name, price, and description",
      "output_schema": {
        "type": "object",
        "properties": {
          "product_name": { "type": "string" },
          "price": { "type": "string" },
          "description": { "type": "string" }
        }
      },
      "execution_type": "single"
    }' | jq -r '.data.id'
)"
echo "$TASK_ID"

Step 2: run the task

curl -sS -X POST "$TITAN_API_URL/api/v1/tasks/$TASK_ID/run" \
  -H "Authorization: Bearer $TITAN_TOKEN"
The run endpoint returns 202 Accepted with an ExecutionResponse inside data. The execution UUID is data.id (not execution_id):
{
  "success": true,
  "data": {
    "id": "e5f6a7b8-c9d0-4123-ef45-678901abcdef",
    "task_id": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
    "status": "queued",
    "created_at": "2025-01-01T00:01:00Z"
  }
}

Capture EXECUTION_ID

export EXECUTION_ID="$(
  curl -sS -X POST "$TITAN_API_URL/api/v1/tasks/$TASK_ID/run" \
    -H "Authorization: Bearer $TITAN_TOKEN" | jq -r '.data.id'
)"
echo "$EXECUTION_ID"

Step 3: poll execution status

curl -sS "$TITAN_API_URL/api/v1/executions/$EXECUTION_ID" \
  -H "Authorization: Bearer $TITAN_TOKEN"
The JSON body is again the standard envelope; poll data.status until it reaches a terminal value. Typical statuses include:
StatusMeaning
queuedExecution was created and is waiting for work
runningWork is actively being processed
pausedExecution was paused
completedExecution finished successfully
completed_with_errorsFinished with partial failures
failedExecution finished with an error
cancelledExecution was stopped
Keep polling until the execution reaches a terminal state.

Step 4: export the result

Once the execution is completed (or completed_with_errors when data still exists), export the dataset. GET /api/v1/executions/:id/results/export returns a file attachment (raw JSON, NDJSON, or CSV per the format query parameter). It does not use the { success, data } envelope. The default format is json.

Export JSON to a file

curl -sS "$TITAN_API_URL/api/v1/executions/$EXECUTION_ID/results/export" \
  -H "Authorization: Bearer $TITAN_TOKEN" \
  -o results.json

Download CSV

curl -sS "$TITAN_API_URL/api/v1/executions/$EXECUTION_ID/results/download" \
  -H "Authorization: Bearer $TITAN_TOKEN" \
  -o results.csv
GET /api/v1/executions/:id/results (metadata) does use the standard success envelope; see the Task Service API group in the API Reference tab (OpenAPI) for the exact data shape.

Step 5: access media if present

If your output schema includes media fields, those fields will contain Titan-managed URLs. Use the execution media download endpoint when you want to resolve those assets directly.

End-to-end (one script)

set -euo pipefail
TASK_ID="$(
  curl -sS -X POST "$TITAN_API_URL/api/v1/tasks" \
    -H "Authorization: Bearer $TITAN_TOKEN" \
    -H "Content-Type: application/json" \
    -d '{
      "name": "Quick test",
      "urls": ["https://example.com"],
      "objective": "Extract the page title",
      "output_schema": { "type": "object", "properties": { "title": { "type": "string" } } },
      "execution_type": "single"
    }' | jq -r '.data.id'
)"
EXECUTION_ID="$(
  curl -sS -X POST "$TITAN_API_URL/api/v1/tasks/$TASK_ID/run" \
    -H "Authorization: Bearer $TITAN_TOKEN" | jq -r '.data.id'
)"
while true; do
  ST=$(curl -sS "$TITAN_API_URL/api/v1/executions/$EXECUTION_ID" \
    -H "Authorization: Bearer $TITAN_TOKEN" | jq -r '.data.status')
  echo "status=$ST"
  echo "$ST" | grep -qE 'completed|failed|cancelled|completed_with_errors' && break
  sleep 3
done
curl -sS "$TITAN_API_URL/api/v1/executions/$EXECUTION_ID/results/export" \
  -H "Authorization: Bearer $TITAN_TOKEN" -o results.json

Troubleshooting

ProblemCheck first
401 Unauthorized or 403 ForbiddenToken validity and scopes
Task created but execution stays queuedWorker availability and scheduler health
Execution completed but no data is availableResult export path and ingestion completion
Confusing response shapesAPI reference overview and HTTP errors and exceptions

Next steps