> ## Documentation Index
> Fetch the complete documentation index at: https://webscraping.titannet.io/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# titan_fetch

> Turn up to 100 known URLs into clean markdown or text, with boilerplate stripped and per-URL failures reported separately.

Use `titan_fetch` when your agent already has URLs and needs their content. Pages render in real browsers before extraction, so JavaScript-driven sites return content rather than an empty shell.

**Required scope:** `mcp:fetch`

## Parameters

<ParamField body="urls" type="string[]" required>
  URLs to fetch, up to 100 per call. Each is validated before the run starts.
</ParamField>

<ParamField body="format" type="string" default="markdown">
  `markdown` or `text`. Markdown preserves headings, lists, and links; text is flat prose.
</ParamField>

<ParamField body="only_main_content" type="boolean" default="true">
  Strip navigation, headers, footers, and sidebars, keeping the article body. Set `false` when you need the full page.
</ParamField>

<ParamField body="include_links" type="boolean" default="true">
  Include the page's outbound links as a structured `links` array.
</ParamField>

<ParamField body="include_image_links" type="boolean" default="true">
  Include image URLs and alt text as a structured `image_links` array.
</ParamField>

<ParamField body="max_chars_per_url" type="integer" default="12000">
  Character cap per page. Content is truncated on a character boundary, so multibyte text is never cut mid-character.
</ParamField>

<ParamField body="freshness" type="string" default="cache_ok">
  How current the content must be.

  * `cache_ok` — a recent cached copy is acceptable, and fastest
  * `prefer_live` — try live, fall back to cache
  * `live_only` — always fetch live
</ParamField>

<ParamField body="wait_for_completion" type="boolean" default="true">
  Wait for results inside the call. Set `false` for large URL batches.
</ParamField>

<ParamField body="timeout_seconds" type="integer" default="30">
  Seconds to wait, capped at 30.
</ParamField>

<ParamField body="idempotency_key" type="string">
  Deduplicate repeated calls for 24 hours. See [Idempotency and retries](/docs/mcp/idempotency-and-retries).
</ParamField>

## Example

```json theme={null}
{
  "urls": [
    "https://example.com/blog/scaling-postgres",
    "https://example.com/blog/connection-pooling"
  ],
  "format": "markdown",
  "only_main_content": true,
  "max_chars_per_url": 8000
}
```

## Response

Successful pages and failures come back separately, so an agent never has to infer which URLs worked.

```json theme={null}
{
  "run_id": "3c7d1a92-6f48-4b25-9e03-7d1a4c8b2f65",
  "status": "completed",
  "pages": [
    {
      "url": "https://example.com/blog/scaling-postgres",
      "final_url": "https://example.com/blog/scaling-postgres",
      "title": "Scaling Postgres past a single node",
      "status_code": 200,
      "content_format": "markdown",
      "content": "# Scaling Postgres past a single node\n\nConnection limits become...",
      "links": [
        { "url": "https://example.com/blog/connection-pooling", "text": "connection pooling" }
      ],
      "image_links": [
        { "url": "https://example.com/img/topology.png", "alt": "Replication topology" }
      ],
      "metadata": {},
      "retrieved_at": "2026-07-27T09:20:11Z"
    }
  ],
  "failed": [
    {
      "url": "https://example.com/blog/connection-pooling",
      "code": "fetch_failed",
      "message": "no record returned for url"
    }
  ],
  "usage": {
    "backend_execution_id": "3c7d1a92-6f48-4b25-9e03-7d1a4c8b2f65",
    "billing_unit": "successful_delivery",
    "records_returned": 1,
    "credits_estimated": 2,
    "credits_consumed": 1,
    "billing_status": "billable",
    "failure_count": 1
  },
  "warnings": [],
  "request_id": "req_2d9b4f1a7c3e8056"
}
```

### Page fields

| Field            | Meaning                                                       |
| ---------------- | ------------------------------------------------------------- |
| `url`            | The URL you requested                                         |
| `final_url`      | Where it landed after redirects                               |
| `title`          | Page title                                                    |
| `status_code`    | HTTP status of the final response                             |
| `content_format` | `markdown` or `text`                                          |
| `content`        | Extracted content, truncated to `max_chars_per_url`           |
| `links`          | Outbound links with anchor text, when `include_links` is true |
| `image_links`    | Image URLs with alt text, when `include_image_links` is true  |
| `metadata`       | Page metadata such as description and author, when available  |
| `retrieved_at`   | RFC 3339 timestamp                                            |

### Failure entries

Each entry in `failed` carries the requested `url`, a `code`, and a `message`. Common codes:

| Code           | Meaning                                                    |
| -------------- | ---------------------------------------------------------- |
| `unsafe_url`   | Rejected by URL safety checks before the run started       |
| `fetch_failed` | The page produced no record—unreachable, blocked, or empty |

## URL safety

URLs are validated before any work begins. The server rejects:

* Non-HTTP schemes such as `file://`, `ftp://`, and `data:`
* `localhost` and loopback addresses
* Private IP ranges and link-local addresses
* Cloud metadata endpoints

Rejected URLs land in `failed` with code `unsafe_url` and cost nothing. If **every** URL is rejected, the call returns an `unsafe_url` error and starts no run. Redirects are re-checked at each hop, so a public URL cannot redirect into a private address.

See [Limits and safety](/docs/mcp/limits-and-safety).

## Credits

**One credit per successfully extracted page.** URLs that fail are not billed. In the example above, two URLs were requested, one succeeded, and one credit was consumed.

`credits_estimated` reflects what you requested; `credits_consumed` reflects what was delivered.

## Partial results

With many URLs, expect `status: "partial"`—some pages extracted, others failed. This is normal and usually still useful. Read `pages` for what worked and `failed` for what did not, then decide whether to retry the failures.

## Tuning content size

`max_chars_per_url` is the main lever on how much context a fetch consumes:

| Goal                   | Suggested value               |
| ---------------------- | ----------------------------- |
| Quick relevance check  | 2000                          |
| Summarize an article   | 8000                          |
| Full document analysis | 12000 (the default) or higher |

Combine a small `max_chars_per_url` with `only_main_content: true` when you are triaging many pages and will re-fetch the good ones in full.

## Next steps

* [titan\_search](/docs/mcp/tools/titan-search) — find URLs to fetch
* [titan\_crawl](/docs/mcp/tools/titan-crawl) — when you need a whole site instead
* [Research agent example](/docs/mcp/examples/research-agent)