Skip to main content

Documentation Index

Fetch the complete documentation index at: https://webscraping.titannet.io/docs/llms.txt

Use this file to discover all available pages before exploring further.

Titan fits anywhere you need repeatable access to the web and related APIs: scheduled monitors, one-off research, or software that must stay grounded in live sources. The sections below are not a feature checklist—they are common shapes of work teams implement once they treat tasks as the unit of orchestration and let the platform own browsers, queues, and execution history.

​
Grounding for LLMs and agents

Agents and copilots need current context without hard-coding every hostname. Titan’s action types map cleanly to how those systems already think:
  • Search — Resolve a user question or topic into candidate pages or documents before you fetch deeply.
  • Crawl — Widen or deepen coverage when the answer lives behind hubs, categories, or pagination.
  • Scrape — Turn selected URLs into structured records (prices, specs, citations) your model or tools can consume safely.
  • API call — Pull normalized data from partner APIs, pricing feeds, or your own backends, then combine with on-page signals where APIs fall short.
A multi-step task can encode that whole retrieval subgraph—search then scrape, or API then crawl then scrape—so your agent issues create / run, polls executions, and reads results, instead of operating headless browsers and rate limiters itself. That division of labor is the usual pattern for real-time intelligence: the agent decides; Titan retrieves at scale with consistent IDs for billing and analytics. See Action types overview for how single- and multi-step tasks are modeled in the API.

​
E-commerce and market intelligence

Merchandising and growth teams routinely track prices, availability, ratings, shipping claims, and promotional content across large SKU sets and competitor sites.
PatternHow teams use Titan
Fixed URL monitorsKnown product detail URLs on a schedule—classic extraction or a scrape-focused task with a stable output schema.
Discovery-heavy monitorsSearch or crawl to discover listing URLs when catalogs move, then scrape into the same schema your dashboards and alerts already expect.
Web plus official APIsAPI call where you have credentials and contracts, plus scrape for on-page signals the API does not expose.
Chaining steps with previous_step inputs lets you separate “find what changed” from “extract what we store,” which keeps monitors easier to maintain when layouts shift but navigation or search facets stay stable.

​
Research, trust, and operations

The same building blocks show up across industries:
  • Lead and directory research — Search vertical directories, normalize listings with scrape, hand off to CRM or enrichment pipelines.
  • Brand and compliance — Crawl approved retailer or partner lists, scrape claims and disclaimers, compare against policy keywords or approved wording.
  • Supply and logistics — API steps for carriers or ERP systems, scrape for exception portals or unstructured notices, one task for end-to-end orchestration.
  • Knowledge bases and RAG — Scheduled multi-step jobs that refresh document chunks tied to stable entity IDs, so retrieval indexes stay aligned with what the web actually says today.
In each case the value proposition is the same: discovery, extraction, and API glue in one platform, with datasets, history, and usage tied to the same task and execution identifiers.

​
How you typically ship it

  • One-off runs — Prove a schema or answer an ad-hoc question; create a task, run once, export results.
  • Scheduled programs — Price checks, compliance sweeps, or feed-style collection on a cron you control.
  • Template-led workflows — Share patterns across teams so validation, limits, and scripts stay consistent.
  • Dashboard-first operations — Operate and inspect work through the UI when you are not building a custom surface yet.

​
Next steps