Skip to main content

Scenario

You track pricing coverage across competitor blogs. You do not have a fixed URL list—you have a query. You want one task that discovers candidate pages, then extracts a structured record from each of them, without running two separate jobs and stitching results together yourself. This is what an execution plan is for: a task with more than one action step, where a later step reads the output of an earlier one.

Architecture

Step 1: create the task with an execution plan

Each step needs a step_id, an action_type, and an input_source. Set next_step_id to wire steps together, and set input_source: previous_step on the downstream step so it consumes what the upstream step discovered.
template_slug values must exist in your deployment’s template catalog. List what is available with GET /api/v1/templates, or see Use templates and preview runs.

Step 2: run the task

The response contains the execution_id for this run.

Step 3: watch step progress

A multi-step execution reports per-step state, so you can see which stage the run is in rather than only an overall status.
Each entry carries a step_id, status, expected_count, and processed_count. The extract step stays queued until discover produces URLs for it.

Step 4: read results per step

Results are addressed per step, so you can inspect what discovery found separately from what extraction produced.
To export everything the run produced in one file, use GET /api/v1/executions/$EXECUTION_ID/results/export.

Retry behavior across steps

retry_policy is per step: Leave fail_execution unset when partial coverage is more useful than no results—a discovery step that finds 20 of 25 pages usually still has value.

Why this example matters

Multi-step plans are what separate a scraping job from a pipeline. The same shape scales to:
  • searchscrape for discovery-first monitoring
  • crawlscrape for site-wide extraction
  • scrapeapi_call for enrichment against your own services
Agents get the same capability without composing plans by hand—see MCP tool reference.

Next steps