CLI
Fetch
Fetch URL content operations
Fetch URL content operations
fetch › bulk
Bulk fetch operations
| Option | Description |
|---|---|
-h, --help | display help for command |
fetch › bulk › create
Create a bulk fetch job
| Option | Description |
|---|---|
--file <path> | Path to a newline-delimited file. Each line is a plain URL, or a JSON object carrying "url" plus per-item config. A line whose only key is "url" behaves exactly as before (sent as a bare URL string); any line with additional keys switches the payload to the per-item shape. |
--webhook-url <url> | HTTPS webhook URL notified when the job completes |
--webhook-secret <secret> | HMAC secret used to sign the completion webhook (omitted → unsigned) |
--defaults <json> | Defaults applied to every entry, as a JSON object (omitted → none). Unknown keys are rejected by /v1/bulk (422) and ignored by scrape bulk / fetch bulk; fetch bulk additionally rejects browser-only keys (cookies, wait_until, wait_for_selector, delay_ms) with a 400. |
--metadata <json> | Arbitrary metadata echoed on the job envelope, as a JSON object, e.g. '{"run":"7"}' (omitted → none) |
-h, --help | display help for command |
$ bytekit fetch bulk create --file <path> --webhook-url <url> --webhook-secret <secret> --defaults <json> --metadata <json>fetch › bulk › get
Get a bulk fetch job
| Option | Description |
|---|---|
-h, --help | display help for command |
$ bytekit fetch bulk get <id>fetch › create
Fetch URL content (POST)
| Option | Description |
|---|---|
--url <url> | URL to fetch |
--format <format> | Response format (markdown or html) |
--country <cc> | Two-letter country code to route the request through (e.g. us, de) |
--timeout-ms <n> | Per-request timeout in milliseconds |
--cache-ttl <ttl> | Cache freshness: 0, Nh (hours), or Nd (days), e.g. 48h, 2d, 0 (omitted → server default) |
--markdown-mode <mode> | Markdown mode: article, raw, llm |
--markdown-query <q> | BM25 query string for relevance-ranked markdown filtering (omitted → disabled) |
--markdown-links <mode> | Link rendering: inline, references, none, text (omitted → server default) |
--markdown-images <mode> | Image retention: inline, references, none, text (omitted → server default) |
--with-links-summary | Append a Links footer to the markdown output (omitted → off) |
--with-images-summary | Append an Images footer to the markdown output (omitted → off) |
--markdown-compact | Collapse excessive whitespace in the markdown output (omitted → off) |
--markdown-filter-images | Filter low-signal images from the markdown output (omitted → off) |
--markdown-include-media | Return rich link/image objects and a tables array (omitted → off) |
--markdown-include-warnings | Include markdown-pipeline and tag-filter warnings (omitted → off) |
--markdown-include-stats | Include a top-level stats object (chars, tokens, blocks) (omitted → off) |
--custom <json> | User payload echoed on the envelope as a JSON object, e.g. '{"job":"1"}' (omitted → none) |
-o, --output <file> | Write the fetched body bytes to the given file path |
-h, --help | display help for command |
$ bytekit fetch create --url <url> --format <format> --country <cc> --timeout-ms <n> --cache-ttl <ttl> --markdown-mode <mode> --markdown-query <q> --markdown-links <mode> --markdown-images <mode> --with-links-summary --with-images-summary --markdown-compact --markdown-filter-images --markdown-include-media --markdown-include-warnings --markdown-include-stats --custom <json>fetch › get
Fetch URL content (GET via query params)
| Option | Description |
|---|---|
--url <url> | URL to fetch |
--format <format> | Response format (markdown or html) |
--country <cc> | Two-letter country code to route the request through (e.g. us, de) |
--timeout-ms <n> | Per-request timeout in milliseconds |
--cache-ttl <ttl> | Cache freshness: 0, Nh (hours), or Nd (days), e.g. 48h, 2d, 0 (omitted → server default) |
-o, --output <file> | Write the fetched body bytes to the given file path |
-h, --help | display help for command |
$ bytekit fetch get --url <url> --format <format> --country <cc> --timeout-ms <n> --cache-ttl <ttl>