All nodes/Integrations/HTTP & Web
Fetch Webpage
A web page fetching tool for the AI Agent: reads the content of a URL and returns text. The agent calls it to retrieve the content of specific pages. Connects to the agent's 'Agent extensions' input.
Type in the graph: fetch_webpage
Tool
Try it
Minimal working scenario
- Execute + Data
- Extension
- LLM
- Memory
- Execute + Data + Streaming
Runs as pasted
When to use it
The tool an agent uses to read one specific page: the node downloads the HTML, strips the markup and hands the model plain Markdown. It is usually paired with Web Search — the search finds addresses, this node opens them. When the URL is known in advance and you need an API response with a status code and headers, use HTTP Request: that one is a graph step, not a tool.
This node exists only as a tool: its single tool_out port goes to the extensions port of
the AI Agent. It has neither an execution input nor a data
output.
How it works
The model passes one argument — the page address. Requests into private networks and to loopback are blocked, and every redirect hop is validated again.
The main content is extracted from the HTML: first <main>, then <article>, then a div
whose class looks like content/main/article, and only then the whole body. Scripts,
styles, navigation and footers are dropped, and the rest is converted to Markdown. A
response whose content type is not HTML comes back as it is.
There are three limits, and they apply in different places. Max response size (bytes) (1,000,000 by default) truncates the raw response before conversion. Max characters (500–50,000, 8000 by default) truncates the finished Markdown and appends a note about the omitted characters. Timeout (seconds) is 20 by default, 120 at most.
Common mistakes
- A page rendered by scripts. There is no browser here: you get the document shell without the data.
- Raising only the character limit. The raw response is cut first — raise both limits together, or the conversion sees truncated HTML.
- PDFs and other binary formats. Such a response is not HTML, it is returned as is, and the model receives unreadable text.
- Expecting the page text in the graph. Only the model sees the content; it never reaches a port.
- Large pages in a long conversation. Every fetch adds up to Max characters to the context — paid input tokens on every following turn.
Inputs
—
Outputs
| Port | Wire | Payload | Notes |
|---|---|---|---|
Agent extensiontool_out | Extensionlink_extension | — |
Configuration
| Field | Type | Default | Description |
|---|---|---|---|
Follow redirectsfollow_redirects | boolean | true | Follow a 3xx redirect to the address the server points at. |
Max charactersmax_chars | integer | 8000 | Maximum characters to fetch |
Max response size (bytes)max_response_bytes | integer | 1000000 | Hard cap on raw response size before HTML→Markdown conversion |
Stream fetch progress in Catchstream_progress_in_catch | boolean | true | Show step-by-step progress in the Catch Tool Call stream. |
Timeout (seconds)timeout_seconds | number | 20 | HTTP request timeout |