Firecrawl Local: an agent scraper that is 37x faster and uses 230x less memory
Firecrawl, a service that turns websites into clean text for AI agents, has released Firecrawl Local, a version you run yourself. The team rewrote the core scrape operation, which downloads a page and cleans it, in Rust. It is 37 times faster than the previous self-hosted version and uses 11 MB at idle, 230 times less. A parser this small can run in a CI container next to every agent.
AI-processed from X: @firecrawl; edited by Hamidun News
Firecrawl, a service that turns web pages into clean text that language models can read, has released Firecrawl Local. It solves a problem familiar to anyone building agents: for a model to read documentation or a website, the page must first be downloaded and stripped of ads, menus and scripts. The earlier local self-hosted version, the one a team installs on its own server, was heavy and slow.
Now the repository has been rebuilt. The core scrape operation, which the authors call the scrape primitive, is written in Rust. It runs 37 times faster than the previous self-hosted version and takes just 11 MB of memory at idle, 230 times less.
For developers, this means the parser can sit next to the agent instead of living on a separate machine.
What the numbers mean
The announcement contains three numbers. The first is 37x: that is how much faster the basic operation of fetching a page and turning it into clean text runs compared with the previous self-hosted version of Firecrawl. The second is 11 MB at idle: the memory the process uses when there are no requests. That is 230 times less than the previous version. Simple arithmetic puts the old version at roughly 2.5 GB, but that is a calculation from the stated figures, not a separate measurement. The third is 100% open source code.
Imagine you are testing an agent in CI. Every run needs a page parser. Previously, that meant spinning up a hefty service with noticeable memory consumption. Now it is a small process that starts alongside the tests and uses almost nothing until it is needed.
Compatibility is preserved. According to Firecrawl, everything works out of the box with its MCP servers and SDKs. If an agent is already connected to Firecrawl through MCP, moving to the local version should not require rewriting the integration: you only need to change the address the agent uses to fetch pages.
Why it matters for agent developers
The first benefit is privacy and cost. If scraping runs on your machine or inside your own perimeter, internal addresses, pages behind a corporate VPN and sensitive data do not leave for an external cloud. For intranets, private documentation and test environments, this is often the only acceptable option.
The second benefit is density. At 11 MB idle, the service can run in a container, on a laptop, on a small server or inside a CI pipeline. You can keep several instances running in parallel without watching your memory bill.
The third benefit is loop speed. An agent that looks up information in several steps waits for a response at each one. When the basic operation runs faster, total response time drops. In an assistant that reads documentation during a conversation, this feels like the agent "started thinking faster."
What to check
The figures come from Firecrawl's own announcement, and the post includes no independent measurements. The comparison is with the previous self-hosted version, not with other scrapers, so it is too early to say "fastest on the market." The 37x speedup applies specifically to the basic scrape operation. Crawling a large site in full, rendering pages with heavy JavaScript and getting past anti-bot protection depend on many other factors, and you should not expect that speedup across the whole pipeline.
The practical plan is simple. Take the repository, run your own real pages through it and compare three things: response time, memory use and the quality of the resulting text. Pay particular attention to hard cases such as dynamic pages and long tables. If quality has not dropped, running locally can replace cloud calls for some tasks and simplify your architecture at the same time: fewer external dependencies, fewer limits, fewer bills.
For the industry, this is one more signal that agent tooling is moving away from heavy server stacks toward lightweight open components written in fast languages such as Rust. Scraping is a basic building block of almost any agent, and the lighter it is, the easier it is to embed everywhere.
- Nov 20, 2026Standard Bank cuts AI app review from two weeks to two hours
- Nov 20, 2026Schmidhuber and Pentland Explain Why Temperature Doesn't Make an Agent Curious
- Nov 20, 2026AI Agent Calls Patients Before Appointments: 10% More Completed Screenings Than With IVR
- Nov 20, 2026In six weeks, Claude derived an exact c/9.1266 bound in black hole physics
Key Questions Answered
What the numbers mean?
The announcement contains three numbers. The first is 37x: that is how much faster the basic operation of fetching a page and turning it into clean text runs compared with the previous self-hosted version of Firecrawl. The second is 11 MB at idle: the memory the process uses when there are no requests. That is 230 times less than the previous version. Simple arithmetic puts the old version at roughly 2.5 GB, but that is a calculation from the stated figures, not a separate measurement.
Why it matters for agent developers?
The first benefit is privacy and cost. If scraping runs on your machine or inside your own perimeter, internal addresses, pages behind a corporate VPN and sensitive data do not leave for an external cloud. For intranets, private documentation and test environments, this is often the only acceptable option. The second benefit is density. At 11 MB idle, the service can run in a container, on a laptop, on a small server or inside a CI pipeline.
What to check?
The figures come from Firecrawl's own announcement, and the post includes no independent measurements. The comparison is with the previous self-hosted version, not with other scrapers, so it is too early to say "fastest on the market." The 37x speedup applies specifically to the basic scrape operation. Crawling a large site in full, rendering pages with heavy JavaScript and getting past anti-bot protection depend on many other factors, and you should not expect that speedup across the whole pipeline.
Build & Ship Production AI Agents Without Boilerplate Code
Start your free 72-hour trial at Hamidun Academy (academy.hamidun.com) — interactive lessons on Claude Code & Cursor, live Tech Radar, and production agent templates.
The AI world, distilled — once a week
Seven stories that actually mattered, hand-picked. No noise, no reposts, no press releases.
Done! Check your inbox for a confirmation.