Free Intensive Nov 21–22Free 2-day hands-on Vibecoding Intensive on Nov 21–22 at 16:00 MSK (Sat–Sun) · Live Cohort 3 starts Nov 23Join Free →Cohort 3 (Nov 23)
ReleaseCode / SDK · ActionableX: @firecrawl→ original

Firecrawl Local: an agent scraper that is 37x faster and uses 230x less memory

Firecrawl, a service that turns websites into clean text for AI agents, has released Firecrawl Local, a version you run yourself. The team rewrote the core scrape operation, which downloads a page and cleans it, in Rust. It is 37 times faster than the previous self-hosted version and uses 11 MB at idle, 230 times less. A parser this small can run in a CI container next to every agent.

AI-processed from X: @firecrawl; edited by Hamidun News
Firecrawl Local: an agent scraper that is 37x faster and uses 230x less memory
Source: X: @firecrawl. Collage: Hamidun News.
Listen to article
Article Fact Sheet & Key Figures
ParameterDetails
Primary sourceX: @firecrawl ↗
Editor-in-Chief
Scrape in Rust37x faster than the previous self-hosted version
Idle memory11 MB, 230 times less than before
Key metric #3100% open source code; Firecrawl's MCP and SDK work without changes

Firecrawl, a service that turns web pages into clean text that language models can read, has released Firecrawl Local. It solves a problem familiar to anyone building agents: for a model to read documentation or a website, the page must first be downloaded and stripped of ads, menus and scripts. The earlier local self-hosted version, the one a team installs on its own server, was heavy and slow.

Now the repository has been rebuilt. The core scrape operation, which the authors call the scrape primitive, is written in Rust. It runs 37 times faster than the previous self-hosted version and takes just 11 MB of memory at idle, 230 times less.

For developers, this means the parser can sit next to the agent instead of living on a separate machine.

What the numbers mean

The announcement contains three numbers. The first is 37x: that is how much faster the basic operation of fetching a page and turning it into clean text runs compared with the previous self-hosted version of Firecrawl. The second is 11 MB at idle: the memory the process uses when there are no requests. That is 230 times less than the previous version. Simple arithmetic puts the old version at roughly 2.5 GB, but that is a calculation from the stated figures, not a separate measurement. The third is 100% open source code.

Imagine you are testing an agent in CI. Every run needs a page parser. Previously, that meant spinning up a hefty service with noticeable memory consumption. Now it is a small process that starts alongside the tests and uses almost nothing until it is needed.

Compatibility is preserved. According to Firecrawl, everything works out of the box with its MCP servers and SDKs. If an agent is already connected to Firecrawl through MCP, moving to the local version should not require rewriting the integration: you only need to change the address the agent uses to fetch pages.

Why it matters for agent developers

The first benefit is privacy and cost. If scraping runs on your machine or inside your own perimeter, internal addresses, pages behind a corporate VPN and sensitive data do not leave for an external cloud. For intranets, private documentation and test environments, this is often the only acceptable option.

The second benefit is density. At 11 MB idle, the service can run in a container, on a laptop, on a small server or inside a CI pipeline. You can keep several instances running in parallel without watching your memory bill.

The third benefit is loop speed. An agent that looks up information in several steps waits for a response at each one. When the basic operation runs faster, total response time drops. In an assistant that reads documentation during a conversation, this feels like the agent "started thinking faster."

What to check

The figures come from Firecrawl's own announcement, and the post includes no independent measurements. The comparison is with the previous self-hosted version, not with other scrapers, so it is too early to say "fastest on the market." The 37x speedup applies specifically to the basic scrape operation. Crawling a large site in full, rendering pages with heavy JavaScript and getting past anti-bot protection depend on many other factors, and you should not expect that speedup across the whole pipeline.

The practical plan is simple. Take the repository, run your own real pages through it and compare three things: response time, memory use and the quality of the resulting text. Pay particular attention to hard cases such as dynamic pages and long tables. If quality has not dropped, running locally can replace cloud calls for some tasks and simplify your architecture at the same time: fewer external dependencies, fewer limits, fewer bills.

For the industry, this is one more signal that agent tooling is moving away from heavy server stacks toward lightweight open components written in fast languages such as Rust. Scraping is a basic building block of almost any agent, and the lighter it is, the easier it is to embed everywhere.

Story

Key Questions Answered

What the numbers mean?

The announcement contains three numbers. The first is 37x: that is how much faster the basic operation of fetching a page and turning it into clean text runs compared with the previous self-hosted version of Firecrawl. The second is 11 MB at idle: the memory the process uses when there are no requests. That is 230 times less than the previous version. Simple arithmetic puts the old version at roughly 2.5 GB, but that is a calculation from the stated figures, not a separate measurement.

Why it matters for agent developers?

The first benefit is privacy and cost. If scraping runs on your machine or inside your own perimeter, internal addresses, pages behind a corporate VPN and sensitive data do not leave for an external cloud. For intranets, private documentation and test environments, this is often the only acceptable option. The second benefit is density. At 11 MB idle, the service can run in a container, on a laptop, on a small server or inside a CI pipeline.

What to check?

The figures come from Firecrawl's own announcement, and the post includes no independent measurements. The comparison is with the previous self-hosted version, not with other scrapers, so it is too early to say "fastest on the market." The 37x speedup applies specifically to the basic scrape operation. Crawling a large site in full, rendering pages with heavy JavaScript and getting past anti-bot protection depend on many other factors, and you should not expect that speedup across the whole pipeline.

ZK
Zhemal Khamidun — CPO at Alpina Digital, product manager of AlpinaGPT. Building AI products since 2019. Hamidun News is a pet project that grew into a production aggregator with 6,000+ readers.
✦ Hands-On AI Engineering · Hamidun Academy

Build & Ship Production AI Agents Without Boilerplate Code

Start your free 72-hour trial at Hamidun Academy (academy.hamidun.com) — interactive lessons on Claude Code & Cursor, live Tech Radar, and production agent templates.

Start 72h Free Trial →Vibe-Coding Intensive
What do you think?
Loading comments…