Commoncrawl alternatives

1

CRW Web Scraper

by us

A

Open-source web scraper for AI agents with scrape, crawl, and map tools

2

PyreCrawl

by SanggonBoy

A

Web scraping + research toolkit for AI agents: scrape, crawl, monitor URLs with anti-bot bypass.

3

ReplyNodes

by replynodes.com

A

Web MCP: scrape/crawl sites, web search, brand assets, app stores, YouTube, Reddit, Hacker News.

4

Web Scraping

by superagnt.com

A

Web scraping for AI agents: scrape, search, crawl, map any website to markdown + JSON. No browser.

5

SeleniumBase MCP

by seleniumbase

A

Stealthy browser automation, testing, and web-scraping via CDP Mode.

6

AIHawk

by feder-cr

A

AI browser agent: browses, clicks, types, and reads real web pages from plain-English instructions.

7

Brightdata

by brightdata

B

Bright Data's Web MCP server enabling AI agents to search, extract & navigate the web

8

Scrapeless

by scrapeless-ai

A

Web search, browser automation, scraping, crawling and CAPTCHA solving for AI agents.

9

Web Browser (kogiQA)

by atagon.com

C

This web browser has been designed to help your agent debug and develop complex web applications.

10

shaft-mcp

by ShaftHQ

A

SHAFT browser automation, capture, and offline deterministic failure diagnosis

11

MCP

by foura.ai

B

Reliable web access for AI agents: smart HTTP, rotating proxies, and full-browser rendering.

12

Crawlberg

by xberg-io

A

Scrape, crawl, and map websites to Markdown or JSON via local CLI.

13

Svipall

by svipall.ilien.dev

C

Read any page past its anti-bot wall, crawl sites, search without an API key, solve captchas locally

14

Novada

by NovadaLabs

A

Search, extract, crawl, map, research, scrape 16 platforms, browser automation, proxy — one API key.

15

EnConvert

by enconvert.com

A

Unified web ingestion for AI agents: perceive, discover, search, extract, ingest, watch, convert.

16

doc-scraper

by Sriram-PR

A

Crawl documentation sites into local corpora agents can search, read, and diff fully offline.

17

Aginx Browser

by aginx.net

B

Stealth web browser for agents: search, fetch, click, download and type in persistent MCP sessions.

18

unbrowser by Unchained

by protostatis

B

Chrome-free MCP web access for agents with low-token page maps and browser escalation hints.

19

FetchV2

by praveenc

A

Fetch and extract web pages, discover links, and read llms.txt documentation indexes.

20

Dsh Verify

by 263311487-ux

A

Agent-built web app quality gate. Real browser is the judge. PASS/FAIL with receipts.

Debug bridge inside your web app: an agent reads the DOM, console and network. Any browser.

22

CrawlEyes MCP Server

by waiky-github

B

Web scraping & search for AI agents: extract, SearXNG search, rerank, robots compliance. No API key.

23

Crawl Readiness

by Grubbomatic

B

Audit any site for 50+ AI crawlers, generate llms.txt, robots.txt and schema, track AI mentions.

24

domscout

by jkresearch8

B

domscout browser API as MCP tools: screenshots, Markdown, structured data, page automation, crawls.