# Universal Scraping Architect

> Plans and builds scraping pipelines with Firecrawl or local Python, and won't hand over the data until it passes a validation check.

A community Claude Code plugin from Alireza Rezvani's claude-skills collection, contributed by Mehansh Barthwal. It turns a scraping request into a pipeline: it picks the right way to extract the data, budgets for API quota and tokens, and validates every result before handing it over.

What you can do with it
- Scrape public, JavaScript-heavy or many-page sites through Firecrawl, with your own API key
- Extract data from local PDF, Excel, CSV or simple HTML files with Python (requests, BeautifulSoup and pandas), without sending them to an outside API
- Combine the two: Firecrawl to find and pull pages, pandas to clean and structure the output
- Crawl documentation into Markdown, chunked to fit an LLM's context
- Get CSV for tables, JSON for nested data or Markdown for text, with row counts and empty values reported

What's in the plugin
- The universal-scraping-architect skill, with Firecrawl and local-extraction guides and script templates you can run offline first
- A cs-scraping-architect agent that runs the whole pipeline
- A /cs-scrape command: give it a URL or a file and the output format you want

Guardrails it applies
- A validation script has to pass before any data is delivered; empty or malformed output gets re-extracted
- It checks robots.txt, adds rate limits and won't scrape sites that disallow it
- It flags hardcoded API keys and moves them to the FIRECRAWL_API_KEY environment variable
- It warns before private local files go to an outside API, and adds pagination and checkpoints to large jobs

How to install it
In Claude Code, run /plugin marketplace add alirezarezvani/claude-skills, then /plugin install universal-scraping-architect@claude-code-skills. It's also part of the collection's engineering-advanced-skills bundle.

Good to know
- Firecrawl mode needs your own Firecrawl API key, and works on Firecrawl's free tier. The local mode needs Python with pandas, requests and beautifulsoup4.
- It's a community project, under the MIT licence.

- Type: Plugin
- Made by: alirezarezvani
- Category: Web & browser
- Formats: Claude plugins
- Version: 2.14.1
- Website: https://github.com/alirezarezvani/claude-skills/tree/main/engineering/universal-scraping-architect
- Source: https://github.com/alirezarezvani/claude-skills/tree/HEAD/engineering/universal-scraping-architect
- Page: https://pluginboard.ai/plugins/universal-scraping-architect

## Works with

- Claude
- GitHub Copilot
- Claude Code
- OpenClaw

## Install

### Claude

In Claude (web or desktop): Customize → Plugins → Add → Add marketplace, enter alirezarezvani/claude-skills, then add universal-scraping-architect

### Claude Code

In Claude Code:

```text
/plugin marketplace add alirezarezvani/claude-skills
/plugin install universal-scraping-architect@claude-code-skills
```

---
From Pluginboard (https://pluginboard.ai).