Universal Scraping Architect

A community Claude Code plugin from Alireza Rezvani's claude-skills collection, contributed by Mehansh Barthwal. It turns a scraping request into a pipeline: it picks the right way to extract the data, budgets for API quota and tokens, and validates every result before handing it over. What you can do with it - Scrape public, JavaScript-heavy or many-page sites through Firecrawl, with your own API key - Extract data from local PDF, Excel, CSV or simple HTML files with Python (requests, BeautifulSoup and pandas), without sending them to an outside API - Combine the two: Firecrawl to find and pull pages, pandas to clean and structure the output - Crawl documentation into Markdown, chunked to fit an LLM's context - Get CSV for tables, JSON for nested data or Markdown for text, with row counts and empty values reported What's in the plugin - The universal-scraping-architect skill, with Firecrawl and local-extraction guides and script templates you can run offline first - A cs-scraping-architect agent that runs the whole pipeline - A /cs-scrape command: give it a URL or a file and the output format you want Guardrails it applies - A validation script has to pass before any data is delivered; empty or malformed output gets re-extracted - It checks robots.txt, adds rate limits and won't scrape sites that disallow it - It flags hardcoded API keys and moves them to the FIRECRAWL_API_KEY environment variable - It warns before private local files go to an outside API, and adds pagination and checkpoints to large jobs How to install it In Claude Code, run /plugin marketplace add alirezarezvani/claude-skills, then /plugin install universal-scraping-architect@claude-code-skills. It's also part of the collection's engineering-advanced-skills bundle. Good to know - Firecrawl mode needs your own Firecrawl API key, and works on Firecrawl's free tier. The local mode needs Python with pandas, requests and beautifulsoup4. - It's a community project, under the MIT licence.

• “Scrape the product listings from this site into a CSV.”

  Copy

• “Get the data from this table as clean JSON.”

  Copy

• “Crawl these docs into Markdown I can give an LLM.”

  Copy

• “Pull the line items out of these PDF invoices without sending them to an outside API.”

  Copy

• “My scraper keeps breaking. Fix its brittle selectors.”

  Copy

• “Scrape the pricing page and validate the output before you give it to me.”

  Copy