XCrawl Skills

The repository's stated aim is to help AI agents call the XCrawl API consistently. It holds five skill definitions for the XCrawl API, each a SKILL.md file describing when to use it, which parameters to send and what comes back, with cURL and Node examples that can run as-is.

The skills cover the main XCrawl jobs. The default skill handles direct lookups and single-URL extraction. Scrape returns one page as Markdown, links or structured data, synchronously or as an async job. Map lists the URLs a site exposes, useful for limiting a crawl to something like /docs/. Crawl runs a bounded multi-page crawl, for example two levels deep and 100 pages, and polls until it finishes. Search returns results for a query with location and language controls.

You need curl, Node and an XCrawl API key. Register on XCrawl's dashboard, activate the free plan with 1,000 credits, and put the key in a local config file at ~/.xcrawl/config.json, which every skill reads. A shared contract normalizes inputs and outputs, so the same skills can sit behind different agent runtimes, and by default they pass XCrawl's raw API responses straight through.

This is a set of instructions for XCrawl's API, not a standalone tool, so its usefulness depends on XCrawl itself, which is run by Xcrawl LLC. The repository doesn't include a license file, and its last update was in March 2026. Scraping and crawling other people's sites can conflict with those sites' terms of use, so it's worth checking before pointing an agent at them.