Crawly McCrawlface is a fast, focused macOS web crawler that extracts text content and SEO metadata from any website
Crawly McCrawlface is a fast, focused macOS web crawler that extracts text content and SEO metadata from any website.
FEATURES
- Crawl entire websites via sitemap discovery and link following
- Download website as a version that can be opened offline
- Extract clean text content with automatic noise removal (scripts, nav, footers, cookie banners)
- Extract SEO metadata: meta tags, Open Graph, Twitter Cards, canonical URLs, hreflang tags
- Export as plain text (single file or one per page) or XLIFF for translators
- Filter by language to crawl only specific content
- Adjustable crawl speed: Polite, Normal, or Fast
- Depth control to limit how deep the spider goes
- Include subdomains or stay on the main domain
- Pause, resume, and cancel crawls at any time
- Live progress with real-time stats and scrolling log
- Response time tracking with color-coded performance indicators
- Visual sitemap tree view of crawled site structure
- Page detail view with full extracted text, SEO data, and outgoing links
- Search and filter through crawl results
- robots.txt support (opt-in)
- HTTP Basic Auth for crawling protected sites
- Optional crawl log export with statistics
EXPORT FORMATS
- Text: Single combined file or one file per page
- XLIFF: Industry-standard translation format
- Crawl log: Detailed report with URLs, word counts, and statistics
Built natively for macOS with SwiftUI.