Scrapling

Adaptive web scraping framework

Adaptive web scraping framework supporting single requests to large-scale crawling. Features dynamic scraping with Playwright, stealth capabilities, and multiple selector methods (CSS/XPath).

Author: D4Vinci

Stars: 36,406

GitHub

Install: pip install Scrapling

Key Features

• Dynamic content scraping with Playwright integration
• Flexible selectors: CSS, XPath, and regex patterns
• Scales from single requests to large-scale crawling
• Stealth capabilities to avoid bot detection
• Supports both synchronous and asynchronous scraping

Use Cases

• Price comparison site building — Automatically collect product data from multiple e-commerce sites for price comparison pages.
• JavaScript-heavy website scraping — Extract content from SPAs and JavaScript-rendered pages using Playwright automation.
• Large-scale data pipeline — Efficiently crawl thousands of pages with async processing for bulk data collection.