Scrapy: Difference between revisions
Jump to navigation
Jump to search
(Created page with "{{Data mining and learning analytics tools |field_screenshot= |field_name=Scrapy |field_developers=Scrapy community |field_license_type=Free&Open source |field_free_software_l...") |
Cybermancer (talk | contribs) m (Added link to Python article) |
||
| (2 intermediate revisions by one other user not shown) | |||
| Line 1: | Line 1: | ||
{{Data mining and learning analytics tools | {{Data mining and learning analytics tools | ||
|field_screenshot= | |field_logo=Scrapy.jpg | ||
|field_screenshot=Scrapy screenshot.jpg | |||
|field_name=Scrapy | |field_name=Scrapy | ||
|field_developers=Scrapy community | |field_developers=Scrapy community | ||
| Line 8: | Line 9: | ||
|field_last_version=0.22.2 | |field_last_version=0.22.2 | ||
|field_website=http://scrapy.org/ | |field_website=http://scrapy.org/ | ||
|field_description= | |field_description=Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing. | ||
Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing. | |||
Features | Features | ||
*Scrapy was designed with simplicity in mind, by providing the features you need without getting in your way | *Scrapy was designed with simplicity in mind, by providing the features you need without getting in your way | ||
| Line 15: | Line 15: | ||
*Scrapy is used in production crawlers to completely scrape more than 500 retailer sites daily, all in one server | *Scrapy is used in production crawlers to completely scrape more than 500 retailer sites daily, all in one server | ||
*Scrapy was designed with extensibility in mind and so it provides several mechanisms to plug new code without having to touch the framework core | *Scrapy was designed with extensibility in mind and so it provides several mechanisms to plug new code without having to touch the framework core | ||
*Scrapy is completely written in Python and runs on Linux, Windows, Mac and BSD | *Scrapy is completely written in [[Python]] and runs on Linux, Windows, Mac and BSD | ||
*Scrapy comes with lots of functionality built in. Check this section of the documentation for a list of them. | *Scrapy comes with lots of functionality built in. Check this section of the documentation for a list of them. | ||
*Scrapy is extensively documented and has an comprehensive test suite with very good code coverage | *Scrapy is extensively documented and has an comprehensive test suite with very good code coverage | ||
