The educational technology and digital learning wiki
Jump to navigation
Jump to search
|
|
| (One intermediate revision by one other user not shown) |
| Line 1: |
Line 1: |
| {{Data mining and learning analytics tools | | {{Data mining and learning analytics tools |
| |field_screenshot=Scrapy.jpg | | |field_logo=Scrapy.jpg |
| | |field_screenshot=Scrapy screenshot.jpg |
| |field_name=Scrapy | | |field_name=Scrapy |
| |field_developers=Scrapy community | | |field_developers=Scrapy community |
| Line 14: |
Line 15: |
| *Scrapy is used in production crawlers to completely scrape more than 500 retailer sites daily, all in one server | | *Scrapy is used in production crawlers to completely scrape more than 500 retailer sites daily, all in one server |
| *Scrapy was designed with extensibility in mind and so it provides several mechanisms to plug new code without having to touch the framework core | | *Scrapy was designed with extensibility in mind and so it provides several mechanisms to plug new code without having to touch the framework core |
| *Scrapy is completely written in Python and runs on Linux, Windows, Mac and BSD | | *Scrapy is completely written in [[Python]] and runs on Linux, Windows, Mac and BSD |
| *Scrapy comes with lots of functionality built in. Check this section of the documentation for a list of them. | | *Scrapy comes with lots of functionality built in. Check this section of the documentation for a list of them. |
| *Scrapy is extensively documented and has an comprehensive test suite with very good code coverage | | *Scrapy is extensively documented and has an comprehensive test suite with very good code coverage |
Latest revision as of 14:26, 1 November 2015
Scrapy 0.22.2 (2014/02/14)
Developed by: Scrapy community
License: Public Domain
Web page : Tool homepage
Tool type :
The last edition of this page was on: 2014/02/26
The Completion level of this page is : Low
The last edition of this page was on: 2014/02/26 The Completion level of this page is : Low
SHORT DESCRIPTION
[[has description::Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.
Features
- Scrapy was designed with simplicity in mind, by providing the features you need without getting in your way
- Just write the rules to extract the data from web pages and let Scrapy crawl the entire web site for you
- Scrapy is used in production crawlers to completely scrape more than 500 retailer sites daily, all in one server
- Scrapy was designed with extensibility in mind and so it provides several mechanisms to plug new code without having to touch the framework core
- Scrapy is completely written in Python and runs on Linux, Windows, Mac and BSD
- Scrapy comes with lots of functionality built in. Check this section of the documentation for a list of them.
- Scrapy is extensively documented and has an comprehensive test suite with very good code coverage]]
TOOL CHARACTERISTICS
Usability
Authors of this page consider that this tool is '.
Tool orientation
This tool is designed for general purpose analysis.
Data mining type
This tool is made for '.
Manipulation type
This tool is designed for '.
| Tool objective(s) in the field of Learning Sciences |
|
☑ Analysis & Visualisation of data
☑ Predicting student performance
☑ Student modelling
☑ Social Network Analysis (SNA)
☑ Constructing courseware
|
☑ Providing feedback for supporting instructors:
☑ Recommendations for students
☑ Grouping students:
☑ Developing concept maps:
☑ Planning/scheduling/monitoring
☑ Experimentation/observation
|
ABOUT USERS
Tool is suitable for:
Students/Learners/Consumers
Teachers/Tutors/Managers
Researchers
Developers/Designers
Organisations/Institutions/Firms
Others
FREE TEXT
Tool version : Scrapy 0.22.2 2014/02/14
(blank line)
Developed by : Scrapy community
(blank line)
Tool Web page : http://scrapy.org/
(blank line)
Tool type :
(blank line)
License:Public Domain
| 
|
SHORT DESCRIPTION
Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.
Features
- Scrapy was designed with simplicity in mind, by providing the features you need without getting in your way
- Just write the rules to extract the data from web pages and let Scrapy crawl the entire web site for you
- Scrapy is used in production crawlers to completely scrape more than 500 retailer sites daily, all in one server
- Scrapy was designed with extensibility in mind and so it provides several mechanisms to plug new code without having to touch the framework core
- Scrapy is completely written in Python and runs on Linux, Windows, Mac and BSD
- Scrapy comes with lots of functionality built in. Check this section of the documentation for a list of them.
- Scrapy is extensively documented and has an comprehensive test suite with very good code coverage
TOOL CHARACTERISTICS
| Tool orientation |
Data mining type |
Usability |
| This tool is designed for general purpose analysis. |
This tool is designed for . |
Authors of this page consider that this tool is . |
| Data import format |
Data export format |
| . |
. |
| Tool objective(s) in the field of Learning Sciences |
|
☑ Analysis & Visualisation of data
☑ Predicting student performance
☑ Student modelling
☑ Social Network Analysis (SNA)
☑ Constructing courseware
|
☑ Providing feedback for supporting instructors:
☑ Recommendations for students
☑ Grouping students:
☑ Developing concept maps:
☑ Planning/scheduling/monitoring
☑ Experimentation/observation
|
Can perform data extraction of type:
Can perform data transformation of type:
Can perform data analysis of type:
Can perform data visualisation of type:
(These visualisations can be interactive and updated in "real time")
ABOUT USER
| Tool is suitable for: |
| Students/Learners/Consumers:☑ |
Teachers/Tutors/Managers:☑ |
Researchers:☑ |
Organisations/Institutions/Firms:☑ |
Others:☑ |
| Required skills: |
| Statistics: |
Programming: |
System administration: |
Data mining models: |
OTHER TOOL INFORMATION
|
| Scrapy screenshot.jpg
|
| Scrapy.jpg
|
| Scrapy
|
| Public Domain
|
| Free&Open source
|
| Scrapy community
|
| 2014/02/14
|
| 0.22.2
|
| http://scrapy.org/
|
| [[has description::Scrapy is a fast high-level screen scraping and web crawling framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from data mining to monitoring and automated testing.
Features
- Scrapy was designed with simplicity in mind, by providing the features you need without getting in your way
- Just write the rules to extract the data from web pages and let Scrapy crawl the entire web site for you
- Scrapy is used in production crawlers to completely scrape more than 500 retailer sites daily, all in one server
- Scrapy was designed with extensibility in mind and so it provides several mechanisms to plug new code without having to touch the framework core
- Scrapy is completely written in Python and runs on Linux, Windows, Mac and BSD
- Scrapy comes with lots of functionality built in. Check this section of the documentation for a list of them.
- Scrapy is extensively documented and has an comprehensive test suite with very good code coverage]]
|
| General analysis
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| Low
|
