- 1 Definition
- 2 Mathematics skills
- 3 The six proficiency levels of the PISA 2006 science skills
- 4 Setting up SPSS files
- 5 Links
- 6 Bibliography
PISA = Programme for International Student Assessment
“ PISA assesses how far students near the end of compulsory education have acquired some of the knowledge and skills that are essential for full participation in society. In all cycles, the domains of reading, mathematical and scientific literacy are covered not merely in terms of mastery of the school curriculum, but in terms of important knowledge and skills needed in adult life. [..] n the PISA 2003 cycle, an additional domain of problem solving was introduced to continue the examination of cross-curriculum competencies.” (, retrieved 18:31, 30 September 2006 (MEST)).
PISA is often quoted in discussions about educational policy. Most people do not really know what is measured and just quote rankings.
1.1 The achievement studies
PISA 2000 achievement studies usually included three modules and that measured applicable skills in three areas: “Are students well prepared to meet the challenges of the future? Are they able to analyse, reason and communicate their ideas effectively? Do they have the capacity to continue learning throughout life?” (Adams & Vu, 2002: Introduction)
- Mathematics: 32 items, representing approximately 60 minutes of testing time
- Sciences: 35 items, representing approximately 60 minutes of testing time for each.
- Reading: 141 items representing approximately 270 minutes of testing time for each.
“PISA 2000 was a paper-and-pencil test, with each student undertaking two hours of testing (i.e., answering one of the nine booklets). Pencils, erasers, rulers, and, in some cases, calculators, were provided. [...] The 141 main study reading items were organised into nine separate clusters, each with an estimated administration time of 30 minutes. The 32 mathematics items and the 35 science items were organised into four 15-minute mathematics clusters and four 15-minute science clusters respectively. These clusters were then combined in various groupings to produce nine linked two hour test booklets.” (Adams & Vu, 2002: 23).
These booklets (also called cognitive booklets (as opposed to the questionnaires) tested different modules. “Reading items occur in all nine booklets, and there are linkages between the reading in all booklets. This permits all sampled students to be assigned reading scores on common scales. Mathematics items occur in five of the nine booklets, and there are links between the five booklets, allowing mathematics scores to be reported on a common scale for five-ninths of the sampled students. Similarly, science material occurs in five linked booklets, allowing science scores to be reported on a common scale for five-ninths of the sampled students.” (Adams & Vu, 2002: 23).
PISA 2003 added:
- Problem solving (cross-curicular competencies)
- Countries can choose and add their own tests and surveys. E.g. in Switzerland there was a survey of 9th graders (in addition to 15 year olds).
1.2 The survey questionnaires
In addition to tests, PISA also administers questionnaires to both individual learners and schools. Datasets that combine questionnaire results and test results are available. “A Student and a School Questionnaire were used in PISA 2000 to collect data that could be used in constructing indicators pointing to social, cultural, economic and educational factors that are thought to influence, or to be associated with, student achievement.” (Adams & Vu, 2002: 33)
The student questionnaire included items for:
- Basic demographics
- Family background and measures of socioeconomic status
- Student description of school/instructional processes
- Student attitudes towards reading and reading habits
- Student access to educational resources outside school
- Institutional patterns of participation and programme orientation
- Student career and educational expectations
The school questionnaire included:
- Basic school characteristics
- School policies and practices
- School climate
- School resources
- The Information Technology or Computer Familiarity Instrument (computer literacy)
2 Mathematics skills
2.1 Design of the tests
According to quotations from PISA (2003):
2.2 The six proficiency levels of the PISA 2003 mathematical skills
See also learning level
3 The six proficiency levels of the PISA 2006 science skills
According to the PISA 2006 technical manual (OECD 2009), the six proficiency levels on the science scale are the following:
These levels can then be described at a further operational level with other scales, e.g. the summary descriptions of the six proficiency levels for using science (OECD 2009: 300) defines level 5 perfomance with these terms:
4 Setting up SPSS files
4.1 PISA 2012
You can download the data sets from Database - PISA 2012. You then can use SPSS or SAS control files to get nice ready-to-go data files.
- Dealing with the student questionnaire (tested on Jan 2014 with SPSS 22)
- Download Student questionnaire data file (about 230 MB compressed)
- Uncompress. This will give a 1.17GB INT_STU12_DEC03.txt file (or similar). We suggest to uncompress to a simple file location on Windows (e.g. c:\pisa\INT_STU12_DEC03.txt). Do not copy to "My Documents". On Linux, it doesn't matter since you probably know how to use a computer ...
- Download SPSS syntax to read in student questionnaire data file
- Open the *.SPS file in SPSS. It should open in the Syntax Editor
- Fix the second line (specify the location of the *.txt file, e.g. c:\pisa\INT_STU12_DEC03.txt) !!!
- Since the data uses a English
.separator, you may have to keep the first line, but I had to remove it when I used the french speaking interface of SPSS. I finally switched SPSS to English, something I do anyhow after installing a new version.
- Run the SPS file. Make sure to run all the commands to the bitter end or you only will import partial information. On a decent computer this won't last for very long, max. a few minutes on a multi-core system with enough memory and an SSD card.
Click the Green button
- If SPSS seems to be happy, do the following:
Menu: Run -> To End
- If SPSS is unhappy, then just try to get the first two lines right. E.g. if you can't switch to English, learn how to tell SPSS how to use local ways (e.g. search for "SPSS set decimal dot PISA")
Menu File -> Save
To do country-specific analysis, we suggest to create new data files, e.g. just consider splitting the file (see the data menu) with the first variable:
Menu: Data -> Split into Files Add Country Code Define an output directory (optional) Click ok
- The file names will use: ISO-name.sav, e.g. CHE.sav (but you can change that)
... Ready to go :)
- OECD PISA .org Home page (stable URL)
- Australian websites - have additional stuff like sample questions or a mirror for the datasets
- PISA Sample Questions
- http://pisaweb.acer.edu.au (PISA 2000 Data Archive)
- http://pisaweb.acer.edu.au (PISA 2003 Data Archive)
- PISA 2012 Data
- PISA Instruments
5.2 PISA 2012
- PISA 2012 Assessment and Analytical Framework
- ISA 2012 Assessment and Analytical Framework Glance at this first !
- PISA 2012 test items
- PISA 2012 Student questionnaires
- PISA 2012 included one main questionnaiare and several other ones. As of Jan 2014, we did not find any single documentation
- All the questionnaires are available from the Database - PISA 2012] page. There are four variants of the student questionnaire. However, we found that Student Questionnaire (main survey) found on mypisa is the best starting point. But you do need in addition the ICT and Educational career (EC) ones in addition.
- Watch out for the questions codes (in grey on top of each question)
- If you plan to conduct data analysis you also might download the code book (i.e. a dump of information that you will have in your SPSS/SAS file)
- PISA 2012 Student Questionnaires (alternative)
- Student Questionnaire (probably the most complete one)
- ICT Familiarity component for Student Questionnaire
- Educational Career component for Student Questionnaire
- PISA 2012 Data
- These are available as compressed txt file plus SPSS/SAS Control files that you will have to run
- Database - PISA 2012
- The questionnaire file includes about 500'000 entries.
- PISA 2012 Other questionnaires
5.3 Country specific
- Swiss pisa data
- FORS Swiss foundation for research in social sciences. You can get some of the swiss datasets from here (but takes some time to click through).
- Swiss PISA website
- Swiss Public Use Sample PISA 2003 (used in the MALTT methodology class)
To do (there is a whole lot of literature).
- PISA test items and school textbooks related to science: A textual comparison (2008). PISA test items and school textbooks related to science: A textual comparison, Science Education, Abstract
- McGaw, B. (2002, October). Raising the bar and reducing failures: A possible dream. Invited paper given at the ACER conference “Providing world-class education: What can Australia learn from international achievement studies?”, Sydney.
- Martin Brunner, Cordula Artelt, Stefan Krauss, Jurgen Baumert, Coaching for the PISA test, Learning and Instruction, Volume 17, Issue 2, April 2007, Pages 111-122, ISSN 0959-4752, DOI:http://dx.doi.org/10.1016/j.learninstruc.2007.01.002.
- Raymond J. Adams, Margaret Wu, Programme for International Student Assessment, Organisation for Economic Co-operation and Development. OECD Publishing, 2002, ISBN 9264199519. This publication is also avaible as: PISA (2000) Technical Report (PDF/English version)
- PISA (2003), Learning for Tomorrow's World. PDF
- Organisation for Economic Co-Operation and Development (2001). Knowledge and Skills for Life: First Results from PISA 2000. Paris: OECD Publications.
- Organisation for Economic Co-Operation and Development (2002a). Manual for the PISA 2000 Database. Paris OECD Publications.
- Organisation for Economic Co-Operation and Development (2002b). Sample Tasks from the PISA 2000 Assessment. Paris OECD Publications.
- OECED (Organisation for Economic Co-Operation and Development) (2009, PISA 2006 TECHNICAL REPORT. ISBN 978-92-64-04808-9