PISA

Draft

Definition

PISA = Programme for International Student Assessment

“PISA assesses how far students near the end of compulsory education have acquired some of the knowledge and skills that are essential for full participation in society. In all cycles, the domains of reading, mathematical and scientific literacy are covered not merely in terms of mastery of the school curriculum, but in terms of important knowledge and skills needed in adult life. [..] n the PISA 2003 cycle, an additional domain of problem solving was introduced to continue the examination of cross-curriculum competencies.” ([1], retrieved 18:31, 30 September 2006 (MEST)).

PISA is often quoted in discussions about educational policy. Most people do not really know what is measured and just quote rankings.

PISA 2000 usually included three modules

• Mathematics
• Sciences
• Reading

PISA 2003 added:

• Problem solving (cross-curicular competencies)

Options:

• Countries can choose and their own.
• E.g. in Switzerland there was a survey of 9th graders (in addition to 15 year olds).

In addition to tests, PISA also administers questionnaires to both individual learners and schools. Datasets that combine questionnaire results and test results are available.

Mathematics skills

Design of the tests

According to quotations from PISA (2003):

In total, 85 mathematics items were used in PISA 2003. These tasks, and also those in reading, science and problem solving, were arranged into half-hour clusters. Each student was given a test booklet with four clusters of items - resulting in two hours of individual assessment time.

These clusters were rotated in combinations that ensured that each mathematics item appeared in the same number of test booklets, and that each cluster appeared in each of the four possible positions in the booklets.

Such a design makes it possible to construct a scale of mathematical performance, to associate each assessment item with a point score on this scale according to its difficulty and to assign each student a point score on the same scale representing his or her estimated ability.

The relative ability of students taking a particular test can be estimated by considering the proportion of test items they answer correctly. The relative difficulty of items in a test can be estimated by considering the proportion of test takers getting each item correct.

Once the difficulty of individual items was given a rating on the scale, student performance could be described by giving each student a score according to Each student was given a subset from a broad pool of mathematics tasks... ...and their performance was established on a scale... A Profile of Student Performance in Mathematics

the hardest task that they could be predicted to perform.

The six proficiency levels

PISA (2003:47):

At Level 6, students can conceptualise, generalise, and utilise information based on their investigations and modelling of complex problem situations. They can link different information sources and representations and flexibly translate among them. Students at this level are capable of advanced mathematical thinking and reasoning. These students can apply this insight and understanding, along with a mastery of symbolic and formal mathematical operations and relationships, to develop new approaches and strategies for attacking novel situations. Students at this level can formulate and precisely communicate their actions and reflections regarding their findings, interpretations, arguments, and the appropriateness of these to the original situations.

At Level 5, students can develop and work with models for complex situations, identifying constraints and specifying assumptions. They can select, compare, and evaluate appropriate problem-solving strategies for dealing with complex problems related to these models. Students at this level can work strategically using broad, well-developed thinking and reasoning skills, appropriately linked representations, symbolic and formal characterisations, and insight pertaining to these situations. They can reflect on their actions and can formulate and communicate their interpretations and reasoning.

At Level 4, students can work effectively with explicit models for complex concrete situations that may involve constraints or call for making assumptions. They can select and integrate different representations, including symbolic ones, linking them directly to aspects of realworld situations. Students at this level can utilise well-developed skills and reason flexibly, with some insight, in these contexts. They can construct and communicate explanations and arguments based on their interpretations, arguments and actions.

At Level 3, students can execute clearly described procedures, including those that require sequential decisions. They can select and apply simple problem-solving strategies. Students at this level can interpret and use representations based on different information sources and reason directly from them. They can develop short communications reporting their interpretations, results and reasoning.

At Level 2, students can interpret and recognise situations in contexts that require no more than direct inference. They can extract relevant information from a single source and make use of a single representational mode. Students at this level can employ basic algorithms, formulae, procedures or conventions. They are capable of direct reasoning and making literal interpretations of the results.

At Level 1, students can answer questions involving familiar contexts where all relevant information is present and the questions are clearly defined. They are able to identify information and to carry out routine procedures according to direct instructions in explicit situations. They

can perform actions that are obvious and follow immediately from the given stimuli.

See also learning level

Links

• OECD PISA .org Home page (stable URL)
  • From here you can download some free reports and some (huge) datasets
• OECD Pisa Home Page (direct link)

Australian websites - have additional stuff like sample questions or a mirror for the datasets

• PISA Sample Questions
• PISA FTP archive

Swiss pisa data

• FORS Swiss foundation for research in social sciences. You can get the swiss datasets from here.

Canada

• Programme for International Student Assessment (PISA)

Bibliography

To do (there is a whole lot of literature).

• PISA test items and school textbooks related to science: A textual comparison (2008). PISA test items and school textbooks related to science: A textual comparison, Science Education, Abstract

• McGaw, B. (2002, October). Raising the bar and reducing failures: A possible dream. Invited paper given at the ACER conference “Providing world-class education: What can Australia learn from international achievement studies?”, Sydney.

• OECD (2001). Knowledge and skills for life: First results from PISA 2000. Paris: OECD Publications.

• Martin Brunner, Cordula Artelt, Stefan Krauss, Jurgen Baumert, Coaching for the PISA test, Learning and Instruction, Volume 17, Issue 2, April 2007, Pages 111-122, ISSN 0959-4752, DOI:http://dx.doi.org/10.1016/j.learninstruc.2007.01.002. http://dx.doi.org/10.1016/j.learninstruc.2007.01.002.]

• Raymond J. Adams, Margaret Wu, Programme for International Student Assessment, Organisation for Economic Co-operation and Development. OECD Publishing, 2002, ISBN 9264199519. This publication is also avaible as: PISA (2000) Technical Report (PDF/English version)

• PISA (2003), Learning for Tomorrow's World. PDF