Modern Test Design

Modern Test Design

PTE Academic is a multi-level, integrated-skills test of English language proficiency. It is designed to assess English
language competence set in the context of academic programs of study that are available around the world. The
development of the tasks and items for the test followed consultation with external stakeholders and test development
professionals. PTE Academic is supported by two external advisory boards composed of experts in applied lingusitics
and assessment who have overseen the development of the test from a professional perspective.
PTE Academic consists of 20 item types reflecting different modes of language use and setting different response tasks
or response formats. The maximum duration of the test is three hours.
PTE Academic scores will be delivered to the test takers online, typically within five business days. There are two versions
of the score report: one for the test taker and an official institution version. The score report provides three types of
scores: an Overall Score, scores for Communicative Skills (i.e. Listening, Reading, Speaking and Writing) and scores for
Enabling Skills (i.e. Grammar, Oral Fluency, Pronunciation, Spelling, Vocabulary and Written Discourse). The score scale
ranges from 10 to 90.
A number of innovative features have been integrated into the design of PTE Academic. These developments build
on advances in the understanding of language competence in the academic environment, including the importance
of integrated skills, and on advances in technology, particularly in the areas of scoring and reporting of results. As
the worldwide leader in publishing and assessment for education, Pearson is using several of its proven, proprietary,
patented technologies to automatically score test takers’ written and spoken performance on PTE Academic.
3
Major aspects of PTE Academic, which contribute to the reliability of the measurement of the test and the validity of the test
content and purpose, include:
• high levels of reliability, including the automated marking of items
• detailed peer review and sensitivity analysis included as part of test development procedures
• integrated skills item types that reflect the context in which the test taker will use English
• quality assurance procedure at every stage, including detailed analysis of item functioning in
addition to item and task difficulty
• item banking that ensures equated test papers and test security
• a wide-ranging research agenda with conference presentations and peer-reviewed publications
• test development procedures and research open to external scrutiny.

Initial test reliability studies

PTE Academic is the focus of research to ensure the reliability of its scores. Below are two examples of studies that have
been conducted based on samples drawn from the large-scale, two-part field testing program that Pearson implemented to
inform the development of the test.

Reliability of human ratings and machine-generated scores

After calibrating the automated scoring engines, an independent set of test takers’ data which had not been used for
training the scoring engines was used to investigate the reliability of human ratings and machine-generated scores and
the degree to which the machine generated scores can predict human ratings. The overall reliability for both human marks
and for machine marks was 0.97. This is a very high coefficient and demonstrates the accuracy of both human and machine
marking. The correlation between human and machine marking was also very high, at 0.96 indicating that the machinegenerated
scores explain 92% of the variance of the human ratings.
Overall, this study demonstrated that the automated model can predict human ratings to a high degree for this test. In the
future, randomly selected samples from live administrations of PTE Academic will be rescored to ensure that the high level of
accuracy of marking is maintained.