Ensuring Test Validity
To ensure that PTE Academic is valid and fit for purpose, evidence was collected from the first stages of test development
through the implementation and launch. It was important that the test was linked to other external frameworks of language
proficiency so PTE Academic was benchmarked to the Common European Framework of Reference for Languages (CEF),
the worldwide benchmark for language ability, from the test development stage. CEF was developed by the Council of
Europe (2001) to enable language learners, teachers, universities or potential employers to compare and relate language
qualifications by level.
To ensure potential alignment with the CEF, several steps have been taken:
• Test item writers received specific training in understanding, interpreting and using the CEF. They were then asked to
consider the CEF as the construct model for the test design and provide their estimate of difficulty in terms of the CEF levels
of each item they submitted.
• CEF pre-estimates were reviewed by test experts, revised if necessary and stored in the item bank.
• DIALANG items were included as anchors in the field tests.
• Student responses were rated on the CEF scale independent of the test scores by human raters.
• Additional concordance studies have been devised that will provide further concurrent validity evidence by comparing
PTE Academic scores with other major English language tests, i.e. TOEFL scores and IELTS scores. Preliminary estimates have
provided a concordance range for university admission requirements for the three tests. The preliminary results are available
at www.pearsonpte.com
PTE Academic targets the population of students seeking admission to international education programs where the primary
language of instruction is English. These programs vary considerably in terms of content, level and structure. To support
these diverse institutions, PTE Academic is based around general academic contexts, rather than the specific intended line
of study of test takers. Therefore, topics were selected to cover a wide range of academic contexts but avoiding texts that
require specific domain knowledge that may cause bias toward certain test takers.
All test items and tasks underwent a series of review processes at multiple stages, including author and peer review,
sensitivity analysis, internal review, item migration quality checks and item pool review. Similarly, the psychometric
properties of each task and item are ascertained before they can be included in the item bank from which test versions are
drawn by stratified random sampling.
5
Extensive field testing and analysis
Comprehensive procedures, including two rounds of large-scale field tests and follow-up surveys, have been carried out
to ensure the quality of PTE Academic. Field test 1 (August to October 2007) had a sample size of 6,208, and field test 2
(May to June 2008) had a sample size of 4,169. These field tests were conducted with university students who had a similar
level of language proficiency to that of the prospective PTE Academic test takers. A number of English native speakers were
recruited as a control group for item statistics comparison. For the field test data to be representative of the actual test taker
population, careful steps were taken to exclude test taker data with irregularities in test administrations, as well as data of
test takers identified as insufficiently motivated.
The psychometric analysis of PTE Academic field test data accomplished the following:
• helped determine scoring models
• provided data to be used for training and validating intelligent automated scoring systems
• identified items with substandard quality, which were then eliminated
• established how item scores can be combined to generate reporting scores
• established the minimum number of items for each item type in the test structure
• defined difficulty parameters for all items
As part of Pearson’s commitment to continuous improvement, every effort is made to enhance test design. For example,
analysis of the survey results has led to a number of revisions and adjustments of PTE Academic. The revisions include
optimizing timing of the test items; adjusting testing facilities and task instructions; tailoring item difficulty to the target test
taker population; tightening test specifications; and refining test length, test structure and item layout.
In addition, PTE Academic is supported by a growing research agenda. Data from the field test program have been used
by Pearson staff and also by external experts to support investigative studies into various aspects of the development of
PTE Academic. Some of the ongoing and proposed research includes work on the lexical characteristics of PTE Academic
(O’Loughlin, forthcoming); investigation into the relationship between lexico-grammatical aspects of speech, language
proficiency, task type and language background of the oral component; standard-setting research relating to the CEF; and
concordance studies relating PTE Academic scores to other measures of English language proficiency.
Pearson will further develop this strong research base as PTE Academic is implemented to further the company’s
understanding of the complexities inherent in the assessment of English language proficiency and to support its recognition
in the academic and assessment communities.
المادة المعروضة اعلاه هي مدخل الى المحاضرة المرفوعة بواسطة استاذ(ة) المادة . وقد تبدو لك غير متكاملة . حيث يضع استاذ المادة في بعض الاحيان فقط الجزء الاول من المحاضرة من اجل الاطلاع على ما ستقوم بتحميله لاحقا . في نظام التعليم الالكتروني نوفر هذه الخدمة لكي نبقيك على اطلاع حول محتوى الملف الذي ستقوم بتحميله .
|