English-Language Tests

English-Language Tests
Three separate English-language tests were considered in this study: the TOEFL iBT test,
TOEIC test, and TOEIC Bridge test. Each test will be described in turn.
The TOEFL iBT test. The TOEFL iBT test measures the ability of non-native speakers of
English to communicate orally and in writing in English, to understand English as it is spoken in
North American academic contexts (listening skill), and to understand short passages similar in
topic and style to academic texts used in North American colleges and universities (reading
skill).
Writing and speaking are tested in two constructed-response sections. The Speaking
section consists of six tasks. Two are independent speaking tasks that require the candidate to
respond to a question on a familiar topic, and four are integrated tasks that require candidates to
combine both written and spoken information. Topics are either related to campus life or lecture
situations. The Writing section consists of an opinion essay and an integrated essay, which
requires candidates to listen to a brief lecture, read a short passage, and then respond in writing.
Listening and reading are assessed through selected-response (multiple-choice) items. In
the Reading section, candidates are presented with three reading passages with approximately 14
questions per passage. These questions address basic comprehension, making inferences, and
reading-to-learn skills. In the Listening section, there are six tasks: Two are dialogues that are
based on campus-life situations and four are lecture scenarios with five or six questions per
listening passage. The questions address basic understanding, pragmatic understanding, and
connecting information. The questions are presented orally and in written form; answer choices are
in written form only. Each separate test section (Listening, Reading, Writing, and Speaking) is
reported on a scale that ranges from a low of 0 to a high of 30. The TOEFL iBT reliability (internal
consistency) estimates are: .86, Reading; .87, Listening; .90, Speaking; and .78, Writing.
The TOEIC test. The TOEIC test measures the ability of non-native English
communicators to communicate in English in the global workplace. The TOEIC test addresses
listening comprehension skills and reading comprehension skills. The TOEIC test is a selectedresponse
test and test items are developed from samples of spoken and written English from
6
countries around the world. Each section (Listening and Reading) is reported on a scale that
ranges from a low of 5 to a high of 495.
Two optional modules have been added to the TOEIC test that address speaking and
writing skills. The Writing section measures a test taker’s ability to communicate clearly in
written English (general communication skills to function in a workplace). There are three
separate task types: sentence-level writing; responding to a written request; and writing an
opinion essay. Scores on each task type are weighted, with the sentence-level tasks contributing
the least weight and the opinion essay contributing the greatest weight.
The Speaking section measures a candidate’s ability to speak clearly, to communicate, and
to demonstrate understanding of material. There are six tasks of increasing complexity, and a
weighting system is applied to scores, with the more basic tasks contributing less weight than the
more complex tasks. Task 1 (considered the most basic task) consists of a read-aloud section that is
scored for both pronunciation and intonation. In Task 2, candidates describe a picture. Task 3
requires candidates to respond to a series of questions with an audio lead-in that sets the context.
Task 4 is similar, except that the questions are based on textual information. In Task 5, candidates
are presented with a problem that is in audio form only and they are asked to propose a solution.
Task 6 (the most heavily weighted task) requires them to give an opinion on a topic that is
presented in the form of audio and text stimulus. Results for the Writing and Speaking sections are
each reported on a scale that ranges from a low of 0 to a high of 200. The TOEIC reliability
(internal consistency) estimates are: .93, Reading; .92, Listening; .89, Speaking; and .77, Writing.
The TOEIC Bridge test. The TOEIC Bridge test measures the listening-comprehension
and reading-comprehension skills of non-native English communicators at a more basic level
than the TOEIC test. The test consists of selected-response items, and results are reported on a
scale that ranges from 10 to 90. The TOEIC Bridge test reliability (internal consistency)
estimates are .86 for Reading and .86 for Listening.
Report structure. The remainder of this report is presented in three major sections. The
first section describes the standard-setting methods that were implemented to establish the
cutscores corresponding to the CEFR proficiency levels on each of the English-language tests.
The second section focuses on the results specific to the TOEFL iBT test. The third section
focuses on the TOEIC test and the TOEIC Bridge test. It is important that panelists are familiar
with not only the test and its use but also with the test population. This background will help
7
inform their standard-setting judgments. To this end, two different panels of experts (with
minimal overlap) were convened to participate in setting the cutscores on the tests—one panel
for the TOEFL iBT test, and one panel for the TOEIC test and the TOEIC Bridge test. The two
panels reflected the different contexts for which the tests are primarily used: Panel 1 (TOEFL
iBT test; higher education) and Panel 2 (TOEIC test and the TOEIC Bridge test; business). A
small number of panelists (n = 5) had experience with both sets of tests and test populations, and
served on both panels. The composition of each panel is discussed in more detail at that start of
the sections on the panels.