Developing Test Items and Scoring Criteria
Matching the Task to the Purpose
The first step in developing a test item should be to link, directly to the test specifications and
content standards, the content and skill that the item is supposed to measure. If the items require a
high level of English proficiency, unrelated to the construct as defined, this will likely affect the
scores for ELLs as well as students in the general population. For content area assessments, only
include items that require high degrees of English proficiency if they are consistent with the
assessment specifications. Examples of items that require a high degree of English proficiency are
those that ask examinees to identify or provide specific definitions or terminology in English that are
unrelated to the construct, or items that are evaluated based on the quality of the language in a
constructed response.
Item writers and reviewers should work to ensure that all test items maintain specificity in their
match to content guidelines. As part of the process of creating and reviewing test material to ensure
that it is appropriate and accessible to examinees, it is important that item developers, state content
review staff, and state review committees analyze each item critically to ensure that it only measures
the intended construct.
Defining Expectations
Because ELLs—just like students in the general population—come from a wide variety of
cultural and educational backgrounds, item writers should not assume that students have had any
previous experience with given tasks. For example, students should be told explicitly what type of
response is acceptable for a constructed-response question, whether it is a paragraph, complete
sentence, list, diagram, mathematical equation, and so on. Likewise, the criteria for the evaluation of
the response should be made clear to the student. As this may add a significant reading load to the
directions, information about how responses will be scored may be especially helpful if students
receive it prior to the test.
Writing Appropriate Directions
Design directions to maximize clarity and minimize the potential for confusion. Consider
options for simplifying the language used for directions (see below). Also consider presenting the
directions orally or in a language other than English if that will provide the best, most understandable
instructions for ELL examinees (see Testing Accommodations).
Using Accessible Language
Using clear and accessible language is a key component of minimizing construct-irrelevant
variance. However, do not simplify language that is part of the construct being assessed (e.g., the
passages on a reading comprehension test or challenging vocabulary that is part of the construct of a
subject area test). In other cases, though, the language of presentation should be as simple and clear
as possible. Some general guidelines for using accessible language are provided below:
• Use vocabulary that will be widely accessible to students. Avoid colloquial and idiomatic
expressions, words with multiple meanings, and unduly challenging words that are not
part of the construct.
• Keep sentence structures as simple as possible to express the intended meaning. For
ELLs, a number of simple sentences are often more accessible than a single more
complex sentence.
• Avoid use of negatives and constructions utilizing not in the questions’ stems and
options as they can cause confusion, especially for ELLs.
• When a fictional context is necessary (e.g., for a mathematics word problem), use a
simple context that will be familiar to as wide a range of students as possible. A schoolbased
context will often be more accessible to ELLs than a home-based context.
Ask reviewers to note any instances where an item can be simplified or clarified to make the
language more accessible. However, do not change language that is part of the construct being
measured.
Presentation
For all assessments, test developers should be aware of formatting issues. Fonts, font sizes, line
breaks in paragraphs, and test directions should all receive a careful review. ELLs who already have
reading ability in another language may have different levels of familiarity with texts that read from
left-to-right, right-to-left, or top-to-bottom. Therefore, clearly and consistently placing elements such
as pictures, page numbers, and other page elements can greatly improve readability for ELLs as well
as other students.
Fairness and Sensitivity
In order to maximize fairness and accessibility for all students, the ETS Standards for Quality and
Fairness (and, as applicable, the ETS International Principles for Fairness Review of Assessments) require that
test materials “minimize the effects of construct-irrelevant knowledge or skills” and “avoid material
that is unnecessarily controversial, inflammatory, offensive, or upsetting.” In applying these
guidelines, it is important to recognize that ELLs have had extremely diverse life experiences and
may be unfamiliar with many U.S. cultural contexts. One way to increase accessibility for ELLs is to
use school-based contexts for test items as often as is practical. For example, research has shown that
mathematics word problems are more accessible for ELLs when set in a school context (e.g.,
counting things such as notebooks, desks, and erasers) than when set in a home context (e.g.,
counting the number of appliances in the home). Other neutral contexts and topics may be
appropriate, as well; consider all available information about the test-taking population.
المادة المعروضة اعلاه هي مدخل الى المحاضرة المرفوعة بواسطة استاذ(ة) المادة . وقد تبدو لك غير متكاملة . حيث يضع استاذ المادة في بعض الاحيان فقط الجزء الاول من المحاضرة من اجل الاطلاع على ما ستقوم بتحميله لاحقا . في نظام التعليم الالكتروني نوفر هذه الخدمة لكي نبقيك على اطلاع حول محتوى الملف الذي ستقوم بتحميله .
|