The Reading Framework for the National Assessment of Educational Progress (NAEP) contains the rationale for the aspects of reading to be assessed and criteria for development of the assessment. The national consensus process used to develop this Framework was guided by the following:

• A general pattern of consensus development, which was set forth by law and which has evolved over time, that calls for "active participation of teachers, curriculum specialists, subject matter specialists, local school administrators, parents, and members of the general public" (P. L. 100-297, Part C, 1988).

• The decision that the reading assessment would pilot state-by-state comparisons, which increased the importance of the consensus process.

• Recognition that experts, educators, and interest groups in reading often hold diverse and conflicting views that have not been completely clarified, much less settled, by research in the field.

To prepare the new Reading Framework, NAGB awarded a contract to the Council of Chief State School Officers (CCSSO). The consensus process involved a steering committee, a planning committee, and CCSSO project staff. The following describes the composition of each of these groups and outlines the roles each played.

• The steering committee consisted of members representing 16 national organizations (see appendix B). This group met initially in October 1989 to provide guidance for the consensus process and met two additional times to respond to the progress of the work on the Framework and offer guidance. Drafts of each version of the developing Framework were sent to members of this committee for review and reaction.

• The 15-member planning committee of consisted of experts in reading, including several university professors, the dean of a college of education, a classroom teacher, a school administrator, state-level specialists in reading and assessment, and a representative of the business community (see appendix B). This committee was established to identify the goals for the assessment and prepare the Framework. It met with the steering committee early in the project and three subsequent times as it developed the plan for the new reading assessment. A subgroup of this committee met in January 1990 to develop the specifications for the assessment.

• CCSSO project staff included a director, a project coordinator, and a project associate (see appendix B). In addition to meeting with the steering committee and the planning committee, this group maintained regular contact with NAGB staff and Board members.

Steering Committee Guidelines

At its first meeting, the steering committee established guidelines to be followed in developing the Framework:

• Focus on outcomes or be performance oriented, rather than reflect a specific instructional or theoretical approach. The Framework should not focus on the specific reading skills that lead to outcomes but on the outcomes themselves.

• Address the increasing reading literacy needed for employability, personal development, and citizenship.

• Expand the range of assessment tools to include such techniques as open-ended questions and other new approaches and formats appropriate to the objectives.

• Account for contemporary research on reading and literacy without implying a departure from the primary focus on outcomes.

• Provide information to policymakers and educators that will assist them in making decisions about the improvement of educational performance.

• Include general statements about what is reasonable reading performance for 4th-, 8th-, and 12th-grade students.

The development of this Framework for the NAEP in Reading was guided by several considerations and principles -- some resulting from the nature and purpose of the assessment, others reflecting the thinking and values of the planning committee and the steering committee.

The Nature of Assessment

The NAEP in Reading is an assessment conducted on a representative sample basis, not a test of individual performance. It is designed to inform policymakers and the public about educational achievement in the United States in broad terms. It is not designed to diagnose the causes of problems in individual students, schools, or districts. Although analysis of NAEP data can direct and inform research, the assessment is not designed primarily as a research instrument.

NAEP data in reading can be used to inform participating states of some of the strengths and weaknesses of their students' performance. The data can also be used to inform individual states and the country as a whole of some of the links among reading-related activities in schools, student and teacher background characteristics, and performance in certain aspects of reading.

Reading Literacy

The term reading literacy is not intended to imply only basic or functional literacy. Rather the term connotes a broader sense of reading, including knowing when to read, how to read, and how to reflect on what has been read. Contemporary research indicates that reading is a complex process that involves an interaction among the reader, the text, and the context in which something is read. Because reading is not considered to be a simple, unidimensional skill, reading achievement cannot be represented adequately by a single score. Whereas some aspects of reading, such as how well a reader summarizes a passage, can be measured directly, other aspects, such as how a reader utilizes reading strategies, can be measured only indirectly.

Assessment and Instruction

Assessment alone should not drive instruction. Educators, in response to societal expectations, should set goals or standards that assessments can measure and serve. However, teachers often do use assessment tasks to set priorities for what they teach. With this in mind, the planning committee determined that the NAEP in Reading must contain passages and tasks so similar to those that students encounter in classrooms and in their own reading that, should teachers choose to do so, they could use the kinds of passages and tasks found on the assessment to set priorities in their classrooms without distorting instruction.

New Methodologies

A consensus of societal goals and values, the best available reading theory and research, and the wisdom gained from classroom experience -- but not primarily traditional psychometric theory -- should drive the design of the assessment. The Reading Framework must not promote assessment items that in terms of correlating with reading ability, have only statistical utility, but do not correspond to sound concepts of reading or resemble desirable classroom activities. Although the aspects of reading that can be measured in a project of national scope are limited by available resources, development time, and methodology, every effort must be exerted to make the best use of available methodologies and resources. Assessment capabilities must be driven forward so that important aspects of reading that now elude measurement can be addressed in the near future. Toward these ends, new types of items and new methods of analysis were used for the NAEP in Reading, along with well-constructed, multiple-choice items.

The Basis of the Framework

Because the 1992, 1994, and 1998 reading assessments include reporting of state-level results, concerns about the strength of the assessment design and about how the results will be reported have increased. (In 2000, reading will be tested at the national level in grade 4.) Aware of these concerns, the planning committee attempted to develop a Framework that provides for an assessment that reflects both professional consensus and the best technical design possible in a large-scale reading assessment. The committee was aware that the Framework should represent a broad concept of reading that reflects the views of contemporary research, yet recognizes those aspects of reading currently emphasized in local school districts. It also should recognize that a variety of approaches and programs can produce successful readers. Therefore, in preparing this Framework, the committee considered a variety of perspectives, emphases, and opinions among professionals in universities and in state and local school districts. They began their work by identifying the most important goal of reading literacy education.

Previous

Contents

Next