Appendix G: NAEP Technology And Engineering Literacy Preliminary Achievement Level Definitions

Appendix G: NAEP Technology and Engineering Literacy Preliminary Achievement Level Definitions

Congress authorized the National Assessment Governing Board to develop appropriate student achievement levels on the NAEP. The achievement level definitions are statements of what students should know and be able to do on NAEP at grades 4, 8, and 12. To fulfill its statutory responsibility, the Governing Board developed a policy to guide the development of achievement levels for all NAEP subjects. Three levels of achievement were identified to provide the public, educators, and policymakers with information on student performance on NAEP. These levels—Basic, Proficient, and Advanced—are used as a primary means of reporting NAEP results to describe "how good is good enough" at grades 4, 8, and 12.

Table G-1 displays the Governing Board's generic policy definitions for Basic, Proficient, and Advanced achievement that pertain to all NAEP subjects and grades.

Table G-1. Generic Achievement Level Policy Definitions for the National Assessment of Educational Progress

Achievement Level

Policy Definition

Advanced

This level signifies superior performance.

Proficient

This level represents solid academic performance for each grade assessed. Students reaching this level have demonstrated competency over challenging subject matter, including subject-matter knowledge, application of such knowledge to real-world situations, and analytical skills appropriate to the subject matter.

Basic

This level denotes partial mastery of prerequisite knowledge and skills that are fundamental for proficient work at each grade.

During the framework development process the project committees are asked to develop preliminary achievement level definitions, based on the generic policy definitions, to guide item development. Essentially the purpose of these statements is to provide examples of what students performing at the Basic, Proficient, and Advanced achievement levels should know and be able to do in terms of the technology and engineering literacy content areas and practices identified in the framework. The intended audiences for these preliminary definitions are the NAEP assessment development contractor and item writers. The definitions are to be used to ensure that a broad range of items is developed at each grade level. Tables G-2, G-3, and G-4 present the preliminary achievement level definitions for grades 4, 8, and 12 as bullet points to clearly illustrate the technology and engineering literacy content and practices expected at each grade level.

The preliminary definitions include illustrative statements selected from the framework's technology and engineering literacy content and practices. The statements are not intended to represent the entire set of objectives from the assessment targets, practices, or contexts, nor do the preliminary achievement level definitions denote a sense of priority or importance based on the statements selected.

After the assessment is administered, broadly representative panels engage in a standard setting process to determine the achievement level cut scores on the NAEP scale. The cut scores represent the minimum score required for performance at each NAEP achievement level. Part of this standard setting process is the development of a set of paragraphs, derived from the preliminary achievement level definitions, to be used in reporting the NAEP Technology and Engineering Literacy Assessment results to the general public and other audiences. At each grade level there will be paragraphs describing what students should know and be able to do at the Basic, Proficient, and Advanced level in terms of the technology and engineering literacy content and practices identified in the framework.

Further information on NAEP achievement levels can be found at www.nagb.gov.

Table G-2. Grade 4 Preliminary Achievement Level Definitions

 

BASIC

Students know that:

PROFICIENT

Students know that:

ADVANCED

Students know that:

Technology and Society

A new tool, product, or machine may have benefits and costs.

The introduction of a new tool, product, or machine may change how people live and work.

The introduction of a new tool, product, or machine usually brings both benefits and costs, and it may change how people live and work.

Design and Systems

Tools help people do their work, and certain common tools have particular uses.

Different tools are better for different purposes.

Different tools are better for different purposes because they have different features and function in different ways.

Information and Communication Technology

Digital and network tools can be used to answer questions.

Digital and network tools and media resources are helpful for answering questions, but they can sometimes be biased or wrong.

Digital and network tools and media resources are helpful for answering questions, and information gathering can help to determine whether they are biased or wrong.

 

BASIC

Students are able to:

PROFICIENT

Students are able to:

ADVANCED

Students are able to:

Technology and Society

Name two different technologies in their own lives.

Compare the impact of two different technologies on their own lives.

Compare the ways that their lives would change if one or the other technology were to disappear.

Design and Systems

Build a model from a kit with instructions.

Build and test a model to see if it works as intended.

Build and test a model to see if it works as intended, and use test data to determine how to improve the model.

Information and Communication Technology

Use digital and network tools and media resources to collect, organize, and display data.

Use digital and network tools and media resources to collect, organize, and display data in order to answer questions and solve problems.

Select and use digital and network tools and media resources to collect, organize, and display data in order to answer questions and solve problems.

Table G-3. Grade 8 Preliminary Achievement Level Definitions

 

BASIC

Students know that:

PROFICIENT

Students know that:

ADVANCED

Students know that:

Technology and Society

Technology interacts with society, sometimes bringing about changes in a society's economy.

Technology interacts with society, sometimes bringing about changes in a society's economy, and culture, and may lead to new needs and wants.

Technology interacts with society, sometimes bringing about changes in a society's economy, politics, and culture, and often leading to the creation of new needs and wants.

Design and Systems

One tool is better than another for a given task as a result of prior improvements.

Tools have been improved over time to further the reach of hands, voices, memory, and the five human senses.

Tools have been improved over time to further the reach of hands, voices, memory, and the five human senses, and to do so more efficiently, accurately, or safely.

Information and Communication Technology

Some information available through electronic means is exaggerated or wrong.

Increases in the ease by which knowledge can be published have heightened the need to check sources for possible distortion, exaggeration, or misrepresentation.

There are various effective ways to check information available through electronic means to identify possible distortion, exaggeration, or misrepresentation.

 

BASIC

Students are able to:

PROFICIENT

Students are able to:

ADVANCED

Students are able to:

Technology and Society

Identify the impacts of a given technology in a society.

Identify the impacts of a given technology in a society, and predict how it might affect a different society.

Compare the impacts of a given technology on two different societies, noting factors that may make a technology appropriate and sustainable in one society but not in another.

Design and Systems

Design and build a simple model that meets a requirement

Design and build a simple model that meets a requirement, fix it until it works (iteration), test it and gather and display data that describe its properties using graphs and tables.

Design and build a simple model that meets a requirement, fix it until it works (iteration), test it and gather and display data that describe its properties using graphs and tables. Interpret the results to suggest improvements and new designs.

Information and Communication Technology

Select and use appropriate digital and network tools and media resources to collect, organize, and display data.

Select and use appropriate digital and network tools and media resources to collect, organize, analyze, and display supporting data to answer simple questions and test basic hypotheses.

Select and use appropriate digital and network tools and media resources to collect, organize, analyze, and display supporting data to answer complicated questions and test hypotheses, and explain how to go about this to others working on the same project.

Table G-4. Grade 12 Preliminary Achievement Level Definitions

 

BASIC

Students know that:

PROFICIENT

Students know that:

ADVANCED

Students know that:

Technology and Society

Changes caused by the introduction and use of a new technology can be gradual or rapid as well as big or small.

Changes caused by the introduction and use of a new technology can range from gradual to rapid and from subtle to obvious, and can change over time.

Changes caused by the introduction and use of a new technology can range from gradual to rapid and from subtle to obvious and can change over time. These changes may vary from society to society as a result of differences in a society's economy, politics, or culture.

Design and Systems

The development of tools has influenced and advanced society.

 

The evolution of tools and materials has played an essential role in the development and advancement of cities and industrial societies.

The evolution of tools and materials has played an essential role in the advancement of civilization, from the establishment of cities and industrial societies to today's global trade and commerce networks.

Information and Communication Technology

Advanced search techniques can be used with digital and network tools and media resources to locate information.

Advanced search techniques can be used with digital and network tools and media resources to locate information and to check the credibility and expertise of sources.

Advanced search techniques can be used with digital and network tools and media resources to locate different kinds of information and to check credibility and expertise of different types of sources.

 

BASIC

Students are able to:

PROFICIENT

Students are able to:

ADVANCED

Students are able to:

Technology and Society

Analyze the cultural, social, economic, and/or political changes that may be triggered by the introduction of a specific technology into a society.

Analyze cultural, social, economic, and/or political changes that may be triggered by the transfer of a specific technology from one society to another.

Analyze cultural, social, economic, and/or political changes that may be triggered by the transfer of a specific technology from one society to another. Include anticipated and unanticipated effects.

Design and Systems

Construct and test a model to see if it meets the requirements of a problem, then suggest improvements.

Construct and test several models to determine which is best in meeting the requirements of a problem.

Construct and test several models to see if they meet the requirements of a problem. Combine features to achieve the best solution.

Information and Communication Technology

Select digital and network tools and media resources to gather information and data on a practical task.

Select digital and network tools and media resources to gather information and data on a practical task, and justify choices based on the tool's efficiency and effectiveness for a given purpose.

Select digital and network tools and media resources to gather information and data on a practical task, justify choices based on the tool's efficiency and effectiveness for a given purpose, and advise others on which tools and resources would best meet their needs.