Federal Initiatives for Improved Assessment Techniques

Diane S. Reed

Introduction

The Technology Literacy Challenge that was given to the country in 1996 envisioned a 21^st century where all students are technology literate and have access to quality educational resources. In modern classrooms, educational technology improves teaching and learning while also equipping students with the skills needed for tomorrow's workforce.

Federal Initiatives in Technology Studies -Lessons Learned

Evaluation activities at the U.S. Department of Education address what can be learned from the federal investment in educational technology, the effectiveness of federal programs involving educational technology, and how the Federal Government can best catalyze movement toward the National Technology Goals. “E-Learning: Putting a World-Class Education at the Fingertips of All Children,”[U. S. Dept of Education, 2001 #1] the new national plan for educational technology released in January 2001, recognizes the need for evaluation. “While we have learned a tremendous amount about the implementation and use of technologies for teaching and learning in the past few years, the need for an expanded, ongoing national research and evaluation program to improve the next generation of technology applications for teaching and learning is profound.” The e-Learning Report encourages the federal government, foundations, and states to " . . . support the development, dissemination and adoption of user-friendly evaluation tools, including specifying the outcome measures they expect their grantees to measure. It also encourages government, foundations, and states to . . ."make available on the Internet evaluation reports and studies on the use of technology in education so that researchers and evaluators can benefit from lessons learned.

During the past year, the U. S. Department of Education partnered with the North Central Education Lab (NCREL) and sponsored three regional conferences for the state departments of education to come together to develop a systematic approach to evaluation of technology. (Guidelines for Evaluating Technology in Education: Regional Conferences 2000: Denver, Chicago, Atlanta). Foremost in the discussion among regional conference participants was the understanding that current practices for evaluating the impact of technology in education need to be broadened. Although technology often plays a role in innovative practices for teaching and learning, new ways to identify and measure the skills and knowledge that students gain from using technology are lacking. (McNabb, 2000)

In September 2000 the Secretary’s Conference on Educational Technology: Measuring the Impacts and Shaping the Future brought representatives from K-12, higher education, and the technology community together to focus on the effective use of technology and evaluation in our schools and consider the following questions:

• What value-added does technology bring to schools?
  
  
• What does it "take" at the system level to enable learners, teachers, administrators, and communities to use technology effectively?
  
  
• What assessment strategies and designs are currently being used to capture the value-added technology brings to schools?
  
  
• How do schools need to evolve in order to become a high-tech, high-performance enterprise that builds the capacity of learners, teachers, administrators, and community members to use these emerging technologies wisely and effectively.

The conference final report states . . . "If students and teachers are to take full advantage of what technology makes possible in teaching and learning, schools must change. They must become more student-centered, more focused on 21st century skills, more open to innovation through technology, more willing to fully support and grow the infrastructure they install, and they must be lead by educators who recognize the critical role technology plays in defining an excellent education in this digital age. This will not happen without changes in the K-12 assessment." (Lemke, 2000)

To address the overwhelming need for new assessments, the U.S. Department of Education awarded a grant to SRI International to develop a prototype technology-based assessment to help address the dearth of appropriate student learning measures available to inquiry-oriented, technology-supported projects. While these assessment prototypes are still under development, they do offer illustrations of the way that technology supports can make classroom assessment of complex skills more feasible. One major advantage of embedding assessment within learning activities is the heightened focus on learning outcomes. (Means, 2000)
(http://www.ed.gov/Technology/techconf/2000/means_paper.html).

SRI International also received a grant from the U.S. Department of Education to support planning for a major program of rigorous, systematic educational technology research. In their paper, Geneva Haertel and Barbara Means state that the new learning assessments should include:

· "Extended, performance tasks

· Mechanisms for students to reveal their problem-solving, to describe their rationale for proceeding through the task, and to document the steps they follow

· Opportunities to demonstrate social competencies and collaboration

· Scoring rubrics that characterize specific attributes of performance

· Scoring rubrics that can be used across tasks of varying content

· Integration with curriculum content

· Links to content and performance standards

· Content negotiated by teachers" (Haertel, 2001)

(http://www.sri.com/policy/designkt/synthe1b.doc)

In order to encourage this work in the field, the Department of Education, and the North Central Regional Education Lab, awarded three small grants. (1) Cherry Creek, Colorado and Ann Mastergeorge - UCLA, (2) Montgomery County, Maryland and Interactive, Inc, (3) Mantua Elementary School, Fairfax County Virginia and Walter Heinecke -University of Virginia. This work is progressing this year as these school systems begin the process of assessing the effectiveness of technology.

http://www.ed.gov/Technology/techconf/2000/presentations.html

Assessment Expectations in Federally funded Technology Initiatives

The Technology Innovation Challenge Grants Program provides detailed evaluation guidelines. "OERI recommends that the evaluator draw a causal map of the project. The map should provide a visual representation of the chain of events through which the project expects to achieve its outcomes." (OERI, 1999) Guided by such a map the evaluation should yield information about the project that is essential to the success of the project. The framework of the evaluation should also include a plan for assessing whether essential project outcomes are being achieved. The first step in the process should be to collect baseline data on their status and a schedule for continuing to assess and document their milestones. OERI would like to see both quantitative and qualitative measures from studies involving large samples and qualitative data from case studies that take a more in depth look at teaching and learning. The goal is to create a database that documents the impact across large numbers of project participants, and provides information about the role of the project in producing such impact.

OERI expects to receive at least one major report of findings from each project each year. It should be accompanied by an updated version of the overall progress of the evaluation, and updated causal map and evaluation framework. As each project prepares its final report the following set of common accountability the following questions should be answered:

1. What applications of technology were developed with grant funds

2. What Evidence is available about project outcomes and impact

3. What has been learned about the conditions required for that intervention to operate successfully?

4. What is required to ensure that the intervention will be sustained at the local site beyond the grant award period?

5. Has there been any adoption to-date of the intervention beyond the initial site?[OERI, 1999 #6]

Diane S. Reed - is the former Technology Teacher in Residence in the Office of Educational Technology at the U.S. Department of Education. She is currently an Instructional Technology Specialist for Fairfax County Public Schools in Fairfax, Virginia and a doctoral student in Education Evaluation and Policy at the University of Virginia. (http://home.cox.rr.com/dreed2/).

REFERENCES

Haertel G. a. Means., B. (2001). Stronger Designs for Research on Educational Uses of Technology: Conclusion and Implications. Menlo Park, CA, SRI International.

Lemke C. 2000 The Secretary's Conference on Educational Technology: Measuring the Impacts and Shaping the Future. [on-line] available http://www.ed.gov/Technology/techconf/2000/report.html

McNabb M. 2000 Summary of the Regional Conferences [on-line] available http://www.ncrel.org/tech/confrprt/index.html

Means B. a. Penuel., B and Quellmallz, Edys 2000 Developing Assessments for Tomorrow's Classrooms [on-line] available http://www.ed.gov/Technology/techconf/2000/means_paper.html

OERI (1999). Technology Innovation Challenge Grant Evaluation Guidelines. Washington, D.C., U. S. Department of Education.