Cognitive Science and Assessment. ERIC Digest.

by Boston, Carol

Cognitive science is devoted to the study of how people think and learn and how, when, and whether they use what they know to solve problems (Greeno, Collins, & Resnick, 1997; National Research Council, 2001). The cognitive perspective in education encompasses how learners develop and structure their knowledge in specific subject areas and how assessment tasks might be designed to enable students to demonstrate the knowledge and cognitive processes necessary to be judged proficient in these subject areas. This Digest provides educators with an overview of some important facets of cognitive science research and suggests implications for classroom assessment.

HOW DO EXPERTS AND NOVICES DIFFER IN THEIR APPROACH TO PROBLEMS?

Education researchers study the thinking of experts in various subject areas to gain an understanding of what concepts and procedures are most important to teach and how they are interrelated. The concept is that educators can and should be moving students along a continuum toward real-world subject mastery based on a deep understanding of how subject knowledge is organized (Bereiter & Scardamalia, 1986).

When faced with a problem, learners tend to search their memories for a schema, or learned technique for organizing and interpreting information in a certain subject, in
order to solve it (Rumelhart, 1980). Over time, individuals build mental models to guide their problem solving efficiently so they do not depend on trial-and-error approaches and can instead create analogies and make inferences to support new learning (Glaser & Baxter, 1999).

When compared with novice learners, experts in a subject are notable for how
well-organized their knowledge is, which in turn enables them to see patterns quickly, recall information, and study novel problems in light of concepts and principles they know already (Glaser & Chi, 1988). In other words, their schemas are well-connected and they are able to retrieve chunks of information relevant to a task at hand. Experts also have strong problem-solving skills. They know what they know and what they don'tknow, and plan and monitor the implementation of various mental strategies (Hatano, 1990).

COGNITIVE SCIENCE IN THE CLASSROOM

Ideally, developmental models of learning could be created that note the typical
progression and milestones as a learner advances from novice to competent to expert and describe the types of experiences that lead to change. For example, students generally have naive or intuitive understandings of the sciences, based in part on misconceptions that are corrected as they are exposed to new learning (e.g., Gabel, 1994, Feldman & Minstrell, 2000). And while there are individual differences among learners, when large samples are studied, patterns tend to emerge, particularly related to erroneous beliefs and incorrect procedures. For example, there appear to be a certain limited number of "subtraction bugs" that account for almost all of the ways young children make mistakes when learning to subtract two- or three-digit numbers, and these are constant even across languages (Brown and Burton, 1978).

Allowing for variations among learners, it is possible to discover the most common
pathways toward acquiring knowledge and use this information diagnostically. For
example, Case, Griffin, and colleagues have developed an assessment tool based on their empirical research regarding how children from ages 4 to 10 change in their
conception of numbers through growth and practice. While 4-year-olds can count
groups of objects, they have to guess if they face a theoretical question such as, "Which is more--four or five?" Between 4 and 6, most children develop a "mental number line" that helps them envision the answer to such a question, even when actual objects aren't present. Between 6 and 8, children gradually come to envision other number lines for counting by 2s, 5s, 10s, and 100s. By 10, many children have a better understanding of the base-10 number system, which enables them to reach a more sophisticated understanding of concepts such as regrouping and estimation (Case, 1996; Griffin and Case, 1997). Teachers can use assessments based on this research to determine their next steps in arithmetic instruction.

More research has been done about domain structure in some disciplines than in
others. Mathematics, physics, beginning reading, and U.S. history are among the areas that have been studied (see, for example, Niemi, 1996, and Wineburg, 1996).
Subject-area standards such as the National Council of Teachers of Mathematics
Standards generally reflect current thinking on cognitive processes and are a good
place for teachers to begin their explorations of this topic. The National Research
Council's How People Learn: Brain, Mind, Experience, and School
(http://stills.nap.edu/html/howpeople1/) provides another helpful introduction.

HOW DO LEARNERS STORE AND ACCESS KNOWLEDGE?

Memory may be divided into two types: short-term, or working memory, which
determines how much mental processing can go on at any one time, and long-term
memory, where people organize their content knowledge. Short-term memory, or working memory, is connected with fluid intelligence, or the ability to solve new and
unusual problems, while long-term memory is connected to crystallized intelligence, or the bringing of past experience to bear on current problems (Anderson, Greeno, Reder, and Simon, 2000). When students are learning a new skill, they must rely heavily on their working memory to represent the task and may need to talk themselves through a task. As the skill moves into long-term memory, it becomes fluent, and eventually, automatic (Anderson, 1982).

To support the learning process, students can be taught meta-cognitive skills, or
techniques to reflect on and assess their own thinking. To improve reading
comprehension, for example, young children can be taught to monitor their
understanding of passages by asking questions, summarizing, clarifying any
uncertainties, and predicting next events (Palincsar & Brown, 1984).

HOW CAN ASSESSMENT DESIGNERS USE FINDINGS FROM COGNITIVE SCIENCE?

The design of any assessment should begin with a statement of purpose for the
assessment and a definition of the particular subject area or content domain. How do people demonstrate knowledge and become competent in this domain? What important aspects of learning do we want to draw inferences from when measuring student achievement in a given subject area? What situations and tasks can we observe to make the appropriate inferences?

Cognitive science calls for test developers to:

* Work from a deep knowledge of the central concepts and principles of a given subject area, and the most important related information.

* Identify or develop those tasks that allow students to demonstrate their
understanding and skills in these areas, as opposed to rote memorization.

* Make sure tasks or questions are sufficiently complex to get at how students have organized their knowledge and how and when they use it.

* Emphasize the contents of long-term memory rather than short-term, or working, memory by not burdening test-takers withrequirements to track a large number of response options or major quantities of extraneous information while answering a question.

* Emphasize relevant constructs--for example, a mathematics assessment should not over-emphasize reading and writing, unless communicating about mathematics is the skill to be measured.

* Not limit choice of item format. Both multiple-choice and
performance-based assessments have the potential to be effective or ineffective.
Carefully constructed multiple-choice questions can tap complex cognitive processes, not just lower level skills, as traditionally believed. And performance assessments, though generally praised for capturing higher level skills, may inadvertently focus on lower level skills (Baxter & Glaser, 1998; Hamilton, Nussbaum, and Snow, 1997; Linn, Baker, & Dunbar, 1991).

* Regard task difficulty in terms of underlying knowledge of cognitive processes required, rather than statistical information such as how many respondents answered correctly.

At the classroom assessment level, cognitive science findings encourage teachers to:

* Teach learners how and when to apply various approaches and procedures.

* Teach meta-cognitive skills within content areas so learners become capable of directing their thinking and reflecting on their progress.

* Observe students as they solve problems.

* Have students think aloud as they work or describe the reasoning that leads them to a particular solution.

* Analyze student errors on assignments or tests to determine which students got a question or problem wrong and why it appeared difficult for them. Knowing the source of difficulty can lead to more targeted, effective remediation.

Teachers should also be aware that acquiring important knowledge and skills at an
in-depth level takes a significant amount of time, practice, and feedback.


ERIC Identifier: ED481716
Publication Date: 2003
Author: Boston, Carol
Source: ERIC Clearinghouse on Assessment and Evaluation

REFERENCES

Anderson, J. (1982). Acquisition of cognitive skill. Psychological Review, 89, 369-406.

Anderson, J., Greeno, J., Reder, L., and Simon, H.A. (2000). Perspectives on learning, thinking, and activity. Educational Researcher, 229 (4): 11-13.

Baxter, G. and Glaser, R. (1998). Investigating the cognitive complexity of science
assessments. Educational Measurement: Issues and Practices, 17 (3): 37-45.

Bereiter, C. & Scardamalia, M.(1986). Educational relevance in the study of expertise. Interchange, 17 (2): 10-19.

Brown, J.S. and Burton, R.R. (1978). Diagnostic models for procedural bugs in basic
mathematical skills. Cognitive Science, 2, 155-192.

Case, R. (1996). Introduction - Reconceptualizing the development of children's
conceptual structures and their development in middle childhood. Monographs of the Society for Research in Child Development, 61 (1-2): 1-26.

Feldman, A., & Minstrell, J. (2000). Action research as a research methodology for the study of the teaching and learning of science. In E. Kelly & R. Lesh (Eds.), Handbook of Research Design in Mathematics and Science Education. Mahwah, NJ: Erlbaum.

Gabel, D., ed. (1994). Handbook of Research on Science Teaching and Learning. New York: Macmillan.

Glaser, R. and Baxter, G. (1999). Assessing active knowledge. Paper presented at the 1999 CRESST Conference, Benchmarks for Accountability: Are We There Yet? UCLA, Los Angeles.

Glaser, R. and Chi, M. (1988). Overview in M. Chi, R. Glaser, & M. Farr (Eds.), The Nature of Expertise (pp. xv-xxvii). Hillsdale, NJ: Erlbaum.

Greeno, J.G., Collins, A.M., & Resnick, L.B. (1997). Cognition and learning. In D.

Berliner & R. Calfee (Eds.), Handbook of Educational Psychology (pp. 15-47). New York: Simon & Schuster Macmillan.

Griffin, S., and Case, R. (1997). Re-thinking the primary school math curriculum: An approach based on cognitive science. Issues in Education, 3, 1-65.

Hamilton, L., Nussbaum, E., & Snow, R. (1997). Interview procedures for validating science assessments. Applied Measurement in Education, 10, 181-200.

Hatano, G. (1990). The nature of everyday science: A brief introduction. British Journal of Developmental Psychology, 8, 245-250.

Linn, R., Baker, E., & Dunbar, S. (1991). Complex, performance-based assessment: Expectations and validation criteria. Educational Researcher, 20 (8):15-21.

National Research Council (2001). Knowing What Students Know: The Science and Design of Educational Assessment. Washington, DC: National Academy Press.

Niemi, D. (1996). Assessing conceptual understanding in mathematics:
Representations, problem solutions, justifications, and explanations. Journal of
Educational Research, 89, 351-363.

Palinscar, A. and Brown, A. (1984). Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities. Cognition and Instruction, 1, 117-175.

Rumelhart, D. A. (1980). Schemata: The building blocks of cognition. In R. Spiro, B. Bruce, & W. Brewer (Eds). Theoretical Issues in Reading Comprehension (pp. 33-58). Hillsdale, NJ: Erlbaum.

Wineburg, S. S. (1996). The psychology of learning and teaching history. In D. Berliner & R. Calfee (Eds.), Handbook of Educational Psychology (pp. 423-437). New York: Simon & Schuster Macmillan.