by Boston, Carol
Cognitive science is devoted to the study of how people think and learn and how, when, and whether they use what they know to solve problems (Greeno, Collins, & Resnick, 1997; National Research Council, 2001). The cognitive perspective in education encompasses how learners develop and structure their knowledge in specific subject areas and how assessment tasks might be designed to enable students to demonstrate the knowledge and cognitive processes necessary to be judged proficient in these subject areas. This Digest provides educators with an overview of some important facets of cognitive science research and suggests implications for classroom assessment.
HOW DO EXPERTS AND NOVICES DIFFER IN THEIR APPROACH TO PROBLEMS?
Education researchers study the thinking of experts in various subject areas to gain an understanding of what concepts and procedures are most important to teach and how they are interrelated. The concept is that educators can and should be moving students along a continuum toward real-world subject mastery based on a deep understanding of how subject knowledge is organized (Bereiter & Scardamalia, 1986).
When faced with a problem, learners tend to search their memories for a schema, or learned technique for organizing and interpreting information in a certain subject, in
order to solve it (Rumelhart, 1980). Over time, individuals build mental models to guide their problem solving efficiently so they do not depend on trial-and-error approaches and can instead create analogies and make inferences to support new learning (Glaser & Baxter, 1999).
When compared with novice learners, experts in a subject are notable for how
well-organized their knowledge is, which in turn enables them to see patterns quickly, recall information, and study novel problems in light of concepts and principles they know already (Glaser & Chi, 1988). In other words, their schemas are well-connected and they are able to retrieve chunks of information relevant to a task at hand. Experts also have strong problem-solving skills. They know what they know and what they don'tknow, and plan and monitor the implementation of various mental strategies (Hatano, 1990).
COGNITIVE SCIENCE IN THE CLASSROOM
Ideally, developmental models of learning could be created that note the typical
progression and milestones as a learner advances from novice to competent to expert and describe the types of experiences that lead to change. For example, students generally have naive or intuitive understandings of the sciences, based in part on misconceptions that are corrected as they are exposed to new learning (e.g., Gabel, 1994, Feldman & Minstrell, 2000). And while there are individual differences among learners, when large samples are studied, patterns tend to emerge, particularly related to erroneous beliefs and incorrect procedures. For example, there appear to be a certain limited number of "subtraction bugs" that account for almost all of the ways young children make mistakes when learning to subtract two- or three-digit numbers, and these are constant even across languages (Brown and Burton, 1978).
Allowing for variations among learners, it is possible to discover the most common
pathways toward acquiring knowledge and use this information diagnostically. For
example, Case, Griffin, and colleagues have developed an assessment tool based on their empirical research regarding how children from ages 4 to 10 change in their
conception of numbers through growth and practice. While 4-year-olds can count
groups of objects, they have to guess if they face a theoretical question such as, "Which is more--four or five?" Between 4 and 6, most children develop a "mental number line" that helps them envision the answer to such a question, even when actual objects aren't present. Between 6 and 8, children gradually come to envision other number lines for counting by 2s, 5s, 10s, and 100s. By 10, many children have a better understanding of the base-10 number system, which enables them to reach a more sophisticated understanding of concepts such as regrouping and estimation (Case, 1996; Griffin and Case, 1997). Teachers can use assessments based on this research to determine their next steps in arithmetic instruction.
More research has been done about domain structure in some disciplines than in
others. Mathematics, physics, beginning reading, and U.S. history are among the areas that have been studied (see, for example, Niemi, 1996, and Wineburg, 1996).
Subject-area standards such as the National Council of Teachers of Mathematics
Standards generally reflect current thinking on cognitive processes and are a good
place for teachers to begin their explorations of this topic. The National Research
Council's How People Learn: Brain, Mind, Experience, and School
(http://stills.nap.edu/html/howpeople1/) provides another helpful introduction.
HOW DO LEARNERS STORE AND ACCESS KNOWLEDGE?
Memory may be divided into two types: short-term, or working memory, which
determines how much mental processing can go on at any one time, and long-term
memory, where people organize their content knowledge. Short-term memory, or working memory, is connected with fluid intelligence, or the ability to solve new and
unusual problems, while long-term memory is connected to crystallized intelligence, or the bringing of past experience to bear on current problems (Anderson, Greeno, Reder, and Simon, 2000). When students are learning a new skill, they must rely heavily on their working memory to represent the task and may need to talk themselves through a task. As the skill moves into long-term memory, it becomes fluent, and eventually, automatic (Anderson, 1982).
To support the learning process, students can be taught meta-cognitive skills, or
techniques to reflect on and assess their own thinking. To improve reading
comprehension, for example, young children can be taught to monitor their
understanding of passages by asking questions, summarizing, clarifying any
uncertainties, and predicting next events (Palincsar & Brown, 1984).
HOW CAN ASSESSMENT DESIGNERS USE FINDINGS FROM COGNITIVE SCIENCE?
The design of any assessment should begin with a statement of purpose for the
assessment and a definition of the particular subject area or content domain. How do people demonstrate knowledge and become competent in this domain? What important aspects of learning do we want to draw inferences from when measuring student achievement in a given subject area? What situations and tasks can we observe to make the appropriate inferences?
Cognitive science calls for test developers to:
* Work from a deep knowledge of the central concepts and principles of a given subject area, and the most important related information.
* Identify or develop those tasks that allow students to demonstrate their
understanding and skills in these areas, as opposed to rote memorization.
* Make sure tasks or questions are sufficiently complex to get at how students have organized their knowledge and how and when they use it.
* Emphasize the contents of long-term memory rather than short-term, or working, memory by not burdening test-takers withrequirements to track a large number of response options or major quantities of extraneous information while answering a question.
* Emphasize relevant constructs--for example, a mathematics assessment should not over-emphasize reading and writing, unless communicating about mathematics is the skill to be measured.
* Not limit choice of item format. Both multiple-choice and
performance-based assessments have the potential to be effective or ineffective.
Carefully constructed multiple-choice questions can tap complex cognitive processes, not just lower level skills, as traditionally believed. And performance assessments, though generally praised for capturing higher level skills, may inadvertently focus on lower level skills (Baxter & Glaser, 1998; Hamilton, Nussbaum, and Snow, 1997; Linn, Baker, & Dunbar, 1991).
* Regard task difficulty in terms of underlying knowledge of cognitive processes required, rather than statistical information such as how many respondents answered correctly.
At the classroom assessment level, cognitive science findings encourage teachers to:
* Teach learners how and when to apply various approaches and procedures.
* Teach meta-cognitive skills within content areas so learners become capable of directing their thinking and reflecting on their progress.
* Observe students as they solve problems.
* Have students think aloud as they work or describe the reasoning that leads them to a particular solution.
* Analyze student errors on assignments or tests to determine which students got a question or problem wrong and why it appeared difficult for them. Knowing the source of difficulty can lead to more targeted, effective remediation.
Teachers should also be aware that acquiring important knowledge and skills at an
in-depth level takes a significant amount of time, practice, and feedback.
ERIC Identifier: ED481716
Publication Date: 2003
Author: Boston, Carol
Source: ERIC Clearinghouse on Assessment and Evaluation
REFERENCES
Anderson, J. (1982). Acquisition of cognitive skill. Psychological Review, 89, 369-406.
Anderson, J., Greeno, J., Reder, L., and Simon, H.A. (2000). Perspectives on learning, thinking, and activity. Educational Researcher, 229 (4): 11-13.
Baxter, G. and Glaser, R. (1998). Investigating the cognitive complexity of science
assessments. Educational Measurement: Issues and Practices, 17 (3): 37-45.
Bereiter, C. & Scardamalia, M.(1986). Educational relevance in the study of expertise. Interchange, 17 (2): 10-19.
Brown, J.S. and Burton, R.R. (1978). Diagnostic models for procedural bugs in basic
mathematical skills. Cognitive Science, 2, 155-192.
Case, R. (1996). Introduction - Reconceptualizing the development of children's
conceptual structures and their development in middle childhood. Monographs of the Society for Research in Child Development, 61 (1-2): 1-26.
Feldman, A., & Minstrell, J. (2000). Action research as a research methodology for the study of the teaching and learning of science. In E. Kelly & R. Lesh (Eds.), Handbook of Research Design in Mathematics and Science Education. Mahwah, NJ: Erlbaum.
Gabel, D., ed. (1994). Handbook of Research on Science Teaching and Learning. New York: Macmillan.
Glaser, R. and Baxter, G. (1999). Assessing active knowledge. Paper presented at the 1999 CRESST Conference, Benchmarks for Accountability: Are We There Yet? UCLA, Los Angeles.
Glaser, R. and Chi, M. (1988). Overview in M. Chi, R. Glaser, & M. Farr (Eds.), The Nature of Expertise (pp. xv-xxvii). Hillsdale, NJ: Erlbaum.
Greeno, J.G., Collins, A.M., & Resnick, L.B. (1997). Cognition and learning. In D.
Berliner & R. Calfee (Eds.), Handbook of Educational Psychology (pp. 15-47). New York: Simon & Schuster Macmillan.
Griffin, S., and Case, R. (1997). Re-thinking the primary school math curriculum: An approach based on cognitive science. Issues in Education, 3, 1-65.
Hamilton, L., Nussbaum, E., & Snow, R. (1997). Interview procedures for validating science assessments. Applied Measurement in Education, 10, 181-200.
Hatano, G. (1990). The nature of everyday science: A brief introduction. British Journal of Developmental Psychology, 8, 245-250.
Linn, R., Baker, E., & Dunbar, S. (1991). Complex, performance-based assessment: Expectations and validation criteria. Educational Researcher, 20 (8):15-21.
National Research Council (2001). Knowing What Students Know: The Science and Design of Educational Assessment. Washington, DC: National Academy Press.
Niemi, D. (1996). Assessing conceptual understanding in mathematics:
Representations, problem solutions, justifications, and explanations. Journal of
Educational Research, 89, 351-363.
Palinscar, A. and Brown, A. (1984). Reciprocal teaching of comprehension-fostering and comprehension-monitoring activities. Cognition and Instruction, 1, 117-175.
Rumelhart, D. A. (1980). Schemata: The building blocks of cognition. In R. Spiro, B. Bruce, & W. Brewer (Eds). Theoretical Issues in Reading Comprehension (pp. 33-58). Hillsdale, NJ: Erlbaum.
Wineburg, S. S. (1996). The psychology of learning and teaching history. In D. Berliner & R. Calfee (Eds.), Handbook of Educational Psychology (pp. 423-437). New York: Simon & Schuster Macmillan.
Tampilkan postingan dengan label Uncategories. Tampilkan semua postingan
Tampilkan postingan dengan label Uncategories. Tampilkan semua postingan
Meta-Analysis in Educational Research.
by Robert L Bangert-Drowns,
SUNY
Lawrence M Rudner
ERIC Clearinghouse on Assessment and Evaluation
"I had hoped to find research to support or to conclusively oppose my belief that quality integrated education is the most promising approach. For every study that contains a recommendation, there is another, equally well documented study, challenging the conclusions of the first...No one seems to agree with anyone else's approach. But more distressing: no one seems to know what works."
Senator Fritz Mondale's quote illustrates a common plight. Educational research often produces contradictory results. Differences among studies in treatments, settings, measurement instruments, and research methods make research findings difficult to compare. Even frequent replications can prove inconclusive. Literature on a topic may be so extensive as to obscure trends with an overwhelming amount of information.
Meta-analysis is a collection of systematic techniques for resolving apparent contradictions in research findings. Meta-analysts translate results from different studies to a common metric and statistically explore relations between study characteristics and findings.
This article first describes meta-analysis as a research method. The need and general approach are discussed. We then identify some common approaches toward conducting meta-analysis in education and outline their advantages and disadvantages.
META-ANALYSIS AS A RESEARCH METHOD
Gene Glass first used the term "meta-analysis" in 1976 to refer to a philosophy, not a statistical technique. Glass argued that literature review should be as systematic as primary research and should interpret the results of individual studies in the context of distributions of findings, partially determined by study characteristics and partially random. Since that time, meta-analysis has become a widely accepted research tool, encompassing a family of procedures used in a variety of disciplines. A recent search of the ERIC database identified over 800 articles written after 1980 that use or discuss meta-analysis.
Meta-analysis responds to several problems in educational research. First, important issues are studied by numerous investigators. The amount of information on a given topic therefore is often overwhelming and not amenable to summary. Even when there are relatively few studies on a given topic, it is difficult to determine if outcome differences are attributable to chance, to methodological inadequacies, or to systematic differences in study characteristics. Informal methods of narrative review permit biases to remain easily undetected. Reviewers' biases can influence decisions about study inclusion, relative weights given to different findings, and analysis of relations between study features and outcomes. These biases can have clandestine effects when reviewers do not systematically seek to reduce them or provide sufficient information for readers to evaluate their extent.
Meta-analysis typically follows the same steps as primary research. The meta-analyst first defines the review's purpose. Organizing frameworks can be practical or theoretical questions of varying scope, but they must be clear enough to guide study selection and data collection. Second, sample selection consists of applying specified procedures for locating studies that meet specified criteria for inclusion. Typically, meta-analyses are comprehensive reviews of the full population of relevant studies. Third, data are collected from studies in two ways. Study features are coded according to the objectives of the review and as checks on threats to validity. Study outcomes are transformed to a common metric so that they can be compared. A typical metric in educational research is the effect size, the standardized difference between treatment and control group means. Finally, statistical procedures are used to investigate relations among study characteristics and findings.
Criticisms of meta-analysis tend to fall into two categories. Some complain that meta-analysis obscures important qualitative information by "averaging" simple numerical representations across studies. Other critics argue that research is best reviewed by a reflective expert who can sift kernels of insight from the confusing argumentation of a field.
META-ANALYTIC APPROACHES
VOTE-COUNTING -- Some reviews categorize findings as significantly positive (favoring the treatment group), significantly negative, or nonsignificant. The category with the most entries is considered the best representation of research in this area. This as an inexact approach to integrating research. Vote-counting confuses treatment effect and sample size because statistical significance is a function of both. Given the modest power of typical educational research to detect true effects as statistically significant, conclusions from vote-counting can be very misleading.
CLASSIC OR GLASSIAN META-ANALYSIS -- Glass' early meta-analyses set the pattern for conventional meta-analysis: define questions to be examined, collect studies, code study features and outcomes, and analyze relations between study features and outcomes. These early meta-analyses, and later ones following this tradition, share three distinguishing features. First, "classic" meta-analysis applies liberal inclusion criteria. Glass argued that one should not disregard studies on the basis of study quality a priori; a meta-analysis itself can determine if study quality is related to variance in reported treatment effect. Second, the unit of analysis is the study finding. A single study can report many comparisons between groups and subgroups on different criteria. Effect sizes are calculated for each comparison. Third, meta-analysts using this approach may average effects from different dependent variables, even when these measure different constructs.
Glassian meta-analysis has proven quite robust when submitted to critical re-analysis. Its use of conventional statistical tests render the method and its results accessible to most educational researchers. However, using study findings as the units of analysis produces nonindependent data and gives greater weight to studies with many comparisons. Averaging across constructs and including studies with obvious methodological flaws can confuse the reliability of findings.
STUDY EFFECT META-ANALYSIS -- Study effect meta-analysis alters the Glassian form in two ways. First, inclusion rules are more selective. Studies with serious methodological flaws are excluded. Second, the study is the unit of analysis. One effect size is computed for each study. This preserves the independence of the data and gives equal weight to all included studies. Unfortunately, it also reduces the number of data points analyzed in the review. And, of course, a reviewer's biases may operate in decisions to exclude studies.
TESTS OF HOMOGENEITY -- Some reviewers argue that conventional statistical tests are inappropriate for meta-analysis. Homogeneity tests were developed to determine the likelihood that variance among effect sizes is due only to sampling error. If the homogeneity statistic is significant for a group of studies, a procedure analogous to analysis of variance can be used. Studies are repeatedly divided into subgroups according to study features until within-group variation is nonsignificant.
Numerous factors can cause variation in effect sizes: measurement unreliability, range restrictions, reporting errors, within-study statistical adjustments, unreported factors, etc. Homogeneity tests are very likely to indicate heterogeneity among effect sizes even when the variation is of no practical or theoretical importance. Successively dividing subgroups according to these tests can capitalize on chance and cause the incorrect identification of moderators. Kulik and Kulik defend conventional analysis of variance for meta-analysis and suggest that homogeneity tests may ignore an important nesting factor.
PSYCHOMETRIC META-ANALYSIS -- Hunter and Schmidt's approach to meta-analysis combines some of the best features of other approaches. All studies related to a given topic are gathered, regardless of quality. The distribution of effect sizes is corrected for sampling error, measurement error, range restriction, and other systematic artifacts. If the remaining variance is still large, effect sizes are grouped into subsets according to preselected study features, and each subset is meta-analyzed separately. Ideally, the meta-analysis should estimate true treatment effects under conditions typical of those represented in the studies and predict treatment effects under conditions determined by the reviewer. Unfortunately, this technique requires substantial information from individual studies for accurate correction of effect sizes. This information is not always available in research reports.
author: Bangert-Drowns, Robert L. & Rudner, Lawrence M. (1991). Meta-analysis in educational research. Practical Assessment, Research & Evaluation, 2(8). Retrieved October 4, 2011 from http://PAREonline.net/getvn.asp?v=2&n=8 . This paper has been viewed 44,141 times since 11/13/1999.
SUGGESTED READING
Bangert-Drowns, R.L. (1986). Review of developments in meta-analytic methods. Psychological Bulletin, 99, 388-399.
Glass, G.V, McGaw, B., & M.L. Smith (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.
Hunter, J.E., & F.L. Schmidt (1990). Methods of meta-analysis. Newbury Park, CA: Sage.
Kulik, J.A., & C.-L.C. Kulik (1989). Meta-analysis in education. International Journal of Educational Research, 13, 221-340.
SUNY
Lawrence M Rudner
ERIC Clearinghouse on Assessment and Evaluation
"I had hoped to find research to support or to conclusively oppose my belief that quality integrated education is the most promising approach. For every study that contains a recommendation, there is another, equally well documented study, challenging the conclusions of the first...No one seems to agree with anyone else's approach. But more distressing: no one seems to know what works."
Senator Fritz Mondale's quote illustrates a common plight. Educational research often produces contradictory results. Differences among studies in treatments, settings, measurement instruments, and research methods make research findings difficult to compare. Even frequent replications can prove inconclusive. Literature on a topic may be so extensive as to obscure trends with an overwhelming amount of information.
Meta-analysis is a collection of systematic techniques for resolving apparent contradictions in research findings. Meta-analysts translate results from different studies to a common metric and statistically explore relations between study characteristics and findings.
This article first describes meta-analysis as a research method. The need and general approach are discussed. We then identify some common approaches toward conducting meta-analysis in education and outline their advantages and disadvantages.
META-ANALYSIS AS A RESEARCH METHOD
Gene Glass first used the term "meta-analysis" in 1976 to refer to a philosophy, not a statistical technique. Glass argued that literature review should be as systematic as primary research and should interpret the results of individual studies in the context of distributions of findings, partially determined by study characteristics and partially random. Since that time, meta-analysis has become a widely accepted research tool, encompassing a family of procedures used in a variety of disciplines. A recent search of the ERIC database identified over 800 articles written after 1980 that use or discuss meta-analysis.
Meta-analysis responds to several problems in educational research. First, important issues are studied by numerous investigators. The amount of information on a given topic therefore is often overwhelming and not amenable to summary. Even when there are relatively few studies on a given topic, it is difficult to determine if outcome differences are attributable to chance, to methodological inadequacies, or to systematic differences in study characteristics. Informal methods of narrative review permit biases to remain easily undetected. Reviewers' biases can influence decisions about study inclusion, relative weights given to different findings, and analysis of relations between study features and outcomes. These biases can have clandestine effects when reviewers do not systematically seek to reduce them or provide sufficient information for readers to evaluate their extent.
Meta-analysis typically follows the same steps as primary research. The meta-analyst first defines the review's purpose. Organizing frameworks can be practical or theoretical questions of varying scope, but they must be clear enough to guide study selection and data collection. Second, sample selection consists of applying specified procedures for locating studies that meet specified criteria for inclusion. Typically, meta-analyses are comprehensive reviews of the full population of relevant studies. Third, data are collected from studies in two ways. Study features are coded according to the objectives of the review and as checks on threats to validity. Study outcomes are transformed to a common metric so that they can be compared. A typical metric in educational research is the effect size, the standardized difference between treatment and control group means. Finally, statistical procedures are used to investigate relations among study characteristics and findings.
Criticisms of meta-analysis tend to fall into two categories. Some complain that meta-analysis obscures important qualitative information by "averaging" simple numerical representations across studies. Other critics argue that research is best reviewed by a reflective expert who can sift kernels of insight from the confusing argumentation of a field.
META-ANALYTIC APPROACHES
VOTE-COUNTING -- Some reviews categorize findings as significantly positive (favoring the treatment group), significantly negative, or nonsignificant. The category with the most entries is considered the best representation of research in this area. This as an inexact approach to integrating research. Vote-counting confuses treatment effect and sample size because statistical significance is a function of both. Given the modest power of typical educational research to detect true effects as statistically significant, conclusions from vote-counting can be very misleading.
CLASSIC OR GLASSIAN META-ANALYSIS -- Glass' early meta-analyses set the pattern for conventional meta-analysis: define questions to be examined, collect studies, code study features and outcomes, and analyze relations between study features and outcomes. These early meta-analyses, and later ones following this tradition, share three distinguishing features. First, "classic" meta-analysis applies liberal inclusion criteria. Glass argued that one should not disregard studies on the basis of study quality a priori; a meta-analysis itself can determine if study quality is related to variance in reported treatment effect. Second, the unit of analysis is the study finding. A single study can report many comparisons between groups and subgroups on different criteria. Effect sizes are calculated for each comparison. Third, meta-analysts using this approach may average effects from different dependent variables, even when these measure different constructs.
Glassian meta-analysis has proven quite robust when submitted to critical re-analysis. Its use of conventional statistical tests render the method and its results accessible to most educational researchers. However, using study findings as the units of analysis produces nonindependent data and gives greater weight to studies with many comparisons. Averaging across constructs and including studies with obvious methodological flaws can confuse the reliability of findings.
STUDY EFFECT META-ANALYSIS -- Study effect meta-analysis alters the Glassian form in two ways. First, inclusion rules are more selective. Studies with serious methodological flaws are excluded. Second, the study is the unit of analysis. One effect size is computed for each study. This preserves the independence of the data and gives equal weight to all included studies. Unfortunately, it also reduces the number of data points analyzed in the review. And, of course, a reviewer's biases may operate in decisions to exclude studies.
TESTS OF HOMOGENEITY -- Some reviewers argue that conventional statistical tests are inappropriate for meta-analysis. Homogeneity tests were developed to determine the likelihood that variance among effect sizes is due only to sampling error. If the homogeneity statistic is significant for a group of studies, a procedure analogous to analysis of variance can be used. Studies are repeatedly divided into subgroups according to study features until within-group variation is nonsignificant.
Numerous factors can cause variation in effect sizes: measurement unreliability, range restrictions, reporting errors, within-study statistical adjustments, unreported factors, etc. Homogeneity tests are very likely to indicate heterogeneity among effect sizes even when the variation is of no practical or theoretical importance. Successively dividing subgroups according to these tests can capitalize on chance and cause the incorrect identification of moderators. Kulik and Kulik defend conventional analysis of variance for meta-analysis and suggest that homogeneity tests may ignore an important nesting factor.
PSYCHOMETRIC META-ANALYSIS -- Hunter and Schmidt's approach to meta-analysis combines some of the best features of other approaches. All studies related to a given topic are gathered, regardless of quality. The distribution of effect sizes is corrected for sampling error, measurement error, range restriction, and other systematic artifacts. If the remaining variance is still large, effect sizes are grouped into subsets according to preselected study features, and each subset is meta-analyzed separately. Ideally, the meta-analysis should estimate true treatment effects under conditions typical of those represented in the studies and predict treatment effects under conditions determined by the reviewer. Unfortunately, this technique requires substantial information from individual studies for accurate correction of effect sizes. This information is not always available in research reports.
author: Bangert-Drowns, Robert L. & Rudner, Lawrence M. (1991). Meta-analysis in educational research. Practical Assessment, Research & Evaluation, 2(8). Retrieved October 4, 2011 from http://PAREonline.net/getvn.asp?v=2&n=8 . This paper has been viewed 44,141 times since 11/13/1999.
SUGGESTED READING
Bangert-Drowns, R.L. (1986). Review of developments in meta-analytic methods. Psychological Bulletin, 99, 388-399.
Glass, G.V, McGaw, B., & M.L. Smith (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.
Hunter, J.E., & F.L. Schmidt (1990). Methods of meta-analysis. Newbury Park, CA: Sage.
Kulik, J.A., & C.-L.C. Kulik (1989). Meta-analysis in education. International Journal of Educational Research, 13, 221-340.
Langganan:
Postingan (Atom)
