Madhabi Chatterji

User-Centered Assessment Design

An Integrated Methodology for Diverse Populations

• Gebundenes Buch

Produktbeschreibung

"True to its title, this book is in equal parts about assessment design and assessment use. It lies in the incorporation of "user-centered design" principles throughout, founded on a realist view of science and informed by a socioecological perspective. Fundamentally, that means that the book treats the theoretical and methodological considerations pertinent to designing instruments with the help of an integrative Process Model for Assessment Design, Validation, and Use. This model is guided by what is likely to be most important to the primary users of the assessment tools. In the book, I introduce a two-part concept of "utility" as a quality criterion for assessment instruments; utility comprises the dual notions of "usefulness" and "usability" of the tools for its primary users. The book is organized in three main sections. Section I (Chapters 1-3) deals with the foundations; Section II (Chapters 4-7) focuses on design; Section III (Chapters 9-12) is dedicated to validation and use. Chapter 8 is a supplementary chapter on basic statistics"--

Produktdetails

• Verlag: Guilford Publications
• Seitenzahl: 504
• Erscheinungstermin: 12. Dezember 2024
• Englisch
• ISBN-13: 9781462555482
• ISBN-10: 1462555489
• Artikelnr.: 70724431

Autorenporträt

Madhabi Chatterji, PhD, is Professor Emerita of Measurement, Evaluation, and Education at Teachers College, Columbia University, where she founded and directs the Assessment and Evaluation Research Initiative (AERI). AERI is dedicated to promoting meaningful use of assessment and evaluation information to improve equity and the quality of practices and policies in education, psychology, and the health professions. An award-winning, internationally recognized methodologist and educationist, Dr. Chatterji has taught and mentored numerous doctoral students and postdoctoral researchers over her 30-plus-year career. She is author or editor of more than 100 publications and is a Fellow of the National Education Policy Center. A public intellectual, Dr. Chatterji has spoken out frequently on the limitations of large-scale tests and the adverse social consequences of misused high-stakes educational assessments. Her longstanding scholarly interests lie in instrument design, validation, validity, and test use issues; improving program and policy evaluation designs to support evidence-based practices; and closing learning gaps with proximal diagnostic assessments.

Inhaltsangabe

I. Foundations
1. Foundational Concepts in Assessment Design
1.1 Chapter Overview
1.2 Assessments: Old and Emerging Traditions, a Starting Definition, and
Some Distinctions
1.3 Viewpoints on Assessment, Measurement, Testing, and Evaluation
1.4 Role of Assessment in Scientific, Professional, and Practical Endeavors
1.5 Evaluating the Quality of Assessments and Construct Measures: Validity,
Reliability, and Utility
1.6 Integrating Assessment Design, Validation, and Use: A User-Centered
Process
1.7 Summary
2. Why Assess?: Measure-Based Inferences, Uses, Users, and Consequences
2.1 Chapter Overview
2.2 Back to the Future: Early Drivers, Milestones, and Consequences of
Assessment Use
2.3 Modern Drivers and Consequences of Assessment Uses in Education
2.4 Modern Drivers and Consequences of Assessment Uses in Psychology,
Health, Business, and Other Fields
2.5 Applying User-Centered Principles to Improve Practices
2.6 Summary
3. Whom to Assess? and How?: Specifying the Population and the Assessment
Operations
3.1 Chapter Overview
3.2 Why Population Characteristics and the Socioecological Contexts of
Assessments Matter
3.3 What Is Measurement Bias?: Case Studies and Hypothetical Illustrations
3.4 Selecting Assessment Operations for Diverse Populations and
Multidisciplinary Constructs
3.5 Steps and Actions: Specifying Whom to Assess? and How? with the Process
Model
3.6 Summary
II. Assessment Design
4. What to Assess?: Specifying the Domains for Constructs
4.1 Chapter Overview
4.2 Domain Sampling and Domain Specification: Functional Theory and Applied
Illustrations
4.3 Construct Types, Domain Conceptualizations, and Structures
4.4 Domain Specification as a Part of the Process Model: Steps, Techniques,
Guidelines, and Conventions
4.5 Content-Validating Specified Domains
4.6. Summary
5. Designing Assessments with Structured and Constructed-Response Items
5.1 Chapter Overview
5.2 Why the Mechanics of Item Construction Matter
5.3 Cognitive Constructs Measured Best with Structured- or
Constructed-Response Items
5.4 Writing Structured-Response Items: Principles, Guidelines, and Applied
Examples
5.5 Guidelines for Designing Constructed-Response and Essay Tasks
5.6. Instrument Assembly
5.7 An Application with the Process Model: A Case Study of Cognitively
Based Item and Assessment Design to Foster Learning in Long Division
5.8 Summary
6. Designing Behavior-Based, Product-Based, and Portfolio-Based Assessments
6.1 Chapter Overview
6.2 Behavior-, Product-, and Portfolio-Based Assessments: Definitions,
Examples, and Origins
6.3 Advantages of the Performance Assessment Format
6.4 Disadvantages of Performance Assessments: Human Vulnerabilities,
Errors, and Biases
6.5 Three Case Studies: Applying the Process Model to Design and Validate
Performance Assessments
6.6 Summary
7. Designing Survey-Based and Interview-Based Assessment Tools
7.1 Chapter Overview
7.2 Self-Report Instruments: Their Defining Properties and Common
Applications
7.3 Historical Origins of Questionnaires and Attitude Surveys
7.4 Measurement Issues with the Self-Report Modality
7.5 General Design Guidelines for Self-Report Instruments
7.6 Ten More Guidelines for Writing Closed-Ended Survey Items
7.7 A Case Study: Applying the Process Model to Design Two Complementary
Self-Report Tools
7.8 Summary
III. Validation and Use of Assessments
8. Analyzing Data from Assessments: A Statistics Refresher
8.1 Chapter Overview
8.2 Preparing for Data Analysis
8.3 Organizing the Data
8.4 Measures of Central Tendency
8.5 Measures of Variability
8.6 Graphical Displays of Data
8.7 The Standard Normal Distribution and Its Applications
8.8 Correlation Coefficients and Their Applications
8.9 Related Statistical Techniques
8.10 Summary
9. Improving the Inferential Utility of Assessment Results: Methods and
Limitations
9.1 Chapter Overview
9.2 Frames of Reference and Derived Scores
9.3 Using Norms as the Frame of Reference
9.4 Using Criterion Scores or Standards as the Frame of Reference
9.5 Using Self as the Frame of Reference
9.6 Composite Scores
9.7 Grouped Scores, Equated Scales, and Linked Tests
9.8 Summary
10. A Unified Approach to Construct Validity and Validation: Theory to
Evidence
10.1 Chapter Overview
10.2 Construct Validity: An Evolving Concept
10.3 Theoretical Foundations of the Unitarian View of Validation
10.4 Main Clusters and Types of Validity Evidence
10.5 Random Errors of Measurement and Types of Reliability Evidence
10.6 Utility of Measures, Assessments, and Assessment Systems
10.7 Unified Validation Plans
10.8 Chapter Summary
11. Empirical Methods of Validation
11.1 Chapter Overview
11.2 Planning Empirical Validation Studies
11.3 Evaluating Item Performance
11.4 Examining Fairness and Measurement Bias
11.5 Gathering Evidence of Content-Based Validity
11.6 Validating Response Processes: The Cognitive Interview
11.7 Gathering Correlational Evidence of Validity
11.8 Empirical Estimation of Reliability
11.9 Methods to Examine Utility
11.10 Evaluating the Evidence: The PSQI Case Revisited
11.11 Summary
12. User-Centered Assessment Design: Revisiting the Principles,
Comparisons, and Conclusions
12.1 Chapter Overview
12.2 Applying the Principles Undergirding the Process Model: A Summary by
Section
12.3. A User-Centered Design Process: Comparing the Old with the New
12.4 Extended Applications of the Process Model
12.5 The Process Model Compared to Existing Models of Assessment Design
12.6 Connecting the Process Model with the 2014 Standards
12.7 Summary
Glossary
References
Author Index
Subject Index
About the Author