Design, Evaluation, and Analysis of Questionnaires for Survey Research
John Wiley & Sons Inc (Verlag)
978-1-118-63461-5 (ISBN)
Praise for the First Edition
"...this book is quite inspiring, giving many practical ideas for survey research, especially for designing better questionnaires."
—International Statistical Review
Reflecting modern developments in the field of survey research, the Second Edition of Design, Evaluation, and Analysis of Questionnaires for Survey Research continues to provide cutting-edge analysis of the important decisions researchers make throughout the survey design process.The new edition covers the essential methodologies and statistical tools utilized to create reliable and accurate survey questionnaires, which unveils the relationship between individual question characteristics and overall question quality. Since the First Edition, the computer program Survey Quality Prediction (SQP) has been updated to include new predictions of the quality of survey questions on the basis of analyses of Multi-Trait Multi-Method experiments. The improved program contains over 60,000 questions, with translations in most European languages. Featuring an expanded explanation of the usage and limitations of SQP 2.0, the Second Edition also includes:
New practice problems to provide readers with real-world experience in survey research and questionnaire design
A comprehensive outline of the steps for creating and testing survey questionnaires
Contemporary examples that demonstrate the many pitfalls of questionnaire design and ways to avoid similar decisions
Design, Evaluation, and Analysis of Questionnaires for Survey Research, Second Edition is an excellent textbook for upper-undergraduate and graduate-level courses in methodology and research questionnaire planning, as well as an ideal resource for social scientists or survey researchers needing to design, evaluate, and analyze questionnaires.
WILLEM E. SARIS, PHD, is Emeritus Professor in Methodology of the University of Amsterdam and the Universitat Pompeu Fabra, Barcelona. He is Laureate of the 2005 Descartes Prize for “Best Collaborative Research” as member of the Central Coordinating Team of the European Social Survey (ESS) and Recipient of the World Association of Public Opinion Research’s “Helen Dinerman Award” in 2009 for his lifelong contribution to the methodology of Opinion Research. Dr. Saris also received the “2013 Outstanding Service Prize” of the European Survey Research Association. IRMTRAUD N. GALLHOFER, PHD, is a linguist and was senior researcher on projects of the ESS, Research and Expertise Centre for Survey Methodology at the Universitat Pompeu Fabra, Barcelona. She is Laureate of the 2005 Descartes Prize for “Best Collaborative Research” as a member of the Central Coordinating Team of the ESS.
Preface to the Second Edition xiii
Preface xv
Acknowledgments xvii
Introduction 1
I.1 Designing a Survey 4
I.1.1 Choice of a Topic 4
I.1.2 Choice of the Most Important Variables 4
I.1.3 Choice of a Data Collection Method 5
I.1.4 Choice of Operationalization 6
I.1.5 Test of the Quality of the Questionnaire 8
I.1.6 Formulation of the Final Questionnaire 9
I.1.7 Choice of Population and Sample Design 9
I.1.8 Decide about the Fieldwork 10
I.1.9 What We Know about These Decisions 10
I.1.10 Summary 11
Exercises 12
Part I The Three-Step Procedure to Design Requests for Answers 13
1 Concepts-by-Postulation and Concepts-by-Intuition 15
1.1 Concepts-by-Intuition and Concepts-by-Postulation 15
1.2 Different Ways of Defining Concepts-by-Postulation through Concepts-by-Intuition 19
1.2.1 Job Satisfaction as a Concept-by-Intuition 19
1.2.2 Job Satisfaction as a Concept-by-Postulation 20
1.3 Summary 27
Exercises 28
2 From Social Science Concepts-by-Intuition to Assertions 30
2.1 Basic Concepts and Concepts-by-Intuition 31
2.2 Assertions and Requests for an Answer 32
2.3 The Basic Elements of Assertions 33
2.3.1 Indirect Objects as Extensions of Simple Assertions 36
2.3.2 Adverbials as Extensions of Simple Assertions 37
2.3.3 Modifiers as Extensions of Simple Assertions 37
2.3.4 Object Complements as Extensions of Simple Assertions 38
2.3.5 Some Notation Rules 38
2.4 Basic Concepts-by-Intuition 39
2.4.1 Subjective Variables 40
2.4.2 Objective Variables 47
2.4.3 In Summary 49
2.5 Alternative Formulations for the Same Concept 49
2.6 Extensions of Simple Sentences 51
2.6.1 Adding Indirect Objects 51
2.6.2 Adding Modifiers 52
2.6.3 Adding Adverbials 52
2.7 Use of Complex Sentences 53
2.7.1 Complex Sentences with No Shift in Concept 54
2.7.2 Complex Sentences with a Shift in Concept 54
2.7.3 Adding Conditions to Complex Sentences 56
2.8 Summary 56
Exercises 57
3 The Formulation of Requests for an Answer 60
3.1 From Concepts to Requests for an Answer 61
3.2 Different Types of Requests for an Answer 63
3.2.1 Direct Request 63
3.2.2 Indirect Request 66
3.3 The Meaning of Requests for an Answer with WH Request Words 69
3.3.1 “When,” “Where,” and “Why” Requests 70
3.3.2 “Who” Requests 70
3.3.3 “Which” Requests 70
3.3.4 “What” Requests 71
3.3.5 “How” Requests 72
3.4 Summary 74
Exercises 75
Part II Choices Involved in Questionnaire Design 77
4 Specific Survey Research Features of Requests for an Answer 79
4.1 Select Requests from Databases 79
4.2 Other Features Connected with the Research Goal 81
4.3 Some Problematic Requests 83
4.3.1 Double-Barreled Requests 83
4.3.2 Requests with Implicit Assumptions 84
4.4 Some Prerequests Change the Concept-by-Intuition 85
4.5 Batteries of Requests for Answers 86
4.5.1 The Use of Batteries of Stimuli 87
4.5.2 The Use of Batteries of Statements 88
4.6 Other Features of Survey Requests 92
4.6.1 The Formulation of Comparative or Absolute Requests for Answers 92
4.6.2 Conditional Clauses Specified in Requests for Answers 93
4.6.3 Balanced or Unbalanced Requests for Answers 93
4.7 Special Components within the Request 95
4.7.1 Requests for Answers with Stimulation for an Answer 95
4.7.2 Emphasizing the Subjective Opinion of the Respondent 95
4.8 Summary 96
Exercises 96
5 Response Alternatives 98
5.1 Open Requests for an Answer 99
5.2 Closed Categorical Requests 101
5.2.1 Nominal Categories 103
5.2.2 Ordinal Scales 104
5.2.3 Continuous Scales 108
5.3 How Many Categories are Optimal? 111
5.4 Summary 112
Exercises 114
6 The Structure of Open-Ended and Closed Survey Items 115
6.1 Description of the Components of Survey Items 115
6.2 Different Structures of Survey Items 118
6.2.1 Open-Ended Requests for an Answer 119
6.2.2 Closed Survey Items 120
6.2.3 The Frequency of Occurrence 124
6.2.4 The Complexity of Survey Items 125
6.3 What Form of Survey Items Should Be Recommended? 126
6.4 Summary 127
Exercises 128
7 Survey Items in Batteries 130
7.1 Batteries in Oral Interviews 131
7.2 Batteries in Mail Surveys 134
7.3 Batteries in CASI 138
7.4 Summary and Discussion 142
Exercises 144
8 Mode of Data Collection and Other Choices 146
8.1 The Choice of the Mode of Data Collection 147
8.1.1 Relevant Characteristics of the Different Modes 148
8.1.2 The Presence of the Interviewer 149
8.1.3 The Mode of Presentation 151
8.1.4 The Role of the Computer 152
8.1.5 Procedures without Asking Questions 155
8.1.6 Mixed-Mode Data Collection 155
8.2 The Position in the Questionnaire 156
8.3 The Layout of the Questionnaire 158
8.4 Differences due to Use of Different Languages 158
8.5 Summary and Discussion 159
Exercises 160
Part III Estimation and Prediction of the Quality of Questions 163
9 Criteria for the Quality of Survey Measures 165
9.1 Different Methods, Different Results 166
9.2 How These Differences Can Be Explained 173
9.2.1 Specifications of Relationships between Variables in General 173
9.2.2 Specification of Measurement Models 175
9.3 Quality Criteria for Survey Measures and Their Consequences 178
9.4 Alternative Criteria for Data Quality 181
9.4.1 Test–Retest Reliability 181
9.4.2 The Quasi-simplex Approach 182
9.4.3 Correlations with Other Variables 183
9.5 Summary and Discussion 184
Exercises 185
Appendix 9.1 The Specification of Structural Equation Models 187
10 Estimation of Reliability, Validity, and Method Effects 190
10.1 Identification of the Parameters of a Measurement Model 191
10.2 Estimation of Parameters of Models with Unmeasured Variables 195
10.3 Estimating Reliability, Validity, and Method Effects 197
10.4 Summary and Discussion 201
Exercises 202
Appendix 10.1 Input of Lisrel for Data Analysis of a Classic MTMM Study 205
Appendix 10.2 Relationship between the TS and the Classic MTMM Model 205
11 Split-Ballot Multitrait–Multimethod Designs 208
11.1 The Split-Ballot MTMM Design 209
11.1.1 The Two-Group Design 209
11.1.2 The Three-Group Design 210
11.1.3 Other SB-MTMM Designs 211
11.2 Estimating and Testing Models for Split-Ballot MTMM Experiments 212
11.3 Empirical Examples 213
11.3.1 Results for the Three-Group Design 213
11.3.2 Two-Group SB-MTMM Design 215
11.4 The Empirical Identifiability and Efficiency of the Different SB-MTMM Designs 218
11.4.1 The Empirical Identifiability of the SB-MTMM Model 218
11.4.2 The Efficiency of the Different Designs 221
11.5 Summary and Discussion 221
Exercises 222
Appendix 11.1 The Lisrel Input for the Three-Group SB-MTMM Example 222
12 MTMM Experiments and the Quality of Survey Questions 225
12.1 The Data from the MTMM Experiments 226
12.2 The Coding of the Characteristics of the MTMM Questions 229
12.3 The Database and Some Results 230
12.3.1 Differences in Quality across Countries 231
12.3.2 Differences in Quality for Domains and Concepts 234
12.3.3 Effect of the Question Formulation on the Quality 235
12.4 Prediction of the Quality of Questions Not Included in the MTMM Experiments 237
12.4.1 Suggestions for Improvement of Questions 239
12.4.2 Evaluation of the Quality of the Prediction Models 240
12.5 Summary 241
Exercises 242
Part IV Applications in Social Science Research 243
13 The SQP 2.0 Program for Prediction of Quality and Improvement of Measures 245
13.1 The Quality of Questions Involved in the MTMM Experiments 246
13.1.1 The Quality of Specific Questions 246
13.1.2 Looking for Optimal Measures for a Concept 250
13.2 The Quality of Non-MTMM Questions in the Database 252
13.3 Predicting the Quality of New Questions 256
13.4 Summary 261
Exercises 262
14 The Quality of Measures for Concepts-by-Postulation 263
14.1 The Structures of Concepts-by-Postulation 264
14.2 The Quality of Measures of Concepts-by-Postulation with Reflective Indicators 264
14.2.1 Testing the Models 265
14.2.2 Estimation of the Composite Scores 268
14.2.3 The Quality of Measures for Concepts-by-Postulation 270
14.2.4 Improvement of the Quality of the Measure 274
14.3 The Quality of Measures for Concepts-by-Postulation with Formative Indicators 276
14.3.1 Testing the Models 278
14.3.2 Estimation of the Composite Score 281
14.3.3 The Estimation of the Quality of the Composite Scores 282
14.4 Summary 283
Exercises 284
Appendix 14.1 Lisrel Input for Final Analysis of the Effect of “Social Contact” on “Happiness” 284
Appendix 14.2 Lisrel Input for Final Analysis of the Effect of “Interest in Political Issues in the Media” on “Political Interest in General” 285
15 Correction for Measurement Errors 287
15.1 Correction for Measurement Errors in Models with only Concepts-by-Intuition 287
15.2 Correction for Measurement Errors in Models with Concepts-by-Postulation 292
15.2.1 Operationalization of the Concepts 292
15.2.2 The Quality of the Measures 294
15.2.3 Correction for Measurement Errors in the Analysis 297
15.3 Summary 298
Exercises 299
Appendix 15.1 Lisrel Inputs to Estimate the Parameters of the Model in Figure 15.1 300
Appendix 15.2 Lisrel Input for Estimation of the Model with Correction for Measurement Errors using Variance Reduction by Quality for all Composite Scores 301
16 Coping with Measurement Errors in Cross-Cultural Research 302
16.1 Notations of Response Models for Cross-Cultural Comparisons 303
16.2 Testing for Equivalence or Invariance of Instruments 307
16.2.1 The Standard Approach to Test for Equivalence 307
16.3 Problems Related with the Procedure 309
16.3.1 Using Information about the Power of the Test 309
16.3.2 An Alternative Test for Equivalence 315
16.3.3 The Difference between Significance and Relevance 317
16.4 Comparison of Means and Relationships across Groups 318
16.4.1 Comparison of Means and Relationships between Single Requests for Answers 318
16.4.2 Comparison of Means and Relationships Based on Composite Scores 319
16.4.3 Comparison of Means and Relationships between Latent Variables 321
16.5 Summary 324
Exercises 325
Appendix 16.1 The Two Sets of Requests Concerning “Subjective Competence” 326
Appendix 16.2 ESS Requests Concerning “Political Trust” 327
Appendix 16.3 The Standard Test of Equivalence for “Subjective Competence” 328
Appendix 16.4 The Alternative Equivalence Test for “Subjective Competence” in Three Countries 329
Appendix 16.5 Lisrel Input to Estimate the Null Model for Estimation of the Relationship between “Subjective Competence” and “Political Trust” 331
Appendix 16.6 Derivation of the Covariance between the Composite Scores 333
References 336
Index 352
| Erscheint lt. Verlag | 23.5.2014 |
|---|---|
| Reihe/Serie | Wiley Series in Survey Methodology |
| Verlagsort | New York |
| Sprache | englisch |
| Maße | 164 x 243 mm |
| Gewicht | 631 g |
| Themenwelt | Geisteswissenschaften ► Psychologie |
| Mathematik / Informatik ► Mathematik ► Wahrscheinlichkeit / Kombinatorik | |
| Sozialwissenschaften ► Pädagogik | |
| ISBN-10 | 1-118-63461-6 / 1118634616 |
| ISBN-13 | 978-1-118-63461-5 / 9781118634615 |
| Zustand | Neuware |
| Haben Sie eine Frage zum Produkt? |
aus dem Bereich
