Validity Aspects of Vignette Experiments : Expected 'What-If' Differences between Reported and Actual Behavior

Kurzfassung/Abstract

Vignettes are short descriptions of real‐life situations that are employed in a survey to assess attitudes and intentions. In many studies, intentions are considered as approximations to actual behavior in real‐life situations. What different applications of vignettes have in common is the effort to adjust the vignettes to the reality of everyday life. Because of this closeness to reality, it is assumed that vignettes are less prone to socially desirable response behavior and show both a high construct and external validity. However, there are only a few studies that have analyzed validity aspects of vignette‐based studies which, overall, led to ambiguous results. A current study by the authors is reported that shows limitations of the construct and the external validity of vignettes. The study comprises a systematic comparison of a field and a vignette experiment, which was based on the counterfactual model of causal inference and made use of propensity score matching. The results indicate that the determination of vignettes' quality is yet to come, particularly in the context of measurement of reported behavioral intentions as approximations to actual behavior. For this reason, we summarize the lessons to be learned from an evaluation of our study with regard to the current state of the art practices.