Kongressbeitrag zum European Congress of Psychology
Kaser, A. & Sachse, Pierre (2011). Advanced online research methods – Bringing together advertisement psychology and web analytics.
12th European Congress of Psychology (ECP 2011), Abstracts, p. 1215, Instanbul, Türkei.
Transkript:
Quality criteria for experiments in media and consumer psychology
Research in media and consumer psychology has often been criticized for not studying processes with actual target groups and within natural settings.
Felser (1997, p. 444) provides a synopsis of quality criteria that studies should meet. They can be used to evaluate the practical relevance of research that is aimed to show the effective-ness of advertisements.
Criteria are:
- Is the stimulus material framed within some context (as in reality) or presented in isolation?
- Can participants choose between different options (as in reality)?
- Are advertisements for other products also shown?
- Are participants presented with the stimulus material more than once?
- Are advertisements created for known (i.e., realistic) or unknown (i.e., unrealistic) products?
Laboratory vs. field studies
Laboratory experiments, as in most research in this field, entail trade-offs. First, laboratory situations can only capture naturally occurring situations to a minor degree (criterion 1). Second, the number of alternative options to choose from as well as products is limited due to pre-selection of stimulus material from researchers (criteria 2 and 3). Third, repeated presentation of stimulus material is hardly possible because it is difficult to tie participants to several trails over a longer time within studies.
Field studies, on the other hand, have better ecological validity, which is concomitant with less internal validity as independent variables cannot be freely and readily manipulated. However, weaknesses and limitations of laboratory and field studies can be overcome by employing online studies.
Data gathering
Once the participant has opened the website in his/her browser, a Web Analytics System protocols any actions of the user. Almost any data can be gathered with this method: be-sides product and purchase decisions, indexes of dwell time and drop-outs can be derived. These can be essential for interpreting results. The data gathered by Web Analytics systems can be readily imported in common statistics packages (e.g., IBM SPSS; R). For simple experiments, the freely available Web Analytics system Google Analytics is sufficient (see for example Aden (2010)).
A new approach: Experiments
Example: lt should be investigated whether user experience influences purchase decisions in online shops.
Procedure:
Generation of the study platform
Two independent variants of the same website are generated, one version with high user experience and the other with low user experience. These versions only vary on circumscribed dimensions that represent the IV manipulation. Usually, this manipulation or the variables being manipulation is the focus of the research.
Fig. 1: Two variants of a website: „High user experience” vs. „Low experience”
Recruiting of participants
Participants can be acquired online by using Google AdWords to create online advertisements embedded in Google. If a user clicks on the ad, (s)he will be automatically forwarded to one of the two versions of the online shop. Thus, the user becomes a participant. Advertising costs are per click (0.50-1.00 €.
Fig. 2: Positioning of ads in Google’s search engine
Conclusion
The combination of online ads, A-B test, and Web Analytics provides an effective and low-cost alternative to traditional laboratory experiments and field-based studies that combines the strengths of both approaches while keeping the limitations to a minimum. Of course, such online designs can only be applied if the research focus allows this and can be readily implemented into an online setting.
The trade-off between experimental manipulation (that provides evident for causal effects) and practical significance in naturalistic designs (Felser, 1997, p. 445) can be overcome by incorporating the presented online approach. Internal validity is given because IVs can be readily manipulated to any extent the researcher wants it, and effects can be traced back to IVs and their manipulation (e.g., in user experience). External/ecological validity, however, is also given because the research context is similar or even identical to real-life surroundings (e.g., there is no difference to a “real” website). The live quality criteria outlined by Felser (1997) are these all met.
Literature and links
Felser, G. (1997). Werbe- und Konsumentenpsychologie: Eine Einführung. Stuttgart: Schäffer-Poeschel.
Aden, T. (2010). Google Analytics: implementieren, interpretieren, profitieren (2. ed.). Munich: Hanser.
Google AdWords can be found on: https://ads.google.com/intl/de_AT/home/
Google Analytics can be found on: https://marketingplatform.google.com/about/analytics/