Crowdsourcing Lunch Seminar

  • Gates Hillman Centers
  • Traffic21 Classroom 6501
  • Assistant Professor
  • Computer Science & Engineering
  • Carnegie Mellon University

Volunteer-Based Online Experiments With Diverse Samples: Lessons Learned from Six Years of LabintheWild

An estimated 95% of our scientific knowledge about people, their behavior, perception, and preferences is based on studies with “WEIRD” samples, an acronym for participants who are Western, Educated, Industrialized, Rich, and Democratic. My work in Human-Computer Interaction has shown that technology developed for WEIRD users---based on knowledge that is derived from studies with mostly US-based student populations---is often misaligned with the preferences, behaviors, and abilities of a large proportion of the world's population. Users differ in their goals, how they perceive information,  and what they can work with most efficiently.

In this talk, I report on six years of experience running the volunteer-based online experiment platform, which has enabled behavioral experiments at larger scale and with less WEIRD participants than feasible in laboratory studies or on Mechanical Turk. LabintheWild enables participants to compare themselves to others in exchange for study participation; a feedback mechanism that has attracted an average of more than 1,000 participants a day from 230 countries. I present ten main lessons learned from this experience and show how LabintheWild experiments have enabled us to build volunteer-powered, self-sustaining design support tools in various domains.   

Katharina Reinecke is an Assistant Professor in the Paul G. Allen School of Computer Science and Engineering at the University of Washington. Prior to joining the University of Washington, she was an Assistant Professor in the School of Information at the University of Michigan. She received her Ph.D. in Computer Science from the University of Zurich and spent her postdoctoral years at Harvard School of Engineering and Applied Sciences. Her research explores how people from various demographic and geographic backgrounds vary in their use of technology with the goal to create user interfaces that automatically adapt to people's abilities, preferences, and perception. To find out how people differ, she co-founded the volunteer-based online experiment site, which has enabled her to study several million participants from 230 countries. Katharina's work has been recognized with several Best Paper awards and nominations at premier venues in Human-Computer Interaction (ACM CHI, ACM CSCW) and an NSF CAREER Award.


For More Information, Please Contact: