PETRA: participatory evaluation through redesign and analysis

Compared with single user-computer interactions, evaluating multi-user-computer interactions is much more complex. We argue for multiplicity — of theory, method and perspective — in the evaluation of computer-supported collaborative work (CSCW). This allows us to address both theoretical concerns and practical design issues, and to incorporate the expertise and experiences of both researchers and participants. We propose the PETRA framework, incorporating a theoretically-driven evaluators' perspective to investigate the collaborative activity, and a design-based, user-focused participants' perspective to evaluate the supporting tool. Our study investigated collaborative writing, both in a face-to-face context, and supported by a computer-based group editor. In our instantiation of the PETRA framework, we used distributed cognition and a form of breakdown analysis to investigate the development of shared understanding in the two different mediating settings; and devised a rapid prototyping session (inspired by participatory design) to elicit participant reactions to and redesigns of the tool interface. Our findings show that computer-supported shared understanding develops technologically, using social coordination as a repair mechanism; and that the collaborative tool must be particularly sensitive to issues of awareness, communication, focus and ownership.