Museum Guide 2.0 – An Eye-Tracking based Personal Assistant for

Museum Guide 2.0 – An Eye-Tracking based Personal Assistant for

Proceedings of the international conference: “Re-Thinking Technology in Museums 2011: Emerging Experiences”. 26-27 May 2011, Limerick, Ireland Edite...

56MB Sizes 0 Downloads 10 Views

Recommend Documents

Das Sehen. ○ Die Blickbewegung. ○ Der Blickverlauf. ○ Was ist Eyetracking? ○ Eyetracking: Funktionsprinzip. ○

Intelligent Personal Assistant - Kamitis
Thanks to the rise in smartphone sales, there is a growth in Intelligent Personal. Assistant (IPA) usage. In fact, assis

Donna - A web based AI Personal Assistant - International Journal of
This project, Donna, aims at creating a personal assistant based on artificial intelligence .... Assistant. Google. Assi

Knowledge Graphs for Personal Assistant - CEUR Workshop
Aug 11, 2017 - From Facts to Acts: Knowledge Graphs for Personal. Assistant ... projects included the Google Assistant N

Eyetracking Aufzeichnungsverfahren
14.05.2009 - Eyetracking. Ziele des Eyetracking. Aufzeichnung von Augen- bzw. die Blickbewegungen einer. Testperson und

Personal Assistant Details Heather Mills
Heather Mills. Address: Crieff. Daytime Phone: 07969 376773. Mobile: 07969 376773. Email: [email protected] Are

TV Spielfilm wird Personal Assistant
12.10.2017 - „Was läuft um 20.15 Uhr?“ Wer diese Frage stellt, bekommt künftig Antwort von Google – und das dank

Diagnostik und Eyetracking - Alcon
Mit der VERION Referenzeinheit nehmen Sie wichtige diagnostische Messungen vor: • Dynamische Keratometrie. • Limbusp

Personal Assistant Administration Support - eTeach
Jun 22, 2017 - Filton Road, Bristol BS7 0XZ | 0117 377 2000 ... expressing an interest in the permanent post of part tim

Personal Assistant to the Ambassador
Position description: This is a demanding role in a busy working environment. As PA to the Ambassador and event manager

Proceedings of the international conference: “Re-Thinking Technology in Museums 2011: Emerging Experiences”. 26-27 May 2011, Limerick, Ireland

Edited by: Luigina Ciolfi ([email protected]) Interaction Design Centre, University of Limerick Katherine Scott ([email protected]) Irish Museums Association Sara Barbieri ([email protected]) Interaction Design Centre, University of Limerick

Printed in Ireland by thinkk creative & the University of Limerick ISBN/S 1-905952-31-7 978-1-905952-31-1

“Re-thinking Technology in Museums 2011” is sponsored by:

This compilation is copyright ©2011 by Ciolfi, Scott & Barbieri. Copyright of each individual article remains with the respective author(s). Permission to copy without fee all or part of this work is granted provided that the copies are not made or distributed for direct commercial advantage

Conference Committees Conference Chair Luigina Ciolfi, Interaction Design Centre, University of Limerick Organising Committee Luigina Ciolfi, Anne Murphy, Department of Computer Science and Information Systems, University of Limerick Katherine Scott, Irish Museums Association Irish Museums Association Workshop Coordinators Kieran Burns Brian Crowley Paul Doyle Innovation Forum Coordinators Gabriela Avram, University of Limerick & IxDA Limerick Marc McLoughlin, University of Limerick & IxDA Limerick Hunt Museum Liaison Dominique Bouchard Webmaster and Technical Support Fabiano Pinatti, University of Limerick Graphic Design Nora O’Murchú, University of Limerick Programme Committee Linda Ballard, Ulster Folk and Transport Museum, UK Liam Bannon, University of Limerick, Ireland Dominique Bouchard, The Hunt Museum, Ireland Luigina Ciolfi, University of Limerick, Ireland Brian Coates, University of Limerick, Ireland Eileen Coates, Independent Curator, Ireland Michael Cooke, Trinity College Dublin, Ireland Patrick Cooke, University College Dublin, Ireland Vincenzo D’Andrea, University of Trento, Italy Lily Díaz, Aalto University, Helsinki, Finland Maria Economou, University of the Aegean, Greece Mikael Fernström University of Limerick, Ireland Areti Galani, Newcastle University, UK Elisa Giaccardi, Universidad Carlos III de Madrid, Spain Georgina Goodlander, Smithsonian American Art Museum, USA Halina Gottlieb, Interactive Institute, Sweden Paul Green, Cork Institute of Technology, Ireland Jon Hindmarsh, King’s College London, UK Eva Hornecker, University of Strathclyde, UK Tobie Kerridge, Goldsmiths University of London, UK Ann Light, Sheffield-Hallam University, UK Carlo Maiolini, Museo Tridentino di Scienze Naturali, Italy John McCarthy, University College Cork, Ireland Marc McLoughlin, University of Limerick, Ireland Annette Moloney Independent Curator, Ireland Helen Monaghan, National Gallery of Ireland Joan Nordbotten, University of Bergen, Norway Shaleph O’Neill, University of Dundee, UK Daniela Petrelli, Sheffield-Hallam University, UK Palmyre Pierroux, University of Oslo, Norway


Pille Pruulmann-Vengerfeldt, University of Tartu, Estonia Elisa Rubegni, Universita’ della Svizzera Italiana, Switzerland Mariana Salgado, Laurea University of Applied Sciences, Finland Dagny Stuedahl, University of Oslo, Norway Loic Tallon, Pocket-Proof, UK Michael Twidale, University of Illinois Urbana-Champaign, USA Dirk vom Lehn, King’s College London, UK Kevin Walker, University of the Arts London, UK


Table of Contents Conference Committees


Introduction by Luigina Ciolfi, Katherine Scott, Dominique Bouchard, Kieran Burns and Marc McLoughlin

P. 1

Multi-touch Interfaces in Museum Spaces: Reporting Preliminary Findings on the Nature of Interaction by Jenny Kidd, Irida Ntalla and William Lyons

P. 5

Digital Natives: Creating Emergent Exhibitions through Digital Technologies by Rachel Charlotte Smith, Ole Sejer Iversen and Christian Dindler

P. 13

The Role of Appropriation in the Design of Engaging Artefacts by Tom Flint and Phil Turner

P. 25

Co-Design of the Interactive Museographic Exhibits: the MIME Case Study by Christophe Bortolaso, Cédric Bach, Francis Duranthon and Emmanuel Dubois

P. 37

Towards the Wild: Evaluating Museum Installations in Semi-Realistic Situations by Eva Hornecker and Emma Nicol


Historical Orchestra: An Interactive Audio/Visual Cultural Experience by Ferhat !en and Lily Díaz

P. 61

Applying Basic Design Principles to Technology in Museum by Mark Leslie

P. 71

Promising Beginnings? Evaluating Museum Mobile Phone App by Maria Economou and Elpiniki Meintani

P. 87

Museum Guide 2.0 – An Eye-Tracking Based Personal Assistant for Museums and Exhibits by Takumi Toyama, Thomas Kieninger, Faisal Shafait and Andreas Dengel

P. 103

Exploring Exhibitions in Virtual Worlds: Case Studies in Using Open Technologies by Alan Miller, Sarah Kennedy, Lisa Dow and Colin Allison

P. 111

Living with a Virtual Agent: Seven Years with Embodied Conversational Agent at the Heinz Nixdorf Museums Forum by Thies Pfeiffer, Christian Liguda, Ipke Wachsmuth and Stefan Stein


Touching the Stars by Niall Smith, Alan Giltinan, Paul Walsh, Clair McSweeney and Frances McCarthy

P. 133

Re-Thinking Museum Assemblies by Dagny Stuedahl and Ole Smørdal

P. 141

Museum as a Playground? Designing digitally enhanced play spaces for children's engagement with museum objects by Katarzyna Warpas

P. 151

Digitizing the Valuable – Value the Digitized by Anne Britt Ylvisåker

P. 167

Paper Title: Mobile Phones in U.S. Science Museums: A Report of an Exploratory Survey by Hyunsook Youn and Marian Stewart Titus

P. 177

LodeStar: a Mobile Device to Enhance Visually Impaired People Experience of Cultural and Naturalistic Places by Francesco Bellotti, Riccardo Berta, Alessandro De Gloria and Massimiliano Margarone

P. 189

Prototype Places: Curating Practice-Based Research in a Museum Context by Deborah Turnbull, Matthew Connell and Ernest Edmonds

P. 203

Asian Heroes in Cultural Heritage Context: An Interactive Installation for Children Combining Language and Mythology by Ng Bee Chin, Fabrizio Galli and Halina Gottlieb

P. 215

Re-Positioning Exhibits in Time and Space by Florian Windhager & Eva Mayr

P. 235

Increasing the Usability of the Museum: Four Studies by Pille Pruulmann-Vengerfeldt and Pille Runnel

P. 243

A Virtual World Installation in an Art Exhibition: Providing a Shared Interaction Space for Local and Remote Visitors by Spyros Vosinakis & Ioannis Xenakis

P. 253

Introduction Luigina Ciolfi*, Katherine Scott^, Dominique Bouchard†, Kieran Burns^ and Marc McLoughlin*º * Interaction Design Centre, Dept. Of Computer Science and Information Systems, University of Limerick (Ireland) [email protected] ^ Irish Museums Association (Ireland) [email protected] [email protected] † The Hunt Museum, Limerick (Ireland) [email protected] º IxDA Limerick (Ireland) [email protected]

1. About the Conference This edited collection includes the research papers written for “Re-Thinking Technology in Museums 2011: Emerging Experiences” and presented at the conference on May 26-27 2011 in Limerick (Ireland). We are delighted to see this event happen again at the University of Limerick after the very successful 2005 edition. In 2005, “Re-Thinking Technology in Museums” gathered a group of academics and practitioners discussing novel ways of conceptualizing the museum experience in light of the presence of interactive technologies (Ciolfi et Al, 2005). The second edition of the conference on the theme of “Emerging Experience” furthers the discussion on novel approaches for understanding people’s experiences in museums, galleries, and other places of heritage, and for designing interactive technologies to support these experiences. In recent years, the increased presence of mobile smart appliances such as smart phones, and the growth of social media and social networks have impacted on the strategies deployed by museums and exhibition sites to invite, engage and connect with visitors and stakeholders (Tallon & Walker, 2008). It is important to reflect on how museums/galleries and visitors have been affected from an experiential point of view: what have museums and galleries become? And what about the role of visitors? How are meaningful and rewarding experiences emerging in this context? A lively stream of research has surfaced seeking answers to these questions and investigating notions of participation, co-curatorship and experience design in cultural heritage through social and collaborative media (Simon, 2010; Giaccardi & Palen, 2008). “Re-Thinking Technology in Museums 2011” invited submissions discussing conceptual approaches to studying the experiential qualities of visits (such as social interaction, development of a sense of place, learning and critical reflection); descriptions of methodological approaches for understanding visitors’ experience; case studies describing museum’s staff experiences in understanding and supporting visitors by means of technology; case studies describing the design of interactive (low-tech and hightech) exhibitions, and the deployment of other IT tools such as social media; case studies describing the development of educational workshops and/or installations in museums and galleries aided by interactive technologies. We also wished to convene a conference representing different stakeholders with an interest in technology and heritage, and encouraged social scientists, museum education and communication experts, curators, exhibition designers, interaction designers and computer scientists to submit a paper. We particularly welcomed submissions from museum and gallery experts, describing their current practices in supporting visitors’ experiences and their concerns in supporting visitors’ engagement, interest and informal learning, as much as from researchers exploring the potential of new technologies and suggesting innovative designs.


A great accomplishment of “Re-Thinking Technology in Museums 2011” is the partnership between the Interaction Design Centre at the University of Limerick and the Irish Museums Association in hosting and shaping the conference’s thrust. It is extremely significant that a bridge has been established between academic research and a network of professionals and institutions in providing a venue for discussion for a multidisciplinary community. The Interaction Design Centre (IDC)1 is a multi-disciplinary research group studying the design, use and evaluation of information and communications technologies. The focus is on human-centred design, with a strong interest in collaborative settings, exploring the design and use of novel interactive and communicative artefacts to support human activities. Work in the IDC covers a wide spectrum, from the design and evaluation of new media installations and interfaces to field studies of technology in use in different settings. The IDC has conducted research on the theme of technology design for cultural heritage for well over ten years, experimenting with innovative exhibitions (Ferris et Al, 2004; Ciolfi et Al, 2007; Ciolfi & McLoughlin, 2011), exploring the potential of visitor and staff participation (Ciolfi & Bannon, 2007; Ciolfi, Bannon & Fermström, 2008) and reflecting on the appropriateness and value of technology in a variety of exhibition settings (Ciolfi, Bannon & Fermström, 2001; Avram, Ciolfi & McLoughlin, 2010). The Irish Museums Association2 is a voluntary, not for profit association, founded in 1977. It is dedicated to promoting the interests of museums and the Irish museums community (those who work and are interested in museums) throughout Ireland, north and south of the border. The Irish Museums Association aims to promote professional practice in museum management, collections care and visitor services; provide a platform for anyone interested in Irish museums to provoke debate on museum ideas; and, in co-operation with other professional bodies and central and local government, to achieve a sustainable level of museum provision and operation in Ireland. The IMA is making a special contribution to the conference with “Irish Museums & Technology: Ideals & Realities”, a Workshop which will feature three invited talks: Eoin Kilfeather from DIT & Mark Maguire from IMMA (jointly presenting the “Decipher” project), Jennifer Mc Creagh from Fota House and the Irish Heritage Trust, and Paul Savage from Zolk-C. This practical and interactive workshop was designed to honestly evaluate how Irish museums are engaging with the Internet, information technology, and social networking. Emphasis is placed on how IT has impacted positively and negatively areas such as curation and interpretation, education, and marketing. Participants are encouraged to share their own experiences as part of the process in building a common understanding of how technology is and isn’t working in museums in Ireland. The partnership between the IDC and the IMA is at the core of the conference. Moreover, the event benefits from the contribution of two other collaborators: the Limerick Chapter of the Interaction Design Association and the Hunt Museum. The Interaction Design Association (IxDA)3 is a global network dedicated to the professional practice of Interaction Design. With the help of more than 20,000 members since 2004, the IxDA network provides an online forum for the discussion of interaction design issues and other opportunities and platforms for people who are passionate about interaction design to gather and advance the discipline. IxDA Limerick is the local chapter of the IxDA; it supports the local Interaction Design community by hosting talks from leading national and international Interaction Design researchers and practitioners. The IxDA Limerick group is organizing an Innovation Forum for “Re-thinking Technology in Museums 2011”. The Forum provides opportunities for companies, individuals and organisations to showcase technologies and services that are designed to support visitors’ experiences for museums, exhibitions and other heritage sites, or technologies that hold potential for such domains. The forum also provides significant networking opportunities between heritage professionals and design practitioners – two communities that have few occasions to discuss and create new partnerships. The forum consists of an exhibition running in parallel to the conference and a dedicated “pecha kucha” session where the Forum presenters can pitch their demo to all the conference delegates. The Hunt Museum4 in Limerick a has a permanent collection of around 2,000 objects which include decorative and fine arts objects from the ancient world to the twentieth century. Its archaeological collection contains artefacts from ancient Greek, Roman, Egyptian and the Olmec civilisations and there is also an important collection of Irish archaeological material ranging from Neolithic flints and 1 4 2 3


Bronze Age material, including a Bronze Age shield and cauldron. Other collection highlights include Christian objects such as the unique ninth-century Antrim Cross. One of the strengths of the Hunt collection is the medieval material, which include statues in stone and wood, painted panels, jewellery, enamels, ivories, ceramics, crystal and crucifixes. Eighteenth and Ninteenth-century decorative arts are also represented with fine examples of silver, glass and ceramics. Paintings on display in the Museum include works by Pablo Picasso, Pierre Auguste Renoir, Roderic O’Conor, Jack B. Yeats, Robert Fagan and Henry Moore. The Hunt Museums offers a lively programme of events and educational workshops, concerts, lectures and is supporting the conference by providing free access to the delegates during their stay in Limerick and by contributing to the organization and running of some of the conference sessions. We hope to have provided an exhaustive overview of the extraordinary network of collaborations that has made “Re-Thinking Technology in Museums 2011” such a rich and varied event.

2. About the Papers What follows in this book is the selection of 22 research papers that have been included into the main Conference programme after an in-depth peer-review process. The contributions span four continents and a wide range of institutions including museums, galleris, private companies and universities/research centres. The themes emerging from the papers are several. There is a strong element of theoretical reflection and conceptual development: for example, Windhager & Mayr propose a conceptual foundation to guide technology development through a classification of the spatial and temporal features of exhibitions; Stuedahl & Smørdal revisit the notion of “assembly” by introducing elements of Actor Network Theory into its conceptualisation, in order to include visitors’ active participation into the definition of an exhibit. A number of contributions document practical exercises in technology design to enhane visitor experiences and concrete applications of guidelines and principles. For example, Flint & Turner discuss the importance of a design approach to appropriation; Turnbull et Al. reflect on the importance of practice-based research for curation. Runnel & Pruulman-Vengerfeldt write on the importance of principles of good usability to achieve participation. From a methodological point of view, Hornecker & Nicol discuss the need to conduct technology evaluation “in the wild” in order to learn about the visitor experience. From the perspective of a design company, Leslie suggests a set of guidelines emerging from his reflections over many years of practice in designing engaging and successful interactive exhibits for a variety of sites. The programme features a number of case studies describing the life of actual interactive exhibitions, such as “NIME” described by Bortolaso et Al., and the “Historical Orchestra” (!en & Díaz). Pfeiffer et Al. provide an account of a 7-year trial of a virtual agent at the Heinz Nxdork MuseumsForum, showing how museum technology is now mature enough to allow not just for brief experimentations, but for long-lived interventions. In the group of papers documenting practical applications, two themes stand out as particularly relevant in current research: the first is that of mobile devices and applications. Economou & Meidani give an overview of museum mobile applications and how they fare when subjected to evaluation. Youn & Titus present an exploratory survey of the current use of mobile phoens in U.S. science museums, giving a timely description of how these devices are being employed in a very large “market” for museum technologies. Bellotti et Al., on the other hand, describe an example of a novel application of a mobile device, “LodeStar”, particularly targeted at enhancing the experience of visually impaired visitors, showing how mobile platforms can be configured to address the needs of specific visitor groups. The second dominant theme is that of interaction and participation through social media: Kidd et Al. frame their entire paper on a vision of the museum as “social”, thus suggesting how considering this dimension of the visitor experience is paramount when creating new interactive experiences. Rachel Smith et Al. present a case of design essentially targeted to audience participation in the “Digital Natives” project. Warpas discusses in specific children’s participation, articulating the vision of the museum as a playful participative context for a particular visitor group. Finally, a number of papers describe experimental trials of “new” technology in cultural heritage settings. These contributions document cutting-edge technology and novel attempts to fit it to the museum space, and they are important efforts in order to think of the future of interactive technologies for visitor experience in museums and to inspire novel design and development approaches. Toyama et Al. present an experimental demo using eye-tracking, whereas Vosinakis & Xenakis describe the creation of an entirely virtual 3D exhibition space to enhance the work of artist Ioannis Xenakis, and discuss how the vistual space created links between local and remote visitors.


Overall, we feel that this collection of papers offers a tremendous representation of current work on supporting visitor experiences of cultural heritage. We are very proud of presenting these contributions to the conference delegates in Limerick, and to readers worldwide.

Acknowledgements We wish to acknowledge the contribution of all the Programme Committee members for their thorough, insightful and helpful comments on all the submissions that were received. Thanks to all the members of the IDC who provided much-needed help on a large number of big and small tasks around the conference. In particular, thanks to Fabiano Pinatti for maintaining the website, to Nora O’Murchú for her design of the conference’s graphic theme, and to Sara Barbieri for assisting with the editing of this volume and contributing to the conference’s social media presence. The conference would not have been possible without the logistical support of the Department of Computer Science and Information Systems and of Campus Life Services at UL. Finally, we acknowledge the generous sponsorship of Faílte Ireland under the “Meet in Ireland” conference support programme.

References Avram, G., Ciolfi, L. and McLoughlin, M. (2010), "Get Connected: Your Museum and Social Media", Museum Ireland, Vol. 20, Irish Museums Association, pp110-120 Ciolfi, L. and Bannon L.J. (2007), “Designing Hybrid Places: merging interaction design, ubiquitous technologies and geographies of the museum space”, Co-Design, 3:3, 159 - 180 Ciolfi, L., Bannon, L., and M. Fernström (2001). “Envisioning and Evaluating ‘Out-of-Storage’ Solutions”, in Bearman, D. & Garzotto, F. (Eds.), Proceedings of ICHIM01 International Cultural Heritage Informatics Meeting: Milan, September 2001. Ciolfi, L., Bannon, L.J. and Fernström, M. (2008), "Including visitor contributions in cultural heritage installations: Designing for participation", Museum Management and Curatorship, Vol. 23, Issue 4, pp 353-365 Ciolfi, L., Cooke, M., Hall, T., Bannon, L.J. and Oliva, S. (eds.) (2005), Proceedings of the International Workshop “Re-Thinking Technology in Museums: towards a new understanding of people’s experience in museums”, University of Limerick June 2005, ISBN 1-874653-80-1 Ciolfi, L., Fernström, M., Bannon, L.J., Deshpande, P., Gallagher, P., McGettrick, C., Quinn, N., and Shirley, S. (2007) “The Shannon Portal Installation: An Example of Interaction Design for Public Places”, IEEE Computer, 40:7, July 2007, pp 65-72. Ciolfi, L. and McLoughlin, M. (2011), “Physical Keys to Digital Memories: Reflecting on the role of tangible artefacts in "Reminisce" “, in Bearman, D. and Trant, J. (eds.) (2011), Proceedings of Museums and the Web 2011, Museums and Archives Informatics. Ferris, K., Bannon, L., Ciolfi, L., Gallagher, P., Hall, T. and M. Lennon (2004), “Shaping Experiences in the Hunt Museum: A Design Case Study”, Proceedings of DIS04, Boston, August 2004. Giaccardi, E. and Palen, L. (2008), ‘The Social Production of Heritage through Cross-media Interaction: Making Place for Place-making’, International Journal of Heritage Studies 14.3. Simon, N. K. (2010), The Participatory Museum, Museum 2.0, available online at:, last accessed on 29 April 2011. Tallon, L. and Walker, K. (Eds.), Digital Technologies and The Museum Experience, Plymouth: AltaMira Pres


Multi-touch interfaces in museum spaces: reporting preliminary findings on the nature of interaction Jenny Kidd, Irida Ntalla and William Lyons City University London [email protected] [email protected] [email protected] Abstract For the past twenty years there has been a slow trickle of research disseminated through a variety of channels on the nature and use of computer interactives within museum and gallery environments. This research has yet to be consolidated into a robust and coherent evidence base for considering and understanding the continued investment in such interactives by institutions. Simultaneously however, the technology has changed almost beyond recognition from early kiosk-based computer exhibits featuring mostly film and audio content, through to the newer generation of multi-touch interfaces being introduced in the UK and beyond. This paper seeks to establish what can be gleaned from prior research in the field of computer interactives, to inform the study of these latest technological forms. It reports preliminary observations from the study of multi-touch interfaces and discusses issues identified for their continued investigation; principally, asking questions about the nature of experience and interactivity in such encounters and the research methods that we might use to better explore and understand their use in the future.

1 Introduction ‘We are crossing the threshold into the Era of the Good Interface’ (Dodsworth, 1999) For the past twenty years there has been a slow trickle of research disseminated through a variety of channels on the nature and use of computer interactives within museum and gallery environments. This research has yet to be consolidated into a robust and coherent evidence base for considering and understanding the continued investment in such technologies. This is no doubt for a variety of reasons; the case by case (institution by institution) evaluative nature of much of the research; the interdisciplinary scope of much of the investigation meaning that publication takes place in a host of varying locations5; the variety of approaches to study which can make consolidation or comparison of findings problematic; and the number of technologies that are seen to constitute the field (if we can call it that) of computer interactives. Simultaneously however, we have seen the technologies on offer to museums change almost beyond recognition from early kiosk-based computer exhibits (Gammon and Burch, 2008) featuring mostly film and audio content, through to the newer generation of multi-touch interfaces currently being introduced through new exhibitions in the UK and beyond. Such a change means that the variety of encounters with digital information currently on offer at any one site may be very varied in terms of usability, access and (no doubt) use-value; namely, experience. This shift has been accompanied by a rising interest in what mobile media may also add into the mix, with increased research activity around the use of PDAs, mobile phones, QR codes and RFID tags.6 In the last decade, more advanced forms of mixed interactive systems have been developed which include augmented reality (AR), mixed reality


Online databases of conference proceedings like, Museum and Heritage Journals, Education Journals, Technology, Information Science and even Engineering outlets. 6 The technologies are under-researched and there is no consistent body of knowledge but see Hsi 2003; 2004; Fleck et al. 2002, Baldwin and Kuriakose, Lejoe Thomas, 2009; and Lehn and Heath, 2005 on RFID and PDAs (also vom Lehn and Heath, 2003 and Woodruff et al, 2001 on PDAs); Tallon and Walker (ed), 2008 on mobile phones; and more recently Chan, 2009 and Flowers, 2010 on QR codes and apps.


(MR) and tangible user interfaces (TUI) or ‘tangible interactives’.7 These latter terms recognise tangibility and materiality in new ways; embracing the physical embodiment of data, bodily interaction and embeddeness in real spaces and contexts (Eva Hornecker, 2009). Additionally, according to Hornecker the idea of tangible interaction focuses on human control, creativity and social action rather than the representation and transmission of data, a shift which would have significant implications for the role and status of interactives in museum spaces. Such far reaching developments make for a rather incoherent knowledge base, and raise serious methodological concerns. This paper will be unable to elucidate on the breadth, scope and impact of these changes in context, but instead seeks to explore the study of multi-touch interfaces in particular. The distinction is made between various ‘computer interactives’ and ‘multi-touch interfaces’ in a variety of ways. Computer interactives (Figure 1) tend to be static, the size of your average computer monitor, and navigated by a sole individual (indeed a sole finger) providing information on a number of different levels. They tend to be characterised by their kiosk appearance and (necessarily) dark surroundings. Multi-touch interfaces (Figure 2) may house similar kinds of information, but are differentiated by the mode and aesthetics of their presentation. The interface tends to be a flat, horizontal tablet screen that can be accessed (and read) by a variety of people at any given time. These interfaces tend to be more happily integrated into exhibitions as the restrictions on their presentation are fewer.

Figure 1: Computer interactive at the Victoria and Albert Museum

7 Ullmer and Ishii, from the Tangible Media Group at the MIT Media Lab, define TUIs as devices that give physical form to digital information, employing physical artefacts as representations and controls of the computational data (Ullmer and Ishii, 2000).


Figure 2: A multi-touch interface at the Museum of London. Multiple point of entry can be identified. The dubious nature of the ‘interactivity’ on offer through screen-based media at museum sites is notable, but the shift to ‘interface’ demonstrated in this paper is more than mere semantics. It implies that there are important changes to both the hardware and software being utilised, and the inputs and outputs being detected, but falls short of indicating a genuine two-way communication. There are issues of power and authority at play in any museum/visitor transaction, and the easy promise of participation, collaboration and interactivity has been all too seductive in recent years. This paper gives an overview of preliminary findings related to the use of multi-touch interfaces in museum environments along two thematics: the nature of experience and interactivity in such encounters (including levels of sociability and comfort); and the research methods that we might use to better explore and understand their use in the future. It concludes with a number of pointers for further research, and a brief commentary on the positionality of computer interactives more generally within the larger narratives constructed by institutions. In exploring preliminary findings, the paper also seeks to establish what can be salvaged from prior research to inform investigation of multi-touch interfaces in particular. This paper thus begins to explore whether we might at last be entering the ‘Era of the Good Interface’, or if the situation might be better described as ‘business as usual’ in respect to multi-touch interfaces. Are such opportunities for experience and interaction no more than an extension of those activities offered through previous incarnations of the ‘computer interactive’? Or is there scope to explore and recommend them as more social, educative, coherent and fluid contributions to the museum narrative? The approach here is cross-disciplinary, empirical and critical at core, but might be best described as ‘socio-technical’ in perspective (Sawyer, 2005: 9).

2 Preliminary observations This section reports preliminary observations of users’ encounters with computer interactives carried out in 2010 at 9 museums in England. The researcher observed more than 100 screen-based interactions in total, including 30 encounters with multi-touch interactives. The purpose of the observation was to try and establish any shift in terms of the (visible) nature of experience and interaction, and to think creatively about how we might try and capture something of that experience through non-intrusive research methods in the future. A number of promising lines of enquiry were noted, and are detailed here under the headings; experience and interaction, and methodological implications.


2.1 Experience and Interaction The experience of using the multi-touch interface is visibly a significantly more social one than the use of traditional computer interactives observed. One of the major criticisms of computer interactives in the past has been that they can limit users’ social experience of museum environments (a crucial aspect of any museums visit according to Falk and Dierking and a key component of any learning outcome, 2008). It has been noted that the computer screen is not best suited to enabling social interaction (Gammon and Burch, 2008:47), and to expect multiple visitors to feel comfortable using a computer at one time is of course counter to our use of and engagement with computer screens in most of our day to day activity. In the study, people were observed to engage with interfaces simultaneously and in groups from all available sides, and often to carry out multiple ‘tasks’ on the interface at one time. The nature of these encounters of course remains allusive, and establishing a means for gathering information about them continues to present methodological challenges (as will be seen). Such use of the interface does however represent a step towards ‘overhearing’, ‘co-participation’ and ‘multi-party participation’, activities that vom Lehn and Heath are eager to encourage more readily around computer interactives (2005: 15). Indeed, to noted museum consultant Nina Simon, it is only when individuals and groups communicate with one another around content (not through it) that productive participation and interaction are being facilitated (Simon, 2010). It would be interesting to see also how the new generation of multi-touch interfaces and TUIs which incorporate physical artefacts in interactive experiences might (as is their aim) aid users’ collaborative activities with one another. It is crucial to the future research agenda that we understand more about the nature of such social encounters, for example, asking questions about whether they are characterised by frustration, enquiry, knowledge exchange, or are completely ‘off topic’. Observation of social encounters thus raised questions about how we might more usefully conceptualise the ‘social museum’ and understand the relationships between collective and individual engagement; the body and experience; the group and knowledge construction. Understanding where, how and whether computers interfere with, support or enhance such relationships will be of critical importance going forwards. It was noted in the use of digital devices of all types that there were different kinds of permissions at play than at other exhibits on the museum floor. For example, whilst children in groups often ran up to interface exhibits and started engaging, they were less likely to do so until they had been given ‘permission’ when it was a traditional computer interactive. This is perhaps due to the nature of young people’s relationships with monitor based computers at home or school, but does indicate that the multi-touch interface perhaps suffers less from that kind of association. It has been recognised in prior literature that computer interactives tend to appeal particularly to children (Ramsay, 1998; Gammon, 1999) who may not be the target audience for the information, but who will commit (at some level) to using the device none-the-less. ‘Interactivity’ has been central to attempts to engage children and young people within museum spaces for many years, not least because of the roots of that principle in philosophies of experiential learning (Dewey 1997) and constructivism (Piaget 1957). McLean (1993) defines interactive exhibits as ‘those in which visitors can conduct activities, gather evidence, select options, form conclusions, test skills, provide input and alter a situation based on input’. Such things are seen as critical to learning ‘outcomes’. In one study, Hein and Heald (1988) found that multi sensory interactive installations kept visitors for more time at exhibits and subsequently increased visitor knowledge. Nonetheless, interactions with multi-touch interfaces and computer interactives depend largely on their application and their setting. For both children and adults who use computer interactives, there is a recognised problem of pacing which needs further attention in research. Being able to move at one’s own pace through the exhibit is important, yet computer interactives (and especially those that have been in situ for a number of years) can be slow to respond, with delays causing frustration and faltering confidence on the part of the user; they may be quick to assume they have ‘done something wrong’. Some touch-screen interfaces tend to suffer similarly in terms of responsiveness, even though the aesthetics of their presentation might indicate that they are more novel and user-centric. Designers of multi-touch interfaces must consider the fact that any appeal associated with the ‘newness’ or innovation of certain technologies can work as either a help or hindrance when it comes


to prompting participation. For some visitors innovative technologies might be unfamiliar and daunting, for others, those same devices might seem already outmoded. One notable finding from observing an additional group of University students using touch-screen interfaces was that they often expected them to respond in the manner of smartphones or personal tablet devices to which they are now increasingly accustomed, and often found them to be lacking in responsiveness and usability. In conversations during the visit they made ready comparisons to the high resolution retina display and responsive functionality of these other platforms that they are increasingly using on a daily basis. We see how quickly museum visitors (or more likely a portion of museum visitors) use of media outstrips the pace at which the institution can reasonably be expected to update. This problem is unlikely to diminish in coming years unless more is done to embrace the technologies that visitors increasingly come to museums with in their pockets (smartphones, apps and QR codes are increasingly being experimented with, for example at the Museum of London). In terms of positionality, there tended to be far greater flow around multi-touch interfaces than computer interactives. This was due to their design, but also to their positioning within exhibition spaces. They tended to be more integrated into the narrative of the exhibition, and simultaneously were often presented as star attractions due to their novelty and size. Thus, a significantly higher proportion of people who entered the gallery spaces tended to use them than used computer interactives. As Gammon and Burch have said, a device ‘needs to dovetail with the activity of museum visiting’ beyond that, for ultimate use-value, it should be ‘available as soon as it is required and [be] unobtrusive when it is not needed’ (Gammon and Burch, 2008:42). Ensuring a suitable balance between the site and any devices therein should also help to alleviate any concerns that a shift towards creative digital content might facilitate a ‘death of the object’ (Parry, 2007). Delicate balancing and considered exhibition design can create narrative flow. Commitment to multi-touch interfaces did seem to be slightly more varied than that to computer interactives, again this is something that needs more exploration. This is perhaps an inevitable outcome of the increase in social interaction associated with them (and less often immersive personal long-term engagement). Multi-touch and tangible interfaces are designed to be inherently playful, imaginative and immediate. In fact, researchers in Human Computer Interaction Design (HCID) have recently added the goals of enjoyment, emotional engagement, ambiguity and ludic design (Wakkary, Hatala 2006) to their analysis of principles for interaction (alongside usefulness and usability). An understanding of both ergonomic aspects and cognitive psychology developments is perhaps then a desirable basis for designing computer interactives, and unpacking their use in museum spaces. An understanding of the frames of interaction – how they are constituted and understood by visitors also remains crucial because, as Dodsworth notes; ‘Only the very best interfaces can teach you how to use themselves. That is where our youngest devices, computers, fail most miserably – because they are also our most powerful and enable us to make mistakes most efficiently’ (1999). The narrative of the interactive itself is a big part of this. How does the story/game/information unfold? Research shows that users tend to spend longer at interactives when they understand their role within a narrative, and when that narrative has a clear point of completion (Ramsay, 1998). Likewise, some navigational metaphors can serve to frustrate and confuse users (ibid). Simple yet intuitive operations are the gold standard of computer interactivity. This is something that should be a key focus in follow-up research, especially for museums interested in learning. Some authors have maintained that many current models of operation used in museums do not adequately acknowledge that learning may depend on previous knowledge and different sociocultural relations to the museums activities (e.g. Hein, 1998; Hooper-Greenhill, 1994). Nonetheless, emphasis has been given to multiple learning modalities in recent years, creating opportunities at least to access information through different mechanisms and modes of address. The multi-touch interactive is one such mechanism, but to evaluate how people make sense of the information they find through such means, involves further understanding the types of activity they facilitate; are people browsing, searching, exploring or playing (i.e. is their use purposeful, exploratory of playful, Gammon, 1999). This brings us to a discussion of the methodological implications of their study.

2.3 Methodological implications Of course, all kinds of questions emerge from such encounters which we cannot hope to answer through observation alone: Was the nature of the encounter meaningful to the individual or group? Would the


institution consider it to be meaningful? Was meaning making being demonstrated? Is it clear what usevalue the group anticipated in using the interface? Were the group expressing frustration about aspects of the user experience?8 Such questions require us to look beyond the quantitative and measurable aspects of experience, for example the route of enquiry through the interface’s various offerings, or the length of time spent at the interface, toward the qualitative nature of the groups’ exchanges, such as what was actually said, or how the group manoeuvred around and through the space. Some analysis and research has been carried out into such interfaces (Hornecker, 2008) but mainly as lab based user studies aiming to investigate interaction techniques9, and there have been even fewer studies devoted to multi-touch interfaces as they are defined in this paper (see the ShareIT project as an example10). Such studies often use summative evaluation techniques (quantitative and qualitative); mostly in the use of observations and interviews. However, the researchers’ findings are focused primarily on design aspects, perhaps a result of the principally evaluative nature of much of this study (and not a commitment to a wider research agenda). Many such studies collect and analyse data about usability, that is, user performance, yet disregard the knottier issue of user experience, that is, user satisfaction, which is a more resource intensive mode of study dealing in human emotion (Bevan, 2009). The findings from prior studies are vital as they begin to provide an understanding of and appreciation for participatory design and usability (the Design for All principle11), but do less well in helping us understand issues of user experience. It is thus vital to constitute a research approach that understands and positions visitors as protagonists within complex museum narratives which can be understood in more holistic terms. Thus, we would like to propose the pursuit of an inter-disciplinary method where computer sciences and humanities approaches including museum practices collaborate on field studies. Such an approach might reveal narratives about the visitors’ use of and engagement with interfaces as well as an analysis of design usability and functionality (through qualitative approaches such as interviews) coupled with an additional quantitative narrative of the museum visit (provided by sensing technologies such as RFID and other measures). It is important that visitors, beyond making sense of an interactive as a stand-alone interpretive tool, are able to understand that encounter within the overall framework of the museum and their museum visit. Current research and evaluation stops short of providing a means of understanding or articulating how visitors value their interactions with multi-touch interfaces within the larger narratives constructed by institutions, or within the various social encounters they have around them. Observations have revealed that multi-touch interfaces may be challenging some of the findings about computer interactives that have emerged in the past 20 years but have raised numerous other questions about the nature of experience and interaction also. Understanding the contribution of multi-touch interfaces to the museum experience will be crucial in justifying the expenditure that is increasingly being earmarked for them, especially in the face of so many competing technologies and devices. As Falk and Dierking remind us below, it is the relationship between ‘experience’ and ‘value’ (in learning, but also inevitably financial, terms) that museums have the most invested in: It is not enough that new technologies enhance the visitor experience; it needs to be demonstrated that these new technologies enhance the visitor experience better than competing technologies and in ways that are cost-effective. It is not until the field has a strong research base that it will truly be able to both optimize the power of these digital media tools and substantiate their value. (Falk and Dierking, 2008: 28)


Or perhaps, as was noted, these kinds of interfaces become natural gathering sites for parties to re-group before continuing their journey through the museum. 9 Such as the SHAPE project ( and the MUMMI project which are looking into the development and understanding of mixed reality interfaces, including design issues and accessibility matters. Other studies include work on the ‘Tree of Life’ interface 9 in the Berlin museum (Hornecker 2008), work on the exhibition ‘The Fire and the Mountain’ in the Civic Museum of Como (Italy) and an evaluation report on visitors responses on the interactives in V&A museum (McIntyre, 2003). 10 The interdisciplinary ShareIT project investigates the benefits of new shareable technologies by the Open University’s Department of Computing which collaborate with the Psychology Department at the University of Sussex from October 2008. 11 Design for All (DfA) is design for human diversity, social inclusion and equality (EIDD Stockholm Declaration, 2004).


References Abell K. S., Lederman G.N. (2007). Handbook of research on science education. Lawrence Erlbauem Associates Inc., New Jersey, USA Allen S., Gutwill J. (2010). Designing With Multiple Interactives: Five Common Pitfalls. Curator: The Museum Journal, Volume 47, Issue 2 Baldwin, Timothy and Kuriakose, Lejoe Thomas, 2009, ‘Cheap Accurate RFID Tracking of Museum Visitors for Persnalized Content Delivery’ at Bevan N. (2009). What is the difference between the purpose of usability and user experience evaluation method?. Professional Usability Services, London, UK. Chan, Seb, (2009), various posts including on work at the Powerhouse Musuem Dodsworth, Clark, 1999, ‘The Evolution of Tools…and the Devolution of the Users’ in The Informal Learning Review available at Falk, John H., and Dierking, Lynn D., 2008, ‘Enhancing Visitor Interaction and Learning with Mobile Technologies’ in Tallon, Loic nad Walker, Kevin (eds), 2008, Digital Technologies and the Museum Experience: Handheld guides and other media. Lanham, New York, Toronto, Plymouth: Alta Mira Press: Lanham, New York, Toronto, Plymouth. Fleck, Margaret, Frid, Marcos, Kindberg, Tim, Rajani, Rakhi, O-Brien-Strain, EAmonn nad Spasojevic, Mirjana, 2002, ‘From Informing to Remembering: Deploying a Ubiquitous System in an Interactive Science Museum’ available at Flowers, Alex, 2010, ‘Integrating QR codes




Gammon, Ben and Burch, Alexandra, 2008, ‘Designing Mobile Digital Experiences’ in Tallon, Loic and Walker, Kevin (eds), 2008, Digital Technologies and the Museum Experience: Handheld guides and other media. Lanham, New York, Toronto, Plymouth: Alta Mira Press: Lanham, New York, Toronto, Plymouth. Gammon, Ben, 1999, ‘Visitors’ Use of Computer Exhibits: Findings from 5 Grueling Years of Watching Visitors Getting it Wrong’ in The Informal Learning Review available at Gammon, Ben, 2008, ‘Everything we Currently Know About Making Visitor-Friendly Mechanical Exhibits’ in The Informal Learning Review available at Hatala, M., Wakkary, R., Ontologies-based user modeling in an augmented reality system for museums (2005), User-Modeling and User Adaptation Interaction, The Journal of Personalization Research, Vol 15, Issue 3-4, pp. 339-380 Hornecker E.(2008). Tangible Interaction - An Inclusive Perspective. form+zweck 22 (vol 40). English pages 78-85 ( Hornecker E. (2008), "I don't understand it either, but it is cool" Visitor Interactions with a MultiTouch Table in a Museum. Proc. of IEEE Tabletop 2008. 121-128 Hsi, S. (2003), ‘A study of user experiences mediated by nomadic web content in a museum’ in Journal of Computer Assisted Learning, 19, 308-319


McIntyre H.M (2003). ‘Engaging or Distracting? Visitor responses to interactives in the V&A British Galleries’,V&A British Galleries Summative Evaluation, UK Naumann B.A., Schleicher R., Wenchsung I. Views on Usability and User Experience: from Theory and Practice. Deutsche Telekom Laboratories, TU Berlin. Parry, Ross (2007), Re-Coding the Museum: Digital Heritage and the Technoloigies of Change. Routledge. Ramsay, Grahame, 1998, ‘Investigating ‘interactives’ at the Powerhouse Museum: Personal, social and Physical context’ available at Rizzo F., Garzotto F. (2007), ‘‘The Fire and the Mountain’: Tangible and Social Interaction in a Museum Exhibition for Children’, Department of Electronics and Information, Politecnico of Milano. Presented in IDC 2007 Proceedings: Tangible Interaction, Aalborg, Denmark. Sawyer, S. (2005), ‘Social Informatics: Overview, Principles and Opportunities’ in Bulletin of the American Society for Information Science and Technology. June/July 2005 pp. 9-12. Simon, N. (2010), The Participatory Museum. Santa Cruz: Museum 2.0. Tallon, Loic and Walker, Kevin (eds) (2008), Digital Technologies and the Museum Experience: Handheld guides and other media. Lanham, New York, Toronto, Plymouth: Alta Mira Press: Lanham, New York, Toronto, Plymouth. vom Lehn, Dirk and Heath, Christian (2005), ‘Accounting for New Technology in Museum Exhibitions’ in International Journal of Arts Management, VOl.7(3): pp.11-21 vom Lehn, Dirk and Heath, Christian (2003), ‘Displacing the Object: Mobile Technolgoies and Interpretive Resources’ paper from ICHIM 2003, Paris. Archives and Museum Informatics Europe. Woodruff, Allison, Aoki, Paul M., Hurst, Amy and Szymanski, Margaret H. (2001), ‘Electronic Guidebooks and Visitor Attention’ paper from ICHIM 2001 available at


Digital Natives: Creating Emergent Exhibitions through Digital Technologies Rachel Charlotte Smith, Ole Sejer Iversen & Christian Dindler Center for Digital Urban Living & CAVI Department of Information and Media Studies University of Aarhus, Denmark Imvrcs, imvoi, [email protected] Abstract Digital Technology can support the creation of dialogical spaces in the museum, both playful and reflective, that allow audiences to engage in the ongoing construction and reproduction of cultural heritage creating novel connections between self and others and between past, present and future. In this way, digital technology can contribute to the creation of emergent exhibitions in which the exhibition is created in dialogue between audiences and the museum. We present experiences from a current research project, the Digital Natives exhibition, in which digital technology was designed as an integral part of the exhibition to encourage dialogue between audiences and the exhibition materials and thereby investigate how the exhibition emerge as a result of this dialogic coconstruction inside the exhibition space. In short, the opportunities offered by digital technologies prompts us to consider the potential for designing emergent exhibition spaces.

Fig 1. Audiences exploring Digital Natives

Fig 2. Digital exhibition materials

1 Introduction Museums have traditionally been seen as formal places for heritage preservation and display of authentic objects that connect us to history. They provide the public with authoritative historical and cultural knowledge and act as civic educational spaces of reflection about the past, made meaningful in the present (McDonald 2003, Bennett 2004). While these cultural institutions have acted as important ‘bearers’ of heritage and identity a general problem of much heritage communication and research is that they ignore the dialogical aspects of people’s social practices that happen inside, and indeed beyond, these institutions (Handler & Gable 1997). In our approach, the common end of the museum is to transform our vision or to unveil novel perspectives. Embedded in this mission is a fundamental


challenge of bringing together, blending, and exploring what arises in the intersection between audiences’ engagement in the museum exhibition and what we value and construct as heritage. We see this challenge as one of exploring the emergence of exhibition spaces; a challenge that suggest a shift in perspective from designing the museum as a curated and highly scripted space for reflection to one of designing the museum as a emergent space constructed by the connections between audiences’ engaged in the exhibition and the museum. These connections between heritage and audiences engagement are neither given nor stable, rather they can emerge through dialogue and interaction in the museum. We suggest that common digital platforms for user generated content and protocols of networked interaction can contribute to co-created emergent exhibition spaces, reinforcing the museum as a place for ongoing reflections on the past and novel understanding of the present and future. We present experiences from a current research project, the Digital Natives exhibition, in which digital technology was designed as an integral part of the exhibition to facilitate the co-construction of the exhibition as a dialogue between audiences engaging in the exhibition and the available resources and materials. The experiences from Digital Natives provide new insight into the qualities of digital technology for heritage dialogue in museum spaces. Moreover, they challenge our conceptions of the museum institution as one that emerges in the dialogue between audiences and exhibition resources. This will encourage us to rethink understandings and constructions of cultural heritage through the use of new technologies and paradigms of communication.

2 Technology Supported Heritage Communication Relating to issue of interactive technologies in exhibition spaces, numerous studies within the area of Computer Supported Cooperative Work (CSCW) have already addressed the role of technologies in shaping the museum visit, exploring such issues as visitor participation (Heath & Lehn 2008), learning (Pierroux et al. 2007), and social interaction (Heath et al. 2005). These contributions reflect the wider concern for providing new ways for visitors to engage with exhibition spaces. As proposed within the New Museology, and exemplified by Hooper-Greenhill (2001), part of this challenge is to create exhibition spaces based on dialogue that frame the visitors as resourceful individuals and groups that can be invited to participate actively in the museum. A number of contributions have explored the potential of ubiquitous technologies in exhibition spaces, in the form of augmented reality (Woods et al. 2004, Wojciechowski et al. 2004), context aware museum guides (see Raptis et al. (2005) for overview), and various forms of mixed reality that blend physical and digital material inside the exhibition space (Sparacino 2004, Ferris et al. 2004, Hall et al. 2002) and outside the museum (Dähne et al. 2002). Recently, researchers have explored the potential of social technology frameworks for supporting living heritage through e.g. collective memories (Taylor & Cheverest 2009), and storytelling (Leder et. al.), crisis related grass-root heritage (Lui forthcoming) and co-creation and shared experiences of living heritage sites (Giaccardi & Palen 2008). The research in social media platforms to support heritage matters, however, has only to some extend been explored within what Giaccardi (forthcoming) denotes as the official heritage practices, i.e. heritage representations in museums. In parallel, a large of body of research on new media in museums is concerned with identifying how museums can use social technologies as new virtual and distributed platforms of communications, connecting and engaging with audiences outside these institutions, thereby extending the museum space (e.g. Russo et al. 2008, Arvantis 2010, Galani & Chalmers 2010, Deshpande et al. 2007). We acknowledge these characteristics of social platforms and technology for extending the museum space, but are more focused on exploring how digital technologies and platforms inside the museum space can create this connectedness between audiences’ practices and heritage matters through exhibition spaces in the museum. By adding a physical location to the usage of digital technologies, Ciolfi et al. (2007) studies the facilitation of visitors’ own contributions to an interactive exhibition at the Hunt Museum focusing on a selection of the museums’ objects. The aim is to create opportunities for visitors’ contributions to become part in shaping and creating content and messages in the exhibition in order to create deeper participation and reflection from the audience. Following Ciolfi’s line of argument, we perceive the central challenge of new media technologies in museums, not merely as modes of creating audience engagement in already curated messages through new platforms of distributed communication. Rather, our focus is the extent to which these digital technologies can vitalize and renew the role of the museum as a hub for reflecting and constructing cultural pasts, presents and futures, through connecting peoples everyday lives and experiences to concrete exhibition


spaces. In the following, we expand our perspectives on the potential role of digital technologies in the museum space.

3 Creating Emergent Museum Exhibitions In particular we investigate digital technologies and media that support individual and social interaction and user-generated content. These technologies are by definition participatory and dialogical forms of communication and engagement that create social interaction and co-created cultural meanings. They have the capacity to articulate between fragmented real and digital environments, developing novel hybrids and forms of cultural communication. Both interactive technologies and social media, can play an essential role in transforming the museum into a place of dialogue and interaction in novel ways, and strengthen the potentials for museums to act as important cultural connectors of time and space (Castells 2010). As Giaccardi (2011) argues, social media are having a profound impact on heritage matters and communication in terms of social practice, public formation and sense of place, creating new opportunities for people to experience and engage with both historical matters and emerging practices of heritage. Moreover, networked and dialogic forms of communication are social practices that already exist as part of peoples everyday lives and experiences and thus can be actively used as a means or model of communication and interaction emphasizing engagement and dialogue. This can create new relationships between museums and audiences that break down the formal dichotomies between official and living heritage, inside and outside the museum. In this understanding of the exhibition, meanings and reflections emerge from dialogues between audiences and exhibition materials, as well as between audiences themselves, rather than a priori as a result of curated or messages, texts and meanings. The intersections between heritage matters and the audience no longer rely on linear models of communication about an objectified past, but rather on ongoing reflection, negotiation and participation in the present. Each exhibition becomes a framework and vehicle for creating and transforming understandings and experiences of ongoing cultural processes as they emerge in situated meetings and relations constructed in and through emergent exhibition spaces. As such, the museum exhibition, rather than a formal place of education and distanced contemplation where culture is located in authentic objects, texts and materials, becomes an emergent dialogic space for co-created meanings and experiences (Low & Lawrence-Zuniga 2003). These hybrid arenas are characterized, not by their stability or perseverance in time and space, but by their explorative, flexible and transitional nature, extending time and space, creating dialogic relations and connections between heritage matters and audiences’ everyday social practices. Focusing on these emergent spaces allows for a strengthened focus on the role of the audience; both their situated engagement with and contribution to experiences in the exhibition space – but also to their subjective role in defining and co-creating understandings and conceptions of cultural heritage. This extended and constructivist role of the audience oscillate and revitalize connections between formal and living heritage and underline the dialogical nature of cultural processes, or what Bruner (1994) argues, that culture everywhere and always is an invention. In this way active audience engagement is not the end goal in itself, but merely a means in the process of connecting between audiences’ everyday practices and heritage matters. Digital technologies and media can create this connection between audiences and heritage matters inside the museum space, by reinforcing the everyday practices and experiences that people bring to the exhibition space. In their capacity of involving people in situated, dialogic micro-acts of communication and reflection, interactive and social media can contribute to enriching the qualitative and dynamic relations between audiences and cultural heritage matters through the exhibition space. In this sense the overall challenge is not so much, how to use social or interactive technologies for museums, but rather, to rethink the language, design and conception of cultural heritage communication in relation to digital media technologies and contemporary digital cultures. In the following we will describe the Digital Natives project and how we used digital technology as a means of creating emergent exhibition spaces from dialogues between audiences and heritage resources.


Fig 3. The Digital Natives exhibition room

Fig 4. Interactive floor projection

4 The Digital Natives Exhibition Digital Natives is a research and exhibition experiment exploring the intersections of cultural heritage, participatory design and new interactive technologies. The project experimented with possible new futures and innovations of cultural heritage communication and involved creative collaboration between a group of young people, anthropologists, architects and interaction designers through an extended period of nine months. The project focused on a contemporary generation of young people raised in a digital era, surrounded by new media and information technologies, and whose life worlds are said to depart from that of previous generations, both mentally, socially and culturally (Prensky 2001, Ito 2009). The exhibition explored these young people’s everyday cultures, identities and communication practices and experimented with new ways of representing and interacting with these cultures in the context of a concrete museum exhibition. As such the aim of Digital Natives was to create an exhibition in collaboration with a group of young people that explored and expressed the lives and cultures of these so-called natives in a local setting. The project was explorative in nature, actively interweaving understandings and boundaries between cultural heritage, contemporary digital cultures and new media technologies through the design process as well as the final exhibition. Focusing on issues of participation and interaction the aim was to create new modes of communication and engagement that would create emergent dialogical spaces and novel connections between museum space, exhibition and audiences. Digital Natives was held at Aarhus Center for Contemporary Art in December 2010. Five interactive installations were created for the exhibition that focused on the everyday lives and social practices of the seven young ‘natives’ involved in the project in various ways. All installations had a strong focus on social media and interaction design, crossing the borders between culture, art and technology. Here, we will focus specifically on two installations: Google My Head and DJ Station, in order to illustrate how we used digital technologies and media to create novel experiences and dialogic spaces through the exhibition.


4.1 ‘Google my Head’

Fig 5. The Google My Head installation Google my Head is an interactive tabletop installation running on a PC connected to a 72” Evoluce One LCD multi-touch display. In the Google My Head installation, audiences were encouraged to browse in a repository of Digital Natives online and mobile – Facebook, SMS, mobile – updates, pictures and videos continuously posted on the multi-touch screen drawn from a ‘Redia Gallery’ database. At the installation, visitors were confronted with the task of completing the sentence “Digital Natives are: ……” While browsing through the digital traces from various social media, they could choose up to four utterances, pictures or videos that caught their interest and supported their completion of the sentence. The chosen samples were stored in a docking placed at each narrow end of the table. When clicking on a small keyboard icon on the dock, an onscreen keyboard would occur allowing the audience to complete the sentence with statements such as “Digital Natives are CREATIVE”, Digital Natives are “EGOCENTRIC AND SPOILED” or Digital Natives are “NO DIFFERRENT THAN OTHERS” The statement made by audiences were stored in a database and displayed as a part of the Digital Natives exhibition on two 22” touch screens located close to the installation. Here, visitors could see the utterances made by them self and others and explore which kind of digital material had been assigned to support the statement. Moreover, visitors could respond to the utterances by pushing a “like” or “dislike” button on the touch screens, adding their score to the total number of likes and dislikes for each statement. Google My Head illustrates and represents the vast amounts of fragmented information and communication that exist in the lives of digital natives. The selection of materials was made by the young natives themselves, as were the visible tags combining these materials. The audience themselves were motivated to browse and select in the materials, from their own interests, in order to create new connections and statements about Digital Natives. As such, using the form and language of social and digital media, the audience was invited to both explore the characteristics and everyday cultures of Digital Natives and to discuss and contribute to the overall statement about these made through the exhibition.


4.2 ‘DJ Station’

Fig 6. Visitors engaging with DJ Station DJ Station is an interactive and audiovisual installation based on a tangible user interface with fiducial tracking. The DJ Station allowed the audience to interact with the musical universe of the seven digital natives involved in the project, while getting first hand experience with the remix and mash-up cultures that are hallmarks of the digital native generation. Each young native was represented in the installation by a cube with visible fiduciary markers, which played musical loops when placed on the table surface. Each cube represented one person’s musical taste, and each side of the cube contained a unique loop co-produced with the person in question. Flipping the cube to a new side played a new loop, while rotating the cube controlled the volume of the loop. A range of coloured cubes contained unique audio effects that could be applied to the musical loops. Rotating a coloured cube controlled its effect parameters, e.g. the room size of a reverb or the amount of feedback of a delay effect. By placing more musical cubes on the table and applying effects to them dynamically, the user could combine and alter loops and create complex mash-ups. In addition, visual images representing each of the digital natives gathered around the respective musical cubes on the table surface and interacted with images from the other cubes. Eight headphones were attached to the table, all connected to the same audio output. However the table was often used with loudspeakers, making its sounds an integral part of the exhibition space. The tracks created by the audience were streamed live on the exhibitions website. Inspired by the Reactable, DJ Station uses reacTIVision for fiducial tracking, Ableton Live for audio processing and Unity for visuals. DJ Station created an audiovisual universe for exploring and interacting with the young natives through their musical taste and landscape. The language built into the installation was based directly on the remix and mash-up cultures of social media, while focusing on the profound importance of music for these digital generations. They continuously stream, modify and reproduce music through online services such as YouTube, iTunes, MySpace and music producing software. Simultaneously, the installation invited the audience to take part in this ongoing cultural production and reproduction of music by actively engaging with the cubes, mixing and remixing unique tracks from the existing loops and materials. Each visitor became his/her own DJ, creating music both singlehanded and socially, engaging with other visitors.

5 Experiences from the Digital Natives Exhibition Both Google My Head and DJ Station invited audiences to explore and interact through both individual experiences and social engagement. In Google My Head, people browsed the large amounts of digital materials by playing with the multi-touch function of the table, becoming acquainted with the opportunities created through the interface. Visitors selected materials according to their own personal interests, and gradually became more focused in their search when prompted with the question ‘Digital natives are …’. Some enjoyed the personal Facebook statements while others were clearly drawn to the


visual images representing the young people. The selections of digital materials represented everyday events such as situations from the classroom, humorous reflections about food, friends or personal interests, statements about use of technologies, or personal experiences and social events from music festivals, to demonstrations and even family funerals. The selection process forged a kind of in situ curation, where audiences created their own micro-stories about the exhibition subject. Whether they commented on one particular image from a shooting sequence in a computer game, a text message concerning a ‘natives’ fascination for Japanese manga, or gathered and saw several materials as reflecting characteristics of a whole digital generation, was entirely up to them. However, being prompted to make a statement from the specific authentic materials, made them reflect upon and discuss the subjects’ as well as their own relation to digital nativeness in new ways. Once the statements were sent to the two screens, they were made visible to other groups of audiences who could respond to their argument. In this manner the communication and dialogue was instantaneous and collective, creating social engagement between the participants at the installation, while discussing what characterized the digital nativeness of the exhibition subjects and how to capture these in brief comments. The situated concern of the audience was no longer the museums authored or curated story to them as audiences, but rather visitors’ own micro-stories, contributions and reflections about the exhibition subject and their relation to it. Through a series of actions, selections, reflections and communicative acts audiences related the specific content and issues of the exhibited materials to their personal understandings and everyday lives, before contributing these reflections to the collective statement of the exhibition. In this way Google My Head created modes of engagement and interaction that encouraged dialogue and participation both between individual visitors and potentially much larger and unknown audiences. Using and mimicking the language of social media the installation created hybrid and living connections between the exhibition subject(s) and the audiences’ everyday lives.

Fig 7. Collectively exploring Google My Head

Fig 8. Audiences engaging with DJ Station

Whereas Google My Head was reflective and intellectual in focus, DJ Station was playful and creative in its expression. People explored the DJ Station by placing different cubes on the table, turning them, adjusting the distance between them, and gradually making sense of the various functions combining the musical loops and the visual universe of the installation. Creating their own music, visitors playfully mixed, sampled and explored various unique strands and styles of music. The loops on each cube, contained musical styles from techno and ambient beats, to rock, heavy metal and acoustic sounds. This variation allowed the audience to reflect upon contemporary musical trends, to choose and


navigate in the materials according to their own taste, as well as experiment with combinations and mash-ups of otherwise disparate musical genres. Many audiences spent extensive time at the installation, often more than 30 minutes, creating music in small groups of two-three people, but also often engaging with up to six or eight people at a time. As the functions and possibilities of the installation were discovered, they were shared among fellow visitors, or passed on their knowledge to new arriving guests, creating both dialogue and transfer of personal experiences between various groups of audiences. For school classes who already knew each other, the table set about a sense of trial and error teamwork where discoveries where made operating the cubes in turn, and manifest more through gestures, facial expressions and small comments, than verbal discussions. With other individual or pairs of visitors, the presence of co-audiences or the young natives from the project was an assisting resource to verbal explanation, and thereby deeper understanding and engagement with the table. The audience enjoyed the explorative and creative freedom that the DJ Station afforded them, and the feeling of creating something both unique and cool. In this way, the installation allowed the audience to gain insight into specific young peoples contemporary musical life, as well as an opportunity to experience and become part of the social, cultural and communicative processes surrounding these universes. Through both Google My Head and DJ Station, the audience connected with the cultural universe of a specific group of people by using their own physical, social and emotive engagement. In each case the underlying language of interactive and social media was the key to creating the connections and dialogical spaces between the museum space and the everyday lives of particular audiences. These intertwined and emerging spaces for co-creation clearly allowed the audience to gain an important role in the exhibition, revealing new personal perspectives, stories and voices through physical gestures, individual choices and social engagement. Formal or authoritative knowledge about the exhibition subjects were never constructed though the exhibition design. Rather, fragments, possible connections, and arbitrary meanings were ingrained in the actual installations and the representations of materials; – digital materials that the exhibition subjects themselves, had a decisive role in framing, selecting or coproducing. In this way each installation demanded the involvement and reflections of the audience in order to create meaningful experiences; experiences created precisely in a meeting between matters of cultural heritage, the language and design of the installation and the active involvement of the audience. Many visitors emphasised the interactive characteristics and possibility of control, investment and engagement that the installations afforded them as a main and positive experience of the exhibition. They felt invited and included as subjective individuals, rather than dispensable objects, in the exhibition. Moreover, they were prompted to be both individually and socially engaged with the installations as well as their co-visitors while exploring the exhibition space and cultural matters. The fact that the exhibition was not based on material heritage objects but ‘digital cultural objects’ influenced the experience and interaction with them, and allowed visitors to bring their own interests and concerns to the interaction, making them playful and improvisational. They could relate themselves to the fragmented everyday stories, pictures from the Internet, portraits of band covers, travel photos, musical tracks, etc. even if they did not share the same references. Thus, the materials in their own disintegrated manner had a representational value of a specific group of people and their everyday practices, both in cultural content and language, that made it easy for audiences to grip on to, allowing audiences to ‘take charge’ of their situated experiences and, more generally, the way they consumed and produced culture (Bruner 1994). Google My Head and DJ Station illustrate two ends of a continuum, between reflection and playful engagement, where the use of digital technologies and social media connects museum and audiences through a transformed means of communication. Rather than merely opening space for participation, audiences themselves played a vital role in the explorative and dialogic aim of the Digital Natives exhibition. The exhibition simply was unfinalised without their active contributions, connecting cultural heritage issues with their own contemporary practices and understandings in the ongoing cultural consumption and productions of past, present and future.

6 Re-thinking Museums Through Digital Technology As discussed previously, the study of new digital technologies in cultural heritage communication is already a well-established research field. However, the experiences from the Digital Natives exhibition emphasize the importance of more thorough studies of digital technology ability to connect the


everyday practices of audiences to dialogical, reflective and emergent museum and exhibition spaces. From a research perspective, the Digital Native experiment brings about at least three major concerns that are in need of further investigation. First, how can we design co-located digital technologies emphasizing the dialogue between audiences’ everyday practices and contemporary concerns of heritage in the museum? The shift in perspective from communicating authored narratives of heritage, to designing frameworks and media platforms for collective reflection and dialogue initiates novel ways of including emerging cultural issues and fragmented storytelling into ongoing experiences and constructions of cultural heritage. We suggest that this connection is promoted by the dialogical, situated and interactive qualities of digital technologies speaking with and between, rather than to, the audience. This means a reconceptualization of the audience, that allow them to take charge and become subjects in and of their own experiences and understandings (see Iversen & Smith forthcoming). However, we need to conduct more experiments and thorough studies of such technologies applied in museum contexts to understand how, why and when these experiences of engagement and connectedness occur. Secondly, how can digital technologies help transform objects and sites of heritage to cultural matters and materials? The digitalization of culture underlines a constructivist merger between official and living heritage. In focus are no longer frozen cultural and historical objects, but means of dialogue and reflection for discussing the past through the present. Cultural matters are re-conceived as living and processual and merged into novel co-created forms of communication and interaction in the intersection between interaction design, audiences’ response and heritage. Our experiences from the Digital Natives exhibition suggest that the digitalization of cultural materials combined with interactive means of engagement can open up for radically new ways of experiencing and reflecting upon heritage matters. But we need more knowledge about the paradigms of communication and representation that can support meaningful repertoires and transient moments of cultural construction. Thirdly, how can digital technologies transform the museum space from a curated, highly facilitated and predictable experience to an emergent exhibition space? In the Digital Natives exhibition, usergenerated content was not only a transformation of the heritage communication to digital media platforms and more interactive forms of engagement. It was more importantly integrated as an essential dialogical and explorative goal of the project, process and exhibition itself. When merging usergenerated content and heritage subjects through forms of digital technologies in the museum, we need to rethink the relationship between the ‘democratization’ of the museum space and the process through which we curate and design new exhibitions. Our experiences suggest that a further study of the relationship between digital and interactive technologies, contemporary digital cultures and the museum as a physical space can help us design technologically mediated and emergent exhibition spaces that encourage the audiences to contribute to the collective reflection and experience that extends in time and space, inside, as well as outside, the museum exhibition.

7 Conclusion The Digital Natives project has provided us with valuable insights into the potential role of digital technologies in museums. The language and nature of social technologies transform communication to interaction. In McLuhan’s conception, the media becomes the message. But digital technologies are resourceful in the sense that it can support diverse individual and social experiences, both playful and reflective, and forge dialogic participation and engagement in the museum space. As such, using interactive technologies and social media in exhibitions can create more inclusive and non-hierarchical spaces for experiences and expressions of cultural communication that prompts both curators and audiences to constantly challenge constructions and conceptions of cultural heritage, the role of the museum institution and its connection to peoples everyday lives. Digital technologies forge both individual experiences and collective social action, in the ongoing construction, reproduction and distribution of cultural heritage meanings. It can incorporate and distribute both heritage ‘content’ and ongoing modifications from audiences inside and beyond the museum space. Using social media in the design of exhibitions thus dissolves the boundaries between formal and living heritage and help reconnect museum spaces with audiences’ everyday practices through hybrid frameworks of emergent exhibitions. For the museum this means a turn away from paradigms of linear communication and knowledge production, to new dynamic ways of creating and connecting with the fragmented and intersecting social and cultural flows of meaning. In this sense, we 21

not only need to develop and rethink social technologies for the museum. We also need to rethink museums through digital technologies and cultures to align the physical exhibition spaces and the communication strategies with departure in people’s everyday practices and experiences in order to create coherent environments that encourage audiences to reflect, engage and co-create these emergent exhibition spaces with museums.

Acknowledgements This research has been funded by Center for Digital Urban Living (the Danish Council for Strategic Research, grant number 2128-07-0011). We acknowledge the collaborating partners of the Digital Natives project: Center for Advanced Visualisation and Interaction (CAVI), The Alexandra Institute, Moesgaard Museum and Innovation Lab; the young ‘Digital Natives’ involved in the realization of the project: Martin von Bach, Ida Emilie Sabro, Troels Peter Jessen, Metha Rais-Nordentoft, Johan Lindgaard Sundwall, Ane K. Flygaard and Lil Wachmann; the designers of the DJ Station installation: Kasper Aae, Janus Novak, Christoffer Krogsdal, Janne Vibsig, Rasmus Bendtsen, Lasse Damgaard and Sune Hede; as well as Aarhus Center for Contemporary Art for hosting the exhibition.

References Arvantis, K. (2010). “Museums Outside Walls: Mobile Phones and the Museum in the Everyday”. In Parry, R. (ed.), Museums in a Digital Age. London & New York: Routledge. Bakhtin, M. M. [1930s] (1981). The Dialogic Imagination: Four Essays. Holquist, M. (ed.). Austin and London: University of Texas Press. Bennett, T. (2004). Pasts Beyond Memory. Evolution, Museums, Colonialism. London: Routledge. Brown, A. & White, B. (2005), “An Investigation in Colour Perception”, in Pink, D, Grey, E. and Yellow, F. (eds), Proceedings of the Great Colour Conference, Limerick, June 2005. Bruner, E. (1994). “Abraham Lincoln as Authentic Reproduction: A Critique of Postmodernism.” American Anthropologist 96:397-415. Castells, M. (2004). The Rise of the Network Society. Oxford: Oxford University Press. Castells, E. (2010). “Museums in the Information Era: Cultural Connectors of Time and Space.” In Parry, R. (ed.), Museums in a Digital Age. London & New York: Routledge. Ciolfi, L., Bannon, L. J. and Fernström (2007). “Visitors Contributions as Cultural Heritage: Designing for Participation”, Proceedings of ICHIM 2007, Toronto (Canada), October 2007. Dähne, P., Karigiannis, J., & Archeoguide N. (2002). “System Architecture of a Mobile Outdoor Augmented Reality System. In Proceedings of the 1st International Symposium on Mixed and Augmented Reality. IEEE Computer Society, Washington D.C. Deshpande, S., K. Geber, C. Timpson (2009). “Engaged Dialogism in Virtual Space: An Exploration of Research Strategies for Virtual Museums.” In Cameron, F. & S. Kenderdine (eds.) Theorizing Digital Cultural Heritage. Cambrige, Massachusetts & London, England: The MIT Press. Ferris, K. Bannon, L. Ciolfi, L. Gallagher, P. Hall. T. Lennon, M. (2004). “Shaping Experiences in the Hunt Museum: a design case study”, Proceedings DIS2004, August 1–4, 2004, Cambridge, Massachusetts, USA. Galani, A. & Chalmers, M. (2010). “Empowering the Remote Visitor: Supporting Social Museum Experiences among Local and Remote Visitors”.” In Parry, R. (ed.), Museums in a Digital Age. London & New York: Routledge. Giaccardi, E. & Palen, L. (2008). “The Social Production of Heritage through Cross-Media Interaction: Making Place for Place-Making”. International Journal of Heritage Studies, 14: 3, 281-297. Giaccardi, E. (2011). “Things We Value”. Interactions, Vol. XVIII.1, 2011.


Giaccardi, E. (forthcoming). “!"#$%&'(#)%"*+ ,-.$/0)"1+ 2-$)#/1-+ )"+ /+ 3/$#)()4/#%$5+ 6'7#'$-!"# In Giaccardi, E. (ed.), Heritage Matters. Understanding and Experiencing Heritage Through Social Media. Routledge.! Hall, T., Ciolfi, L., Bannon, L., Fraser, M., Benford, S., Bowers, J.,Greenhalgh, C., Hellström, S. O., Izadi, S., Schnädelbach, H., Flintham, M. (2002). “The Visitor as Virtual Archeologist: Exploration in Mixed Reality Technology to Enhance Educational and Social Interaction in the Museum”. In proceedings of VAST'01, ACM, pp. 91-96. Handler, R. & Gable, E. (1997). The New History in an Old Museum. Durham: Duke University Press. Heath, C., Lehn, D. V. & Osborne, J. (2005). “Interaction and Interactives: Collaboration and Participation with Computer-based Exhibits.” Public Understanding of Science, 14 (1), pp. 91-101. Heath, C. & Lehn, D. V. (2008). “Configuring 'Interactivity': Enhancing Engagement in Science Centres and Museums. “ Social Studies of Science, 38 (1), pp. 63-91. Hooper-Greenhill, E. (2001). “Communication and Communities in the Post-Museum - From Metanarratives to Constructed Knowledge.” Paper delivered to the Nordic Museums Leadership Programme, Copenhagen, June 2001. Ito, M. (2009). Hanging Out, Messing Around, and Geeking Out: Kids Living and Learning with New Media. Cambridge, MA: MIT Press. Iversen, O. S. & Smith, R. C. (forthcoming). “Connecting Audiences and the Museum through Social Media. Experiences from the Digital Natives Experiment.” In Giaccardi, E. (ed.), Heritage Matters. Understanding and Experiencing Heritage Through Social Media. Routledge.! Kaplan, A. & Haenlein, M. (2010). "Users of the World, Unite! The Challenges and Opportunities of Social Media". Business Horizons 53 (1): 59–68. Kirschenblatt-Gimblett, B. (1998). “Destination Museum”, in Destination Culture. Tourism, Museums and Heritage. Berkeley, Los Angeles & London: University of California Press. Leder, K., Karpovich, A., Burke, M., Speed, C. Hudson-Smith, A., O'Callaghan, S. Simpson, M. Barthel, R., Blundell, B., De Jode, M., Lee, C., Manohar, A. Shingleton, D., & Macdonald, J. (2010). “Tagging is Connecting: Shared Object Memories as Channels for Sociocultural Cohesion,” M/C Journal of Media and Culture, vol. 13, 2010. Low, S. & Lawrence-Zuniga, D. (2003). “Locating Culture”, in Low, S. & Lawrence-Zuniga, D. (eds) (2003), The Anthropology of Space and Place. Maldon, MA: Blackwell Publishing. Lui, S. (forthcoming). “Socially Distributed Curation: Informing the Next Generation Social Media to Support Grassroots Heritage Practices.” In Giaccardi, E. (ed.) Heritage 2.0. Understanding and Experiencing Heritage Through Social Media. Routledge. MacDonald, S. (2003). ‘Museums, National, Postnational and Transcultural Identities”. Museums and Society, vol. 1(1): 1-16. Pierroux, P., Kaptelinin, V., Hall, T., Walker, K., Bannon, L., Stuedahl, D. (2007). “MUSTEL: Framing the Design of Technology-Enhanced Learning Activities for Museum Visitors”. Proceedings of ICHIM’07, Toronto. Prensky, M. (2001). “Digital Natives, Digital Immigrants.” In On the Horizon, MCB University Press, Vol. 9 No. 5, October 2001. Raptis, D., Tselios, N., Avouris, N. (2005). “Context-based Design of Mobile Applications for Museums: A Survey of Existing Practices”. Proceedings of Mobile HCI, ACM, Salzburg, Austria.


Russo, A., Watkins, J., Kelly, L., Chan, S. (2008). “Participatory Communication with Social Media”. Curator, The Museum Journal Volume 51, Issue 1, pp.21-32 Sparacino, F. (2004). “Scenographies of the Past and Museums of the Future: from the Wunderkammer to Body-driven Interactive Narrative Spaces. Proceedings of the 12th annual ACM international conference on Multimedia. ACM, New York (NY), pp. 72-79. Taylor, N. & Cheverest, K. (2009). Social Interaction around a Rural Community Photo Display. International Journal of Human-Computer Studies 67, 12:1037-1047. West, S. (ed.) (2010). Understanding Heritage in Practice. Manchester: Manchester University Press. Wojciechowski, R., Walczak, K. White, M. Cellary, W. (2004). “Building Virtual and Augmented Reality Museum Exhibitions.” Proceedings of international conference on 3D Web Technology, ACM Press, New York (NY), pp. 135-144. Woods, E., Billingshurst, M., Aldridge, G., Garrie, B., (2004). “Augmenting the Science Centre and Museum Experience.” Proceedings of International Conference on Computer Graphics and Interactive Techniques. Singapore, pp. 230-236.


The Role of Appropriation in the Design of Engaging Artefacts Tom Flint and Phil Turner Centre For Interaction Design Edinburgh Napier University Merchiston Campus Edinburgh EH10 5DT [email protected] [email protected] Abstract Creating engaging artefacts is a key objective for anyone involved in the design and implementation of interactive media. This is particularly true for those artefacts that comprise, complement and enliven modern museums. While recognising that engagement can take a number of different forms, appropriation appears to be the most pertinent here. By appropriation we mean people making an artefact their own, an observation we illustrate with details from an ethnographic study conducted at the Public ( We conclude by observing that by empowering people to make artefacts their own is not merely an effective means of creating engaging artefacts but lies at the heart of user-centred design.

Introduction With every passing year, people’s expectations of interactive media rise whether these are of smart phones, the latest shoot ’em up, tablet computers or the latest installation found in a museum, galley or spontaneous happening. The challenge for such public interactive artefacts is that the design needs to induce, entice, lure or beguile a prospective participant into wanting to interact with the designer’s creation. Artefacts then, must be designed to demonstrate that a small investment in a user’s time will be rewarded with a high level of engagement. This draw has been called fun (Blythe & Hassenzahl 2004), and hedonic stimulation (Hassenzahl, 2004). This focus on the hedonic is perhaps, the most recent challenge for creators of interactive products and is potentially a new paradigm for designers and users alike. Engagement is immediate and visceral (Norman, 2004); users can appraise the appeal of a product in less time that it takes to twinkle an eye – or approximately 50 ms (Lindgaard et al., 2006). Having engaged a potential use, we feel compelled to interact with it (Turner, 2010). This paper focuses on museum and gallery interactive installations and specifically on one interactive exhibit, namely Animo in The Public gallery West Bromwich. We will discuss appropriation and personalisation as important factors in sustained (as distinct from initial engagement) interaction. We will then consider details from an ethnographic study conducted at The Public, which illustrates the importance of appropriation in making an interactive exhibit engaging.

Appropriation Appropriation usually means finding alternate uses for technology beyond those for which it was originally designed – a definition that Alan Dix describes as oxymoronic (Dix, 2007). Yet the very fact that we seek alternate uses is evidence of our engagement with the technology per se. Appropriation is to re-purpose, to re-configure and to re-work but more importantly it is to look afresh, to disclose new possible uses, or affordances (Gibson, 1986, Norman, 1988) and re-appraise their usefulness to us. This re-working, or appropriation has found form in the growing interest in design for serendipity (e.g. Newman et al., 2002) and design for sustainability (e.g. Blevis, 2007); or the closely related interest in ensoulment. Further ‘do it yourself’ design (Blevis and Stolterman, 2007) and designing for hackability (Galloway et al., 2004) have found fresh momentum in recent years. Maclean and his colleagues have pointed out that there is a long tradition of interest in user-tailorable systems which can be dated back 25

to such systems as the Xerox interface (MacLean et al., 1990) where its users were able to re-position on-screen buttons.

Light fingers We can in turn locate this user-configuration within the even longer tradition of adopting, borrowing, stealing (as copyright holders would have it) and re-cycling in the arts. Music and voices are regularly sampled and re-produced in someone else’s musical piece. Picasso famously observed that ‘Bad artists copy. Great artists steal’ and by this reading he ‘stole’ from Delacroix and Velázquez specifically (Anglin Burgard, 1991), whereas Warhol’s (1968) famous “Campbell’s Soup” was prompted by his fondness for the soup and the appropriation of soup can labels. TV series from the 1970 and 1980s are constantly being re-imaged, re-invented together with ill-fitting prequels. Movies employ pastiche, homage, allusion and may simply poke fun. And, of course, there is karaoke. Many forms of art frequently rely on appropriated elements. Art appropriates ideas, images, sound and styles from any and all aspects of natural and artefactual worlds. In essence, appropriation involves the creation of a new disclosive space. However, our position is to propose that the appropriation of technology is a key expression of our engagement with it and occurs when ‘we make it our own’. This is, of course, an everyday occurrence – take a moment to consider just how much of the technology that we use was designed for us personally? The answer is almost certainly none, yet we happily talk about ‘my phone’ and we also (vehemently) distinguish between a work-issued PC, which we are happy for anyone to use, and our (work-issued) laptop that is mine and no one else’s. Appropriation is not only about ownership but it is also concerned with customisation and personalisation. These processes of ‘making technology our own’ can vary from something as trivial such as customising the appearance of the desktop of a computer or the use of mobile phone ‘skins’ through to full blown personalisation which some researchers have called ‘ensoulment’. Ensoulment is the radical personalisation of technology as a means of expressing one’s identity and has clear resonances with Ilyenkov’s discussion of the ideal (Ilyenkov, 1977) as it is concerned with endowing technology with meaning and, perhaps, artefactual affordance. We are also appropriating when we reuse and care for digital artefacts, such interest has been described as sustainability. A further expression of making technology our own is what Silverman and Haddon (1996) have called ‘domestication’ – a metaphor for the ‘taming’ of technology. Together, these different expressions of engagement offer a better reflection the original meaning of appropriation.

Making it our own Personalisation is the process by which an artefact is endowed with greater significance by and for its user. Wells (2000) defines it as “the deliberate decoration or modification of the environment” and associates it with ‘well-being’. Blom (2000) defines it as “a process that changes the functionality, interface, information content, or distinctiveness of a system to increase its personal relevance to an individual”. All of which means that personalisation can take any number of different forms. Blom and Monk (2003) have shown that personalising the appearance of an artefact has cognitive, social and emotional dimensions. Their account of personalisation was based on the analysis of three qualitative studies of the use of web-based personalisation in the UK; as well as Finnish mobile phone users and of the personalisation of home PCs, again in the UK. They describe the cognitive aspects of personalisation as improving ease of use, better recognition of the system and improved aesthetics; (though this might have reasonably have been treated as a category in its own right as the research was concerned with personalising the appearance of artefacts). The social dimensions of personalisation are concerned with reflecting personal and group identity. Finally, the largest category is the emotional effects on the user. These affective consequences include feelings of familiarity, ownership, control, fun, attachment, release from boredom and other positive attachments. In contrast to this push-or systems-driven personalisation there are (at least) two forms of the user changing the appearance or functionality of a digital artefact. These changes can be trivial, expensive, evolutionary or revolutionary.


In addition to personalising the appearance and aesthetics of our everyday technologies we can also add functionality. Default games controllers can be replaced with more realistic (though plastic) sniper rifles for use in combat games. While Guitar Hero is a series of music ‘games’ in which players use a guitar-shaped controller to simulate playing the real thing. Other personalised controllers include skateboards, steering wheels boxing gloves and light sabres. Personalisation is very clearly a source of pleasure and of fun. It is an expression of engagement with the artefact in its own right. Taiwanese manufacturer HTC took personalisation to a new level as they released a mobile phone, the HTC Tattoo, which was marketed around the ability to design your own cover for the phone free of charge ( Personalisation also increases the sense of ownership of the artefact and allows people to express the aspirational, the playful and the gadget-loving sides to their sense of self. The design consultancy BERG 12 has worked with Nokia – the Finnish telecoms giant - to develop prototypes as a means of exploring the personalisation in mobile phones. They report that the project comprised three overlapping strands, namely, craft, hacking and mass customisation13. They define craft as working with people who have specialist skills in the areas of making; hacking is combining the phone with other objects or ‘interfering with the phone functionality’; and they define ‘mass customisation’ as the means and materials available to local manufacturers for small scale production runs.

An Ethnographic Study in The Public The Public is a purpose built digital arts venue in West Bromwich, north of Birmingham in the UK. At the heart of the building is an interactive art gallery containing a number of permanent artefacts that were constructed with the input of various community groups. The building itself is a striking example of Will Alsop architecture, designed from inception as an interactive art space. Lead interactive consultants for the gallery were AllofUS ( In the summer of 2010, The Public agreed to permit the first author to conduct an ethnographic study of the life of the building. The aim of the study was to observe the permanent exhibition in order to inform further study. The exhibits and their participants were observed across the period of a week, allowing an understanding of their use at varying levels of capacity. The methodology of the study was necessarily loose. The main aim was to simply act as participant observer, volunteering help in any capacity The Public required. The permanent exhibition is laid out in space as a linear path and the study concentrates on finding areas of “breakdown” (Winograd & Flores, 1987, Wright & McCarthy, 2010). Breakdowns are noted and observed and a methodology will be developed to study them in detail at a later date. Coincidentally, at the time of the study, The Public had taken on eight work placement students. The training and induction of these students was observed closely and interviews were conducted on their fourth day. The first interview was with a male and a female student and the second interview was with the group as a whole. The investigation also took the form of interview, mainly opportunistic, with visitors, staff and designers of some of the exhibits. Much of the data is still being coded and analysed, though this paper highlights a specific instance of appropriation in practice that was evident from initial observation and reinforced up by interviews with the work experience students. Appropriation is specifically relevant in terms of the interactive installation Animo constructed by the company Utani ( This exhibit is a large scale interactive that requires collaborative interaction.

12 [last retrieved 12 October 2010]

13 [last retrieved 12 October 2010]


The Public’s website describes it thus: “Animo allows visitors to create a 7-frame-wide image using props and any other objects that they wish to use. At least two people are required as one person is required to ‘direct’ and take the pictures and the other is required to ‘act’ and pose in the pictures (alongside any props).”

Figure 1: Animo The interface for Animo is uncomplicated; there are written instructions on the wall and audio instructions that describe the process step by step. The Public often employ demonstrators on Animo to guide visitors through its use. Animo is considered a key point in the building as it forms the entry to the ramp, the area in which the permanent interactive gallery is housed.

Figure 2: Animo interface and props Despite the artefacts apparent ease of use demonstrators are necessary, particularly during quiet periods. On its own, without participants, the use of the interactive environment is not apparent but once in use, it comes to life. Hornecker and Stifter (2006) have also commented on this type of social scaffolding.


Figure 3: Diagram of Animo in action

Figure 4: Animo output. Following is an extract from an interview with a male and female student. Both the interview extracts are from extended interviews discussing the whole gallery but the sections below have been selected because they specifically focus on Animo. The reason this first extract is of interest is the discussion of ownership of the final output. The interviewees highlight the fact that “you can put your own props on” (line 6) and that the interactive works “for you” (line 8). The female is resistant to the installation stating “I don’t like having things like that done” (line 65) but the male is enthusiastic “I’ll just go up and do it.” (line 68). Spontaneous repetition of the activity by the male was observed on numerous occasions. Despite her ambivalence to Animo, the female highlights that she can see other people enjoying the installation and that she gains pleasure from viewing their participation. Animo is “the ice breaker” (line 85) and “brings personality out.” (line 83) I indicates the interviewer, M, the male respondent and F the female. I:

Do you mind describing Animo to me, what goes on in Animo? Imagine that I’d never been there, I can’t see it and you’ve got to describe it to me.



You’ve got seven different frames.




You have like a, there’s a camera that shines on the actual frame and there’s a camera on the table which you can put your own props on so you can be in the actual cartoon


and then take seven different photos and then after that takes your own animation does like little things for you.



So the first time you saw it did you get it straight away?


Well, you


It’s just, you just paper on a table with seven different numbers on.


It did need to be explained, but you sort of got the general gist of it because of the example that’s on, you know that says “welcome to Animo” and you sort of get the gist because of the numbers and that but


how to use it is like a different story like ‘cos it’s very like. M: F:

Holodeck Gerrin out like you stand in front of it and it takes your photo, that’s like fine, you get that but sometimes, if you’ve been on it you then think “how’s that work? “ How the cameras are (m)angled and how does it take


the photos like. That’s what got me like, how many cameras is there? How does the camera work? How does the lighting work? I:

So are you were you were getting more into the technical angle of it and you were trying to understand it from a technical point of view?





So if there’s a family there do you think that they need somebody there to explain it to them?










Even with the, even with the audio instructions?






I just think there’s somebody there needs to give them a push like


Just to supervise all the way through


And like you obviously need someone there to move the pictures.


Yeah, so you’ve got to have more than one person haven’t you, maybe someone there to help you but I’m just thinking, just trying to think of erm you know, if you were going to put that up somewhere else and you had to put it in a shopping centre and you couldn’t afford to have anybody


looking after it, what, how could you do it?



Good question actually erm, for me I’d write like an instruction thing and like put it on a table, but you could also like have more in detail audio.


Cos that like, cos it does give you a like “move to position one” “get ready” but, like if you’ve never done that how do you know what it’s




gonna do? You don’t. M:

Where’s position one ?


And you don’t know it’s gonna take a picture because it doesn’t tell ya


So you think it needs to be even more descriptive?


Yeah, maybe like give you an introduction of the exhibit like


The actual thing


Yeah, like what it actually does


How about just pictures of things that people have done before?




Do you think they’d get it from that? I know that there’s the big one on there.


You could have like different examples, yeah, yeah, yeah, that would be good actually.


So you think that would get it over more?


Cos then, that could give you an idea and if you copy one of them and then you can do your own




So do you think you need to do it more than once to get to to really get to it?




And how many times have you guys done it now?


Ooh, must have lost count


Well, for me, I don’t like having things like that done. Like, I don’t like standing there and watching people like and people watching me having my photo took



But I’ll just go up and do it I just


So for me, I just didn’t really like it didn’t really like appeal to me like I

liked the technical side and how it worked and I think it’s good fun for people who like it. For me, I just don’t like people watching me have my photo took but like when [Male’s name] was doing it, it was good fun, like it was nice to see how people.



Interact with it


Interacting with it


OK so you start and it’s a way for you guys to get to know each other. Did you know each other before this?




we go to the same school.


You go to the same school so it’s a lot easier for you to.


Well, yeah and there’s me, [Male’s name], and then there’s the other [Male1’s name]

As a point of clarity, both Males discussed had the same first name. I:



and we all go to the same school but everybody else doesn’t and I think it was like that was the ice breaker weren’t it kind of cos that was the first


thing we come to on the tour and it was the ice breaker because we was watching everybody have fun. And with things like that, it brings personality out, I think because if [Male’s name] weren’t really talking at the start but when he come on that he was like his funny self so people got that from him



(pointing at male) was that you? Was it that [Male’s name]?


Yeah that [Male’s name] and people got it from him so like you can so like helps

Following is an extract from the group interview, again focusing specifically on Animo. For the group interviews subsequent participants will be referred to as M1, M2 or F1, F2 and so on Relevant here is that the discussion almost immediately introduces appropriation, “You can put anything personal to you on there.” (line 4) The output is described as belonging to the participant “It’s like being in your own little movie” (line 21). I:

So describe Animo, come on everybody describe Animo to me


There’s seven like green screens sort of things and you stand in front of it and you take your picture

F2: 5.


You can put anything personal to you on there, like something you like, and take a picture with it


Something personal, so would I need to come prepared?








The resources are available anyway


All you need to do is bring yourself (laughs)




[] so come on, I’m quite interested in Animo, I’ve never seen Animo and I don’t know what it’s all about [].



It’s like really big, yeah (laughs)




And then there’s seven different slide show things and then you just go there and you just have fun and then you jump around and take pictures and then at the end it’s like a sequence and it goes really fast and it’s really



funky M3:

It’s like being in your own little movie




Yeah, like a storyboard thing


Plus, if you wanted to do that by yourself, it costs a lot. This is free.

Much of the dialogue refers to personalisation and ownership of the final artefact. There is discussion on how the interactive is engaging and enjoyable though not immediately intuitive. Many participants will use their first interaction as a way of becoming familiar with the workings of the interactive and will repeat their attempt, having gained virtuosity through practice. The final output of the interactive is a series of composite pictures of the participant that “acted” with enlarged versions of props placed on the table by the participant who “directed”. The Public provide a variety of props for people to use, often incorporating a theme. For example, various signifiers of local culture such as letters spelling out the phrase “Bostin,” a local slang term; and the coat of arms of the local football club. It is on repeat attempts that participants take ownership of the interactive and appropriate the interface to create their own personal outputs. This was particularly pronounced during observations on a busy Saturday when the building was at capacity. A group of four young people entered Animo and went through the interactive in its prescribed and intended manner. It was clear that these young people enjoyed their experience and they made several repeat attempts. During these attempts, as well as employing the props provided for them, they emptied their pockets and began to use the personal items they carried with them. These items included packets of cigarettes, mobile telephones and cigarette lighters as well as other items.

Figure 5: Output from a group of young people – their faces disguised As well as using their personal items, the group used the opportunity to create a series of group portraits, posing and genuflecting. Many of the poses they struck were reminiscent of record covers by popular hip-hop groups. By adapting the interface and output of the interactive, the youths took ownership of the environment. A good example of appropriation and personalisation through Animo is on display in the offices of The Public. Members of staff there have created a wall of images of each other utilising props that describe their daily activities. For example, the accountant is pictured with a giant calculator and the technical supervisor with a network cable. Animo provides users with an environment in which they can create their own content. The design of the product encourages and rewards appropriation and it is this appropriation that creates engagement.


A consistent request after using Animo is whether users are able to get a copy of the final output. This is currently not possible but the apparent desire for this souvenir demonstrates the high level of satisfaction provided by engaging with this product.

Figure 6: The Office Wall – faces disguised

Discussion We conclude by observing that by empowering people to make artefacts their own is not merely an effective means of creating engaging artefacts but lies at the heart of user-centred design. Animo was originally designed to enable participants to storyboard seven frames of an animation. The interactive can be employed in this manner but through use, it has become common for participants to create engaging image series with increased creativity. Users are not able to make Animo “their own” per se but they are given the opportunity to produce highly personal output. This output is seen as belonging to the user. From the conversations with the work experience students and the actions of the visiting youths as well as observation we can gather that Animo is a good example of appropriation producing engagement. The factors to consider are how to demonstrate the engagement of a product from quick observation and how to allow the deliberate appropriation and reinterpretation of the product. When users feel that that they have made the product their own through appropriation, they are keen to repeat their experience. Repeat experience is a clear sign of engagement.

Acknowledgements Many thanks to all staff at The Public, particularly Graham Peet, Exhibitions Manager.

References Akah, B. and Bardzell, S. (2010) Empowering Products: Personal Identity through the Act of Appropriation. Proceeding s of CHI 2010: Work in Progress, pp. 4021-4026. Anglin Burgard, T. (1991) Picasso and Appropriation. The Art Bulletin. 73(3), pp. 479-494. Blevis, E. (2007). Sustainable interaction design: invention & disposal, renewal & reuse. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA, April 28 - May 03, 2007). CHI '07. ACM Press, New York, NY, 503-512. Blevis, E. & Stolterman, E. (2007). Ensoulment and Sustainable Interaction Design. Proceedings of International Association of Design Research Societies Conference IASDR 2007. Hong Kong, China: HKPT Blom, J (2000) Personalisation - a taxonomy. Extended abstracts CHI 2000, pp. 341-342.


Blythe, M and Hassenzahl, M (2004) The Semantics of Fun: Differentiating Enjoyable Experiences in Blythe, M, Overbeeke, K, Monk, A and Wright, P (Eds) Funology: From Usability to Enjoyment (pp 91-100) The Netherlands: Kluwer Academic Publishers. Dix, A. (2007) Designing for Appropriation. Proc. HCI 2007, 27-30. Galloway, A., Brucker-Cohen, J., Gaye, L., Goodman, E., and Hill, D. (2004) Design for hackability. Proceedings DIS'04. ACM Press, New York, NY, pp. 363-366. Gibson, J (1986) The Ecological Approach to Visual Perception Lawrence Erlbaim Associates Hillsdale New Jersey. Hassenzahl, M. (2004) The interplay of beauty, goodness, and usability in interactive products. Human – Computer Interaction, 19, pp. 319 – 349. Hornecker, E. and Stifter, M. (2006) Digital Backpacking in the Museum with a SmartCard. Proc CHINZ ‘06 ACM Press, New York, NY pp 99-107 Ilyenkov, E. (1977) Problems of Dialectical Materialism (Translated by A. Bluden). Progress Publishers. Available from Lindgaard G, Fernandes G., Dudek C. and Brown, J. (2006) Attention web designers: You have 50 milliseconds to make a good first impression! Behaviour & Information Technology, 25(2), 115 – 126 MacLean, A., Carter, K., Lövstrand, L., and Moran, T. (1990) User-tailorable systems: pressing the issues with buttons. Proceedings CHI '90. ACM Press, pp. 175-182. Newman, M. W., Sedivy J. Z., Neuwirth, C. M., Edwards, W. K., Hong, J. I., Izadi, S., Marcelo, K. and Smith, T. F. (2002) Designing for Serendipity: Supporting End-User Configuration of Ubiquitous Computing Environments. Proceedings DIS, 147-156. Norman, D.A (1988) The Psychology of Everyday Things, New York: Basic Books. Norman, D.A. (2004) Emotional design: Why we love (or hate) everyday things. New York: Basic Books. Silverstone, R. and Haddon, L. (1996) Design and the domestication of information and communication technologies: technical change and everyday life. In Mansell, R. and Silverstone, R. (Eds) Communication by Design: The Politics of Information and Communication Technologies. Oxford University Press, New York, NY, pp. 44-74. Turner, P. (2010) The Anatomy of Engagement. Proceedings ECCE 2010, 91-100 Wells, M (2000) Office clutter of meaningful personal displays: the role of office personalization in employee and organizational well-being. Journal of Environmental Psychology, 20, 239-255. Winograd , T. and Flores, F. (1987) Understanding computers and cognition : a new foundation for design. Addison-Wesley: Reading Mass. Wright, P. And McCarthy, J. (2010) Experience-centred design: designers, users, and communities in dialogue. Morgan & Claypool: San Rafael California.



Co-Design of Interactive Exhibits: the MIME case study


Christophe Bortolaso, Cédric Bach University of Toulouse - IRIT [email protected], [email protected]

Francis Duranthon Museum of Toulouse - AMIS [email protected]

Emmanuel Dubois University of Toulouse - IRIT [email protected]

Abstract Cladistics is an abstract and counterintuitive classification of species. In the context of the Museum of Toulouse, it is one of the most important concepts presented to the visitors. To ease the understanding of this classification method and to stimulate the visitors discovering this thematic, we (museologists, designers, computer scientists ...) have envisioned new ways for supporting the exhibits explorations. We chose to explore the use of mixed interactive systems (Augmented reality, Tangible interfaces, etc.) and found that they constitute a good alternative to involve the visitors in such scientific thematic. In this paper, we present the development of MIME, a Mixed Interaction for Museum Environment: its aim is to experience the major principles of cladistics. This paper also describes and illustrates the design of this prototype: it was based on a process dedicated to the development of mixed interactive museographic exhibits. This paper then concludes with lessons learnt from the use of the design process and feedbacks about the prototype and its use into the museum.

1 Introduction Museums play the major function of acquiring, preserving, conserving, managing and presenting collections. Usually collections take the form of galleries, showcases and panels. In such contexts, the visitors remain very lightly involved and are not real actors of their visit (Drioli, 2006). To overcome this consideration, the collaboration between museologists, designers, computer scientists, ergonomists, etc is required to develop solutions that better stimulate the visitors on a thematic and transform the visitor from a spectator to an actor of his/her visit. To reach this goal, museums and science centres add interactivity to support the exploration of their exhibits (Wakkary et al. 2009). For example, visits can be made more engaging through the use of audio multi-media or interactive guides. However, when it comes to explain a complex and abstract phenomenon such as a methodology or a scientific concept, designing such systems becomes harder. To support the design of such interactive experiences, a dedicated process is required (Brown et al., 1998). The use of an appropriate methodology should ease the collaboration between disciplines and allow several specialists to select suitable content and museum settings for the exhibits. Following this goal, we have developed and applied across the past years a co-design process dedicated to guide and structure the cooperation between disciplines; it is also intended to help designers identifying appropriate and innovative forms of exhibitions, especially in complex contexts. In this paper we present the structure of this co-design process through an illustration of the design of “MIME” (Mixed Interaction for Museum Environment). MIME is a prototype related to the domain of cladistics. Cladistics is a counterintuitive classification system of species and organisms and is therefore hard to transmit to the visitors. Using an interactive approach to support the mediation of this theme is therefore particularly suitable. Hereafter, we introduce the principles of cladistics, existing interactive applications in this domain and the basic principles of our prototype.


2 Interactive approaches for presenting Cladistics 2.1. Understanding Cladistics Proposed in the 1950’s, Cladistics is the modern method for the classification of life. This classification method is based on evolution of criteria rather than similarities/differences between species (Hennig, 1950). A classification is represented with evolution trees, called cladograms (see Fig 4), in which leaves represent taxa and in which nodes represent a hypothetical ancestor characterized by an evolutionary stage of a criterion acquired during evolution. However, cladistics is still largely unknown and poorly understood by the large audience: indeed educational experts have pointed out that cladistics is really awkward to learn (Lecointre, 2004). To ease the understanding of cladistics’ principles, many pedagogical applications have been proposed. For example, (Phylogène, 2009), developed by the French National Institute in Pedagogical Research, has been deployed in schools and universities to introduce students to the process of building cladograms. However, the use of Phylogène requires extensive domain knowledge as well as a teacher’s assistance: it is thus inappropriate for museum contexts. Other interactive tools like Arbres (Hyptique, 1998) propose a more informal vision of cladistics. The aim of Arbres is to make accessible the concept of structure and understanding that the tree semantic does not depend of its graphical depiction. 3D animations of rotating branches of cladograms bring a playful touch to the subject. Although Arbres has been developed for museum context, it does not introduce the notion of phylogenetic criteria and all its underlying concepts. These examples illustrates two forms of existing interactive application for Cladistic: they are either complex and require the intervention of a teacher/facilitator, or they simply increase the visitors’ awareness of cladistics (Wagensberg, 2005) and are usable by the general public without the presence of any expert. Our development of a new support for understanding cladistics aims at finding an intermediate position between these two situations. The goal is to present most of the Fig 4. Cladogram example major cladistics principles and to better involve the visitor as an actor of the exhibit. Therefore, in collaboration with curators of the Museum of Toulouse, we designed and implemented new interactive supports that provide a simple experience of cladistics in museums.

2.2 MIME: Exploring a tree of life from its inside MIME stands for Mixed Interaction for Museum Environment and aims at teaching the major notions of cladistics. This prototype provides two views of a cladogram, i.e. a tree of species: 1. A 3D interactive egocentric representation of the cladogram 2. A 2D global representation of the cladogram The 3D interactive representation (Fig 5. n°1) of a cladogram allows users to navigate with an egocentric view inside the branches of the cladogram. In this view, cladogram’s nodes and leaves are represented by rooms connected to each other by tunnels (Fig 5. n°2). In each room, the visitors can change their point of view from left to right and access information cartels providing details about the phylogenetic criterion or specie represented by this room (Fig 5. n°4). For example, panels summarize the list of acquired characteristics since the cladogram root and a brief definition of the current specie/criterion.


Fig 5. Different perspectives of MIME egocentric view Users can also interact with two doors to access the connected rooms (Fig 5. n°5). Opening a door and moving in the tunnel to the next room means acquiring a new phylogenetic criterion and therefore making a step forward into the evolution of life. Going a step backward in the evolution is also supported. Finally, users can also take a look through windows to get a short view on brother criterion/specie (i.e. the brother nodes in the tree). In this view, tunnels are made of bricks that correspond to the bricks constituting the walls of the Museum of Toulouse: this further links the interactive application to the Museum and contribute to the visitor’s immersion into the visit. This representation is intended to stimulate users to visit as many rooms as possible. Overall, the trajectory through the 3D rooms helps in the understanding of the characteristics that define different species (leaves).

Fig 6. Evolution of MIME map view The 2D representation of the cladogram plays the role of a map (Fig 6). This view provides many information and feedbacks that help the visitors in the exploration of the cladogram. When the application starts, the map view of the cladogram (cladomap) is totally hidden (Fig 6 n°1) except the species. To limit user’s spatial localization difficulties in the 3D view, a green rotating arrow on the cladomap indicates the direction of the user‘s point of view and position in the 3D view. In addition, when the user looks through a window in the 3D egocentric view, the current brother node is highlighted on the 2D map view (Fig 6 n°2). Finally to encourage users exploring the cladogram, a percentage indicates the exploration rate. This representation is thus intended to encourage the visitor to explore all the rooms of the 3D representation in order to discover the entire 2D map. With MIME, the visitors are therefore encouraged to explore a 3D cladistics tree and to discover that the complexity of species (tree leaves) depends on the amount of phylogenetic criteria (tree nodes)


visited from the root. As a result, the application aims at explaining one of the most important principles of cladistics: “every species in a cladogram is defined by the set of phylogenetic characters inherited from the cladogram’s root”. Having these rules and messages defined, we then focused on the design of appropriate interactive techniques. Following Wagensberg’s prescriptions would result in the use of real objects (Wagensberg, 2005). But cladistics is an abstract phenomenon for which no real object can be identifiable. Therefore we chose to incorporate the manipulation of physical objects to support the exploration of cladistics concepts: physical objects constitute a tangible support to interact with the representation of the cladistics concepts. Such interactive situations are called mixed interactive systems (MIS) (Dubois, Nigay, and Troccaz, 2001) and are also called augmented/mixed reality systems or tangible user interfaces. Such interactive systems combine: • the computer’s ability to store, retrieve and transform digital data and • the user’s physical abilities and the physical resources surrounding him. Using MIS to support the interaction with multimedia applications triggers the use of physical artifacts; this reinforces the semantic coherence between the manipulations and the object of the exhibit, thus guiding the visitor through the exploration of the presented information. Using appropriate alternative to traditional interaction therefore constitutes an adaptation of Wagensberg’s principles and may be useful to instrument the presentation of methodologies or abstract concepts in a museum context. We developed two different techniques to interact with this environment: each of them involve different aspects of the physicality of the interaction. The first one is based on physical manipulation of a flashlight. As shown in Fig 7, predefined movements of the flashlight are recognized by the system. By handling the flashlight, users can change their point of view, move from one room to another, and access information panels into the egocentric view of the cladogram. The detection of the flashlight’s orientation is based on video detection allowing six degrees of freedom to users. As a result, the visitors just have to handle the flashlight and move it to start interacting with the system.

Fig 7. Description of the flashlight interaction technique

Fig 8. Description of the Cubtile Interaction technique

The second interactive technique is based on the use of the Cubtile™ (de la Rivière et al, 2009), a five faces multitouch interactive cube. As shown in Fig 8, we have associated each face of the cube to a defined action in the MIME environment. Left and right faces are used for changing user point of view respectively to the left and to the right, top face is used to access information cartels, and front face is used to open doors in the environment. The back face remains unused. The design and the evaluation of the two interactive techniques of MIME have been supported by a codesign process dedicated to mixed interactive museographic exhibits (Dubois et al. 2011). In the next section, we show how this process led us to consider all the specificities related to the pedagogical principles, the mixed interaction and ecological integration in the museum.


3 Co-Design Process for Mixed Interaction Applications in Museums From the identification of the thematic, the deciphering of museographic expectations and the definition of the pedagogical message, the design of our system has been fully performed in collaboration with museum experts. However, to identify the appropriate requirements and find the “appropriate design“, we have followed a specific co-design process dedicated to mixed interactive museographic exhibits (Dubois et al. 2011). This one aims to: 1. Facilitate the communication between involved disciplines (musicologists, ergonomist, computer scientists, designers, etc.), 2. Guide the design team throughout the development and 3. Support the exploration of initial expectations rather than just users’ requirements. This process is composed of a set of four major phases (Fig 9), each one covering a different aspect of the design: 1. The “preliminary analysis” phase is devoted to the analysis of the domain, i.e. the identification of a theme, relevant goals and messages, position in the exhibit path and finally, constraints and user profile in the considered domain. 2. The “analysis of interactive principles” phase involves the definition of how to produce an interactive context that will support the expression of the previously identified goals and messages. 3. The “optimization” phase is an iterative design phase that aims at successively improving aspects of the prototype’s design, such as (but not limited to) its interaction, integration and social dimensions. 4. The “production” phase finalizes the prototype, possibly leading to an industrialization of the solution. These four phases define the global structure of the process, separating design issues linked to the museum itself, requirements analysis of the interactive application, design and implementation, and finally its diffusion. Each phase requires the implication of different disciplines, but the weight of the role of each varies from one phase to another.

Fig 9. The co-design process In the following section, we report the use of this design process for the MIME case study and detail for each phase how it has been instantiated and which specificities of the final prototype have been considered.

3.1 Preliminary analysis This phase aims at identifying, with the museum experts, thematic and activities into the museum that may benefits from an interactive application. Before designing MIME, we have identified several such situations. For example, activities that require to handle abstract entities (e.g. energy of two different molecules, classification of life), to manipulate huge elements (e.g. a lake, a volcano), or even situation for which the object of interest cannot be brought into the museum (e.g. a planet, fragile artifacts) constitute relevant thematic to be computerized. In the case of MIME, after many discussions and codesign sessions with the museum curators, we have identified the classification of life (i.e. cladistics) to be an appropriate concept to computerize. For this thematic, we identified with curators a first generic activity: “the exploration of a complex structure”. To understand cladistics it is important to focus on the structure of the tree rather than on the leaves (i.e. the concretization of the living, the species). This step of the design is crucial because it guides all the requirements specified in the following steps of the


process. In addition, this phase is also the place to specify the visitors’ profiles. For MIME, we have extracted from museum’s statistics that public from 15 to 40 years old, with some basic knowledge in science were the most interested in the chosen thematic (i.e. cladistics). At this step of the co-design process, no interactive considerations have been explored yet. But this phase led us to investigate the thematic field and to understand the museum objectives and the visitors’ profiles. The next phase focuses on more specific aspects of the interactive application itself.

3.2 Analysis of interactive principles This second phase aims at defining the overall design of an interactive system which could support the generic activity identified in the previous phase. In this phase, HCI specialists and ergonomists are the main actors and collaborate with domain experts, which are the only ones able to express domain constraints and recommendations. This phase includes three major steps. First, the boundaries of the generic activity have to be defined. It aims at defining the minimal elements required in an interactive context to support the generic activity. When designing MIME the concretization of the generic activity (i.e. the exploration of a complex structure) becomes: “to become acquainted with a complex structure, a user needs path from a starting point to a target”. Based on this instantiated activity, additional specifications of the interaction are required: it includes the definition of Guiding rules and Phasing. Guiding rules provides general recommendations related to the content setting. When designing MIME, we relied on ergonomic criteria (Bach and Scapin, 2010) to support the decision making process. Two guiding rules have been identified: 1. A first person view will be used to navigate into the cladogram in order to reinforce user’s presence (Heeter, 1992) 2. The whole structure will be provided through a second view in order to always have a view of all reachable species 3. Mixed interaction will be used to increase visitor’s immersion and fun and to constitute a tangible support to interact with this representation of the cladistics concepts Phasing is intended to structure the user’s activity with the interactive application into a hierarchical set of sub-activities. For MIME design, we used the MAD task model (Gamboa-Rodriguez & Scapin, 1997) to reflect about action sequence and associated feedbacks. Finally, based on all the identified requirements since the beginning of the process, this phase ends with the elaboration of an initial proposal. This proposal represents a satisfecit (Simon, 1996) reached by the design group. It should compile all the design decisions in a unique artifact to serve as a means of communication understood by every stakeholder, and to reveal aspects that will need further design refinement. It can take any sufficiently representative form of the functional requirement, guiding rules and phasing of the application: a story board, a paper mock-up, low-fi prototype, etc. For MIME, we chose to build a low-fidelity prototype showing the navigation into the tree through a PowerPoint and a paper mock-up representing the external view of the cladogram (Fig 10). At this step, design decisions related to the interaction were limited to the interaction in output. About input interaction, we only chose at this stage the use Mixed Interactive System, so the design had to be considered more deeply. After a collaborative validation of all the design choices, the design process enters into an iterative cycle.

Fig 10. Initial proposal of the MIME application


3.3 Optimization This phase is dedicated to improvements of an interactive experience in a museum. During this phase dimensions such as interactivity, usability, nature and content of the experience, socio-cognitive considerations as well as the museum setting must be taken into account. Considering all these dimensions necessarily requires an iterative and incremental approach. To optimize the design of MIME, we relied on participatory design techniques (Mackay, 2003) which are commonly adopted in the field of HCI. Participatory design has also proven to well address social considerations, as well as other contexts in which decision making and co-design play a central role (Schuler et al., 1993), which match perfectly our design context. This includes four major steps: • “Analysis” of the situation to identify remaining problems, • “Design” to generate ideas solving the identified problems, • “Prototyping” to concretely represent the generated idea, and finally • “Evaluation” to study the adequacy of this solution with end-users When designing MIME, we have iterated many times over this optimization cycle, however in this paper we focus on the optimization related to the interactive dimensions. However, designing mixed interactive techniques requires more than just a participatory and iterative cycle. To better handle the multidisciplinary design group, to reinforce the exploration of the design space of mixed interactive techniques and to take the opportunity of finding an original and appropriate interactive technique, we relied on a model assisted creative sessions: MACS (Bortolaso et al, 2011). During this session participants representing all the involved stakeholders, generate ideas to find a mixed interaction solution for a defined problem. Their creative activity is supported by the manipulation of a design model of mixed interaction (Gauffre and Dubois, 2011), i.e. a model allowing to represent and to specify different mixed interaction possibilities: participants thus explore the space of possibilities and produce alternative solutions. For MIME, the design space was opened to various input devices, tangible objects, and interactive metaphors. From this MACS, we extracted several solutions, each one related to a different interactive metaphor, using different physical/digital artifacts. Among them, we selected the flashlight solution because of its adequacy between the proposed solutions, the technical limitations and the museum constraints. We also retain the tactile based interaction, because it seems to better attract the visitors to the place where the interaction takes place. It has been further materialized through the use of the Cubtile™. In the next section we present the evaluation of these two prototypes. The evaluations are based on a combined use of lab and field activities (see Dubois et al, 2011 for more details), thus constituting a form of evaluation continuum. We report in the next sections solely the main qualitative results of field evaluations.

3.4 Evaluations The evaluation of prototypes in museums involves many research questions about methods, tools, equipment, metrics, planning, ecological level... We report here some results and feedbacks about the evaluations of the two versions of MIME in the Museum of Toulouse using the FoldI platform (Dubois et al, 2011). The FoldI (i.e. foldable incubator) is a nomad user lab dedicated to field experiments in Museums (see Fig 11), which can be deployed in the museum for short periods. The flashlight based MIME version has been tested by 14 individual volunteers who had previously visited the museum. The other visitors could freely observe the experiment and on their request take part in it. We observed that participants used the flashlight between 10 to 15 minutes and explored between 80 to 100% of the cladogram. During these field experiments, we identified and fixed several unexpected technical issues. For instance, we did not have any control on the ambient light in this space, which caused troubles to our vision-based object detection. Another relevant example of technical problem is related to the height of the participants: we had to regularly adjust the position of the camera used to detect the flashlight‘s position. Indeed, there is a significant difference between an eight-year-old child and a tall young man. These feedbacks led us to change the video based sensing for a more permissive sensor, such as the use of magnetic detection.


Fig 11. Foldable Incubator - a nomad lab for field experiments in Museums From a museographic perspective, participants were satisfied with the prototype and we observed interesting behaviors. For example, during a session, more than 20 free visitors requested to be involved into the session or asked the test monitor about the design/test process or just suggested improvements for the prototype. We also observed after the use of MIME and during the post-session interviews, surprising reminds and insights of the museum cladistics exhibit. Indeed, part of the first floor of the Museum of Toulouse is architecturally organized as a cladogram. However, the single visitors almost never notice it. After the use of MIME flashlight, we got from several participants feedbacks highlighting that MIME took its place into the physical exhibit and helps the visitors to better catch the museographic message: “this game reminds me of the first floor of the museum ... it’s a kind of species classification, I understand now...”. . Regarding to the visitor experience, we extracted interesting social behaviors. For example, during the tests, we observed formation of visitors groups looking at the interaction user/prototype (sometimes around 40 visitors). The participants to the experiment played the role of magnet to visitors. Additionally, during those sessions, the docents took time to concretely experiment the prototype and envision how they could take advantage of this tool to create new entertainments with and for visitors. The Cubtile based MIME version was evaluated in a slightly different way. First, this prototype version was evaluated by groups of 3 to 5 real visitors (i.e. not by invited participants) and second it was performed in an isolated room of the museum. We chose this study configuration to closely observe social interactions inside groups using the MIME Cubtile version. Technically the 3D view was retroprojected (size 230 cm) and the 2D representation of the cladogram (cladomap) was available on a computer screen close to the Cubtile (Fig 12). We did not observed unexpected technical problems in this configuration. 20 visitors took part to the test (7 groups). Through these tests, we observed a great efficiency of the Cubtile in the 3D cladogram exploration task. Indeed, the Cubtile’s materialization of the physical space seems to constitute a real advantage to quickly handle the interactive exhibit. We also observed a deep exploration of the cladogram. Some visitors explore 100% of it; none were under 50%. Users experienced the prototype from 10 to 20 minutes. Regarding the social interaction, we observed some new interesting behaviors, especially with the teenagers. For example, we noticed the apparition of: • co-interaction behaviors (several visitors using one or two faces of the Cubtile at the same time), • alternating roles (a visitor plays the guide by using the cladomap, and another one plays the driver by using the Cubtile) and • distributed roles (one participant plays the driver and the rest of the group guide him/her). This prototype was a real good experience for the teenagers and for families (parents with one or two children) but was perceived as really monotonous and repetitive for couples between 30 and 40 years old.


Fig 12. Installation of MIME and the Cubtile in the Museum of Toulouse From these two experiments we also extracted some interesting suggestions for improvements of the MIME application itself (i.e. independently of the interaction technique). For example, several visitors suggested the adding of graphics, animations, sounds and/or textures in adequacy with species (for now the texture is in adequacy with the museum architecture). They also suggested implementing an animated virtual guide. This kind of suggestions is typically the costly improvements candidate to the last phase of the process: Production (Fig 9). In terms of bad feelings, two participants reported light cases of cyber sickness due to the velocity of movements and the size of the 3D screen.

4 Discussion and lessons learnt To successfully design and implement those advanced interactive exhibits, a multidisciplinary approach was unavoidable. However, establishing a synergy between different experts, in order to take advantage of new technologies in the museum, remains a real challenge. The described and illustrated co-design process in this paper is intended to provide a structure: 1. To organize the multiple resources relevant for the design of interactive experiences in complex contexts 2. To federate the different stakeholders In this section, we report a set of lessons learnt from the use of this co-design process: they partially illustrate how the co-design process answers these two goals. These results have been extracted from the design of MIME and seven other interactive prototypes produced in collaboration with the museum. In the next sections, we first briefly position our approach with regards to existing design processes and then we develop lesson learnt from our experience with the Museum of Toulouse and its curators.

4.1 Existing design approaches Design approaches of interactive systems have clearly been derived from software design methods. Software cascading (Pressman, 92) or spiral (Boehm, 87) cycles are well established and focus on the structuring of steps that are important for the implementation of the systems. More recently Agile approaches tends to better involve the end-user through very brief cycle and frequent feedback from the end-user (Armitage, 2004). But on overall, the goal of these approaches is to finalize a software system rather than to properly answer to users’ requirements. In contrast, HCI design processes primarily focuses on users’ needs and tasks (Dix, 2004). Standards have been developed (Iso, 2002) and used in multimedia and professional contexts. Additional approaches have been proposed to complement the global approaches with specific considerations such as children (Kelly, 2006), disabilities (Sears, 2008) or usability specification and measurement (Scapin and Law, 2007). Alternatively, when users’ needs are not easily understood and identifiable in the earliest steps of the system design, participatory design is often used (Muller, 1992) even in museum context (Koleva, 2009). However, such approaches reach their limit when the initial requirements cannot be directly translated into or related to interactive considerations: in museum contexts, the identification of the knowledge to transfer must be integrated in the global design process but will not find any answer in a traditional visitor centered approach.


For that reason, collaborative design has been quite widely adopted in museum contexts: these approaches tend to ensure that different but complementary considerations such as educational consideration for example will not be left apart. Although it is not easy to perform (Taxen, 2004), it has been successfully applied in different cases (Ciolfi, 2008). As mentioned in (Taxen, 2004) further explanations and clarifications of such approach are nevertheless still required to better involve the different stakeholders and better take advantage of their participation. Our co-design process thus appears to be in line with the most recent evolutions of design processes to be used in museum contexts: it structures and documents the steps, stakeholders and tools available to cover complementary considerations of the design of interactive experiments in museums. From its application by different design groups, we collected substantial hindsight to identify the major benefits of this process.

4.2 Lesson learnt First, the process really appears to support the design activity. Indeed, we have observed that it facilitates the communication among the different experts and guide them through the different considerations to deal with. Multi disciplinary participants have been involved as well as their different design resources: ergonomic criteria have been combined with task tree, museographic requirements have been taken into account during interaction design, etc. In addition, it helps the design group to stay focused on design questions before thinking in terms of technologies. It has been possible to base the design process on requirements that were not limited to user’s needs but that also included museographical expectations in terms of knowledge transfer. This process was also found helpful for designing museographic content and application. Through the generic activities, it supports the homogeneity over the exhibition path and opens a structured and controlled avenue to the insertion of advanced technologies in museum exhibits. Furthermore, we observed that the process is transferable to other design teams. Indeed, four different design teams have successfully applied it for other prototypes intended to deal with different thematic of the Museum (e.g. eutrophication of lakes). Finally, the use of the co-design process confirmed its ability to support an improved form of participatory design approach. Indeed it confers to the users a certain duality: they are first observed and then actors of the design. This is similar to what Boedker (2000) observed. Of course all these observations needs to be further analyzed thorough a more longitudinal study of the benefit of such a process. Further work will also be required to transfer its daily use to museum staff.

5 Conclusions In this paper, we focused on rethinking the design of interactive experiences in Museums and particularly emphasized the case of complex and abstract concepts such as cladistics. We have introduced MIME, a prototype which aimed at experiencing some principles of cladistics. To do so, we relied on mixed interactive systems. These interactive technologies are in line with a modern stream which aims at replacing physical objects into exhibits. As a result, MIME supports the introduction of major concepts of cladistics while avoiding the involvement of teachers and thanks to mixed interaction, MIME deeper engages visitors and transforms their visits into interactive experiences. Other abstract and complex themes of the museum have also benefited from such forms of interactive experiences: we have already designed systems dealing with other aspects of cladistics or eutrophication (i.e. filling of ponds phenomenon). Along our design experiences, multiple considerations, perspectives and expertise have been considered and articulated. As a substitute to actual approaches mainly based on ad hoc and technological explorations, we relied on a co-design process dedicated to museographic exhibits. Our co-design process constitutes a scaffolding for selecting and interleaving the set of design resources which are the most appropriate to the multiple expertise involved. Furthermore, requirements definition, museum exhibit and interaction design, implementation and evaluation phases are covered by this co-design process: our co-design process is therefore guiding the whole development from the earliest analysis phase until the final concrete installation. The use of this co-design process has been illustrated on the MIME case study and revealed its major benefits and limits. Further experiments in terms of transfer to curators, and other museographic theme will be studied in future work.


References Armitage, J. (2004), ‘Are Agile Methods good for design’, ACM Interactions, 11(1), p.14-23. Bach, C. and Scapin, D. (2010), ‘Comparing Inspections and User Testing for the Evaluation of Virtual Environments’, IJHCI 26, 8, 786-824. Bödker, S., Ehn, P., Sjogren, D., and Sundblad, Y. (2000), ‘Co-operative Design — perspectives on 20 years with the Scandinavian IT Design Model’. 9 pages. Boehm, B. (1987) ‘A spiral model of software development and enhancement’,Computer,p. 43-57. Bortolaso, C., Bach, C., Dubois, E. (2011), ‘A combination of a Formal Mixed Interaction Model with an Informal Creative Session’, EICS ’11, ACM Press, (in press). Brown, J., Graham, N., and Wright, T. (1998). ‘The Vista environment for the Coevolutionary design of user interfaces’. CHI’98, ACM Press, 376-383. Ciolfi, L., Bannon, L., and Fernstrom, M. (2008), ‘Including Visitor Contributions in Cultural Heritage Installations: Designing for Participation’, Museum Management and Curatorship 23, 4, 353-365. De la Rivière, J.-B., Orvain, E., Kervégant, C., and Dittlo, N. (2009), ‘The cubtile: 3D multitouch brings virtual worlds into the user’s hands’. SIGGRAPH ASIA ’09, ACM Press, 63. Dix, A., Finlay, J.E., and Abowd, G.D. , Beale, R. (2004), Chap. 6: ‘HCI in the software process’, Human-computer Interaction 3rd Ed., Pearson Education Ltd. Dubois, E., Nigay, L., Troccaz, J. (2001), ‘Consistency in Augmented Reality Systems’, EHCI'2001, Springer Verlag, p. 111-122 Drioli, A. (2006), ‘Contemporary aesthetic forms and scientific museology’. Journal of Science Communication, 5(1). Dubois, Emmanuel. Bortolaso, Christophe. Bach, Cédric. Duranthon, Francis. and Blanquer-Maumont, Anne. (2011), 'Design and Evaluation of Mixed Reality Interactive Museographic Exhibits', International Journal of Arts and Technology (IJART), Inderscience Publishers. Gamboa-Rodriguez, F. and Scapin, D. L. (1997), ‘Editing MAD* task descriptions for specifying user interfaces, at both semantic and presentation levels’, DSV-IS '97, Spain, 4-6 June 1997 Gauffre, G., Dubois, E. (2011), ‘Taking Advantage of Model-Driven Engineering Foundations for Mixed Interaction Design’. MDDAUI. Springer-Verlag, p. 219-240, Vol. 340, Studies in Computational Intelligence, 1 ISO 14915 (2002), ‘Software ergonomics for multimedia user interfaces -- Part 1: Design principles and framework’, International Standards Organisation Hennig, Willi. (1950), ‘Grundzüge einer Theorie der phylogenetischen Systematik’, Berlin : Deutscher Zentralverlag.


Heeter, C. (1992), ‘Being there: The subjective experience of presence’, Presence: Teleoperators and Virtual Environments, 1(2), 262-271. Hyptique. (1998) , last access 2009/10/26. Kelly, S.R., Mazzone, E., Horton, M., and Read, J.C. (2006), ‘Bluebells: a design method for childcentred product development’, NordiCHI ‘10, ACM Press, 361-368. Koleva, B., Egglestone, S.R., Schnädelbach, H., et al. (2009), ‘Supporting the creation of hybrid museum experiences’, CHI ‘09, ACM, 1973-1982. Lecointre, G. (2004), ‘Comprendre et enseigner la classification du vivant’. Guide Belin de l’enseignement. Mackay, W. (2003), ‘Educating Multi-disciplinary Design Teams’, Tales of the Disappearing Computer, 105-118. Muller, M.J., Wildman, D.M., and White, E.A. (1992), ‘Taxonomy of participatory design practices: a participatory poster’. CHI ‘92, ACM Press, 34-34. Phylogène (2009), Pressman, R. (1992), ‘Software Engineering: a Practitioner’s Approach’, McGraw Hill Inc., 3rd edition. Scapin, D.L. and Law, E.L. (2007), ‘Review, Report and Refine Usability Evaluation Methods (R3UEMs)’, COST 294-MAUSE 3rd International Workshop, Athens, Greece, March 5 Schuler, D. and Namioka, A. (1993), Participatory design. L. Erlbaum Associates. Sears, A., Young, M., and Feng, J. (2008), ‘Physical Disabilities and Computing Technologies: an Analysis of Impairments’, The Human Computer Interaction Handbook, Lawrence Erlbaum Associates, p. 829-852 Simon, H.A. (1996), The Sciences of the Artificial - 3rd Edition. The MIT Press. Taxén, G. (2004), ‘ Introducing participatory design in museums’, Participatory design ‘04, ACM , 204-213 Wagensberg, J. (2005), ‘The “total” museum, a tool for social change’. História, Ciências, Saúde – Manguinhos, v.12, p. 309-21. Wakkary, R., Hatala, M., Muise, K., Tanenbaum, K., Corness, G., Mohabbati, B. and Budd, J. (2009), ‘Kurio: a museum guide for families’. TEI ‘09, ACM Press, 215-222.


Towards the Wild: Evaluating museum installations in semi-realistic situations Eva Hornecker and Emma Nicol Department of Computer and Information Sciences, University of Strathclyde, Glasgow, UK [email protected], [email protected] Abstract In this paper we discuss our experiences of conducting user evaluations of technological installations for the new Robert Burns Birthplace museum and the challenges associating with doing this. We believed it to be important to avoid a sterile lab situation, and to be able to observe the social dynamics that evolve around museum installations. As we were unable to conduct studies in the museum itself we devised ways of taking them out of the lab by basing them in a more museum-like or semi-realistic setting. We took efforts to recreate the installation setup when this was likely to influence interaction patterns and could reveal usability issues. Moreover, we invited young families and adult groups to take part in the evaluation sessions to investigate how they would interact with these as a group. We explain our motivations and the tradeoffs involved. We focus on discussing the effects that the semi-realistic setting may have had on the user feedback obtained, reflect on the differing user behaviours that might emerge in the genuine museum setting and make recommendations for future studies.

1 Introduction In the museum and heritage domain, projects are often on a very short schedule, with limited sources of funding. Most technology-based installations are being developed by SME art and design companies, who lack a budget for in-house research and evaluation. This has several consequences. Installation designers primarily rely on their intuition and design experience, and design companies rarely receive detailed feedback. Even imaginative, engaging and visually interesting installations may suffer from usability issues, as installations rarely undergo user testing before deployment. Observations of visitor interaction in museums (Hornecker and Stifter, 2006; Hornecker 2008) found that minute details can hamper the user experience and prevent people from appreciating what an installation has to offer. Moreover, there is not much literature about how to best go about conducting formative user evaluations of museum installations. Most of the museum literature focuses on describing approaches for and findings of visitor studies within museums (Macdonald 2008). These tend to investigate aspects such as navigation, dwelling times, the effects of room layout and exhibit location, and visitor learning. We have found little literature describing the development process of installations and in particular, how to go about testing early prototypes which are not yet at a stage where they could be temporarily deployed inside the future setting. As described above, even with the best content and concepts, glitches in usability can hamper the use experience, and may even prevent what Allen (2004) refers to as ‘immediate apprehendability’ – visitors understanding within the first few seconds of approaching an installation how to interact, and getting the gist of what it is about and what is the reward for interaction. Formative evaluations and usability tests aim to identify problems early-on in development by observing five to ten participants using a system, providing recommendations how to improve it. These methods are in wide use in the software industry where it is important to optimize website layout and navigation, and to ensure that interface metaphors are well understood by users. Traditional usability testing focuses on efficiency, speed, and error rates in individually performed tasks. This assumes core tasks which most users will want to achieve (e.g. place an order in a webshop). In a museum setting it is difficult to tell what ‘a task’ might be – visitors primarily want to be entertained and educated, and their aim might change with what a system offers. Free-choice learning is intrinsically personal (Falk, Dierking and Adams 2006), and thus cannot be prescribed through tasks or measured in learning outcomes. The museum setting offers many distractions, and an installation that is not immediately easy to use and satisfying is quickly discarded for other attractions (Allen, 2004). User testing of museum installations thus needs to investigate both usability and enjoyment. Moreover, the museum situation is inherently social – visitors come in groups, want to share an experience, spend time with children or friends (Kelly et al 2004, Heath and vom Lehn 2002), and are indirectly affected by the presence of other visitors (vom Lehn et al 2007, Hornecker 2010). The social context is identified as one of the core aspects of the visitor experience in the literature


(Falk, Dierking and Adams 2006) In this sense, museum installations are more akin to e.g. social/party games, which often are tested by handing them to beta users, or playtested in labs that simulate a living-room (Pemberton and Griffiths, 2003; Isbister, Schaffer 2008). Study of team-play has proven useful for game designers, providing insights into how teams share control and communicate. In this paper we discuss our motivations and experiences in evaluating a set of interactive installations for the new Robert Burns Birthplace Museum (short RBB museum) with potential users. Our team was responsible for running formative evaluation studies of prototypes for four touchscreen and one tablebased installation and providing reports to the National Trust Scotland and the design company developing the systems. Our reports provided feedback on interaction problems, how users enjoyed the installations, whether they understand how these relate to Robert Burns, and suggestions for re-design. The main visitor groups for the RBB Museum are young families and so-called ‘empty-nesters’. Most of the installations are aimed at engaging children with the themes of the museum: Burns’ life, his poetry, and the era he lived in. They have the form of mini-games, for example, creating a photo-fit of what Burns might have looked like, and having a mouse jump up to grab cheese in the rhythm of a poem. To achieve more ecological validity, we decided to test with groups of users, and to invite families to ‘play’ with the prototypes. One of our main motivations was that interaction issues experienced by a group might be very different from those discovered in single-user trials. Research in Computer Supported Cooperative Work (CSCW) has revealed how requirements for group work can sometimes conflict with requirements for individual work (Gutwin and Greenberg 1998). This very likely would also be the case with edutainment activities. The ability of a game to entertain a family might be essential for its overall success. Would siblings be able to play the games together or would they be fighting? Would the games be interesting enough for children of different ages and their parents to stand back and observe while others play? What age group would like the installations? Might the presence and help of parents make the games accessible to younger children, who are not able to play them on their own? Some of the recent museum studies literature has highlighted the role of family and parent-child interactions, with parents explaining, pointing out things, asking questions (Kelly et al 2004, Sanford et al 2007). This would mean that having children use these systems on their own is not representative of the kinds of interactions occuring in the museum setting. Moreover, one of the installations, an interactive table, was designed explicitly for cooperative (and competitive) gaming. In this paper, we describe how we aimed to account for these factors and discuss some of the questions this left open. While we tried to emulate the group situation of a visit, our user trials nevertheless took place in an artificial situation: families were invited, remained together for the study, with no other activities competing for their attention. The museum was being rebuilt at the time, and we were thus unable to run studies in the real location. Moreover, the versions we were provided with were early prototypes. Our study helped to uncover many issues that could be resolved for the final versions of the installations, including issues relating to group coordination and play. But we anticipate that there might be issues specific to the museum situation that are hard to replicate in an evaluation trial setup. The aim of this paper is to contribute to a reflection on ways of user-testing prototypes aimed at group and family use, and the potential limitations of doing so in a semi-realistic setup. Most of the detailed findings of our evaluation sessions are not relevant in this context, being specific to the installations and the flaws of early prototypes. Our summary of findings thus focuses on issues related to the study setup and our simulation of a group use situation. We present a summary of issues, reviewing our observations with the question of whether similar behaviours would occur in the museum. As followup research, we currently engage in an ethnographic-style observational study to compare behavioural patterns from the user trials with the ‘wild life’ in the RBB museum.

2 Context and Study Setup The Robert Burns Birthplace Museum in Alloway, Ayrshire comprises the cottage where Scotland’s greatest poet was born, some of the landmarks providing the setting for his works, and a recently enlarged and rebuilt museum. The new museum opened in late 2010. It is a popular day trip location for young families, school fieldtrips, and senior adults. One of the aims of the rebuilding of the museum was to engage young children (and adults) in a playful way with Burns’ life and poetry. The


museum makes great efforts to be family-friendly, offering special activities for children, and includes a number of interactive multimedia installations. Burns is lived heritage in Scotland (and to some extent in the wider UK), where a majority of people will learn some of his poems in school and many will at some point attend a Burns Supper. The RBB museum displays objects for veneration (original letters and objects), but in multiple ways also invites curiosity (some artefacts are in cabinets which visitors are invited to open) and playful interaction. Children are allowed to run around and play, and besides the multimedia installations, there are a number of non-software interactive stations, where children may for example draw. During our recently started initial ethnographic observations in the museum we observed that this playfulness extends to many of the older visitors. Our team was commissioned to conduct formative evaluation studies of five of these installations in summer 2010. The software was being developed by Spiral (London, UK). We were provided with one installation at a time, and would be handed the next installation prototype after submitting our report within a given time window. Our remit was to focus on usability, overall enjoyment and game play issues, and to provide concrete recommendations for how to improve the systems. Given the museum was still being built at this time, it was clear that questions of visitor learning would be hard to address, as the systems would be experienced out of context. Moreover, the installations on their own do not have learning objectives in a traditional sense. They are part of a ‘show not tell’ interpretation strategy which is to help visitors enjoy Burns’s heritage in an imaginative and interactive way, and are particularly aimed at engaging young children in a playful way with themes of the museum. We were thus only asked to investigate whether users would realize what the installations were ‘about’ and whether they felt they had learned anything. We now briefly describe the installations and then focus on how we set up and ran the user study sessions.

2.1 Overview of the Installations We here focus on four installations, aimed at children and teenagers, most of which were touchscreenbased. For the purposes of this paper, we only provide a rough overview of these mini-games. It should be noted that at the time of testing, some of these were incomplete, sometimes with only a subset of content being implemented, in other cases employing simple line drawings. PhotoFit (see figure 1 right) presents the visitor with contemporaries’ descriptions of Burns’ appearance (“he had fiery eyes”), who then create a mugshot, selecting facial features from a series of images. PhotoFit is purely touchscreen-based. One can select the nose, mouth, left and right eye (individually) and the overall face, including hairstyle. Touching either on the facial features or the headings on the top navigates between different sets of images and brings up historical descriptions of how Burns looked like. In the museum, this installation is now set next to portraits of Burns’ family. The Poetry game plays the poem ‘To a mouse’ and a piece of Tam O’Shanter spoken by an actor, while the poem’s text moves over the screen. An animation of a mouse runs beneath the text, and at the moments of emphasis a piece of cheese is placed over the words. The user has to press a button in the rhythm of the poem, whenever the mouse is under the cheese, to make it jump up and eat. After completion, one can read the entire poem and is presented with a score of how many pieces one managed to catch. This installation aims to create awareness that poetry has rhythm; the poem is usually learned in school. This installation uses a mix of interaction technologies. While the game is started and one can choose between the two poems by touching the screen, a large button under the screen is used to pound the rhythm of the poem. The Spooky Stories touchscreen game illustrates the poem ’Tam O’Shanter’. In the poem, Tam, a drunken villager leaves the pub on a dark night, encounters mysterious creatures on his way home, running across the graveyard, until he jumps on a horse and rides over the bridge, where the spooky creatures cannot follow. The installation shows the village, and has the user press the creatures from a ‘virtual transparency’ onto the background and move them into the scenery (figure 1 left). The user can navigate through the scenery by moving the transparency to the side of the screen or pressing the arrows on the right and left of the screen. Once completed, an animation shows Tam going home past the creatures.


Finally, the Burns Supper Table is a multiplayer game in five rounds based loosely on the tradition of the Burns supper (see figure 2). It is played around a table, with a projection from the top. Each player has a physical button to interact; the table itself is not touch-sensitive. First, players have to press their button when their segment of the table is highlighted, showing it is their turn, to play the notes of a well-known tune (played during the ‘bringing in of the haggis’ during a Burns supper). In the next round, players stab haggis (a kind of sausage) that move around the space, the button press making an image of a knife poke out. Then, a memory card game starts, alluding to the tradition of giving an ‘immortal memory’ speech on Burns. Then, reminiscent of the ‘toast to the lassies and laddies’, players can throw insults and compliments (in the Scots language) towards two figures in the centre. The game ends with the song ‘Auld Lang Syne’, and players need to make a circle of figures swing their arms in time with the song by pressing the buttons. If they manage to keep in time, their segment lights up.

Fig 1. Tam O’Shanter (Spooky Stories) Touchscreen with transparency on top of the scenery showing the village. PhotoFit game with only the har being selected so far, and the rest only being indicated by sketch drawings. The quotes describe Burns’ hairstyle and colour.

Fig. 2. Some of the games on the Burns Supper Table. Left: One player has just managed to shoot a compliment to the girl in the middle (the figures rotate around the centre, making aiming tricky) and the player is rewarded with a translation of the Scots word ‘clarty’ (filthy, mucky). Right: Stabbing the haggis, with the ‘host’ in the middle. The buttons are located on an outer rim just beneath the hands respectively knifes in the image.


2.2 Setup and Process of the user trials Since the technical and contextual setup can greatly influence usability and social experience of play, changing how visitors use exhibits (Isbister 2010, Monahan 2005), we took care to emulate the final setup as much as possible and to the best of our knowledge. It was crucial to utilize touchscreens for the tests, as touch interaction has different usability requirements than mouse use and offers more affordances for collaborative action. Moreover, display angle and size influence how easy it is to select targets on a touchscreen and affect how easily a parent can observe and scaffold a child’s interactions. The touchscreen games were tested on a large touchscreen (20”). to approximate the final installations. We set them up at about the height and angle indicated by architectural sketches. We also built a table of comparable size and setup to the Burns supper table, enabling us to assess how the game was experienced when played in a group and using buttons (see figure 4). For this, we installed five pushbuttons in a self-built table and connected these via an Arduino controller with the Flash-based game, which was top-projected onto the table. Given the RBB museum is visited by many young families and by groups of older adults, it is important that the installations are enjoyable for groups. Each game was evaluated with six to eight families with children of different ages, and by two groups of ‘empty-nester’ aged adults. Each group had two to four members, sometimes with both parents present, and for about half of the trials, there were two or more siblings. For Spooky Stories and Burns Supper, we asked adults to bring a friend with a child, resulting in a few larger groups (with five to six members). We received written consent either from a caregiver or the participant himself. Each participant was rewarded with an NTS voucher. In a further attempt to make the setting more natural, increasing ecological validity, we looked for opportunities to conduct the studies off University grounds, and were granted access to the library space within the Kelvingrove Art Gallery and Museum (see figure 3). Our participants approached our study through this well known museum, and many would spend more time there either before or after the session. Our intention for this was to put participants at ease and to have them experience the installation in a museum setting. This location furthermore enabled us to recruit a few participants on the spot. However, recruitment of most participants relied on networking and snowballing via an initial group of parents. Due to the complex technological setup required for the Burns Supper table, it had to be tested in our University lab. Within the museum library space, we cordoned off a corner between some bookshelves so that other visitors would not disturb us. At University, where we tested the table, we used our lab which at this point in time was relatively empty (see figure 4). We began the sessions by welcoming participants, explaining the study aims, and handing out consent forms. We then invited them to play while we observed. In usability studies of games (Isbister and Schaffer 2008 ) it has proven beneficial not to provide task lists, but simply let groups play with little intervention, as it allows participants to focus on the game and forget about being observed. Our only instruction thus was to ‘go and play with it’. Only if the group struggled for a considerable time to e.g. discover the basic interaction mechanism did we intervene in order to minimize frustration and to enable the group to experience the main part of the game – our remit after all was to generate feedback on all aspects of the installations. Towards the end we asked parents if they would like to have a go in case they had not yet played. During all this time, we took extensive notes of our observations, including group interaction style, the interactions between children, parental interventions, any issues with the installations, and also noted down things that worked well and were succesfull.


Fig 3. Trial setup in the museum library, with researcher observing and taking notes. Children playing with the ‘Spooky Stories’ touchscreen. After ending the active game play, we asked younger children to draw a picture of themselves playing the game, while parents and older children filled out a questionnaire. This asked them to rate how easy they found the games and how much they enjoyed them. They were further asked to describe what would positely or negatively influence their inclination to use the installation in a museum, to list what exactly they enjoyed and what they found frustrating, what their children liked or disliked, how the installation “could be improved to be better to use and more fun”, and what they ‘learned’ from doing the activity. Sessions ended with a short open-ended group interview, using the younger children’s drawings to get them talking about their experience. In this final conversation, parents would usually provide more context to their questionnaire answers, and sometimes developed further ideas on how to enhance the installations. Each session lasted 40 to 60 minutes.

3 Families in the lab versus ‘in the wild’ in the museum Overall, we could identify many issues in usability and collated feedback on how to improve and expand the mini-games. Our combination of observation, questionnaires, and open discussion proved very useful. We found that often questionnaires reflected users’ final impressions (of eg. how easy a game was to play), but did not always capture initial problems in figuring out what they were required to do in order to play. In a setting with more distractions, the same group of users very likely would give up and leave (cf. immediate apprehendability, Allen 2004). Some interaction issues, such as difficulties in selecting small touch targets, are rarely mentioned, and users may not be consciously aware of these despite obviously struggling. Open discussions and open-ended questionnaire questions provided a wealth of insights into what kinds of things people enjoyed, and how they could imagine extending the installations. In the following, we focus on issues related to family group interactions, and on questions that our study setup did not allow us to answer. While we took care to create a relaxed setting for families, and to remind participants that the installations would be within a museum, we were unable to recreate all aspects of a real-life museum setting. This means that there may be interaction issues which we were unable to test and predict. To investigate this, we are currently conducting an ethnographic-style observational study within the museum. The following discussion is informed by our experience from observational visitor studies within museums and a reflection on whether some of our observations in this user study might be an artefact 54

of the study setup. We start by discussing the variety of group interaction styles observed, and how different games evoked different patterns of group interaction. We then focus on the role of adult scaffolding of children’s interaction and facilitation of group interaction, mediating between siblings, for example. Finally, we discuss issues that highlight the limitations of our study setup. For example, we could not emulate the interaction between groups of strangers, and found that participants often made suggestions for improvement and extension based on experiences with video games used at home that were not appropriate in a museum setting. We conclude with a reflection on the utility of our attempts to emulate the ‘real’ museum situation and the tradeoffs involved.

3.1 Different Installations Evoke Different Group Interaction Styles The style of social interaction differed markedly across installations with a clear pattern emerging for some of the games. Without inviting family groups and extended families (bringing a friend) to our studies, we would have been unable to observe this. In particular, some behavioral patterns only occur in larger groups with e.g. several children. One of our study aims had been to investigate whether the installations would be fun to be played with a sibling present, or whether they would exclude them from participating, potentially resulting in conflict. We also found that some of the games gave more room for parental involvement then others. Adults often kept in the back during the trials, and first had their children try. With younger children, the parents often played first to show how to do it. While adults often took part in games that could be played cooperatively by taking turns (especially PhotoFit), parents largely kept in the background for the Poetry game, and only played after their children had done so several times. This is partially explained by the kind of activity the Poetry game entails, which is difficult to share and requires a lot of concentration in listening, watching the screen (the mouse has to be just under the cheese when the button is pressed), and beating the rhythm. When parents or another sibling played this, it provided an additional learning experience for children, as these could now focus on listening to the poem. The term scaffolding covers helpful guidance and assistance that enables a child to do things he/she cannot yet do fully on his/her own. Parents may scaffold by, for example, explaining what to do next (‘press this’), reading out instructions, and guiding children’s hands. While families engaged in a lot of turn-taking and scaffolding each other with the PhotoFit, the Poetry game was very much an individual activity, with the family watching. Apart from helping to read, there was little scaffolding from parents. This is probably because the game is very fast, leaving no time to explain or give instructions. Here, family members were often competing against each other in rounds, shouting out and memorizing their final score and comparing against previous scores. With Spooky Stories, smaller children sometimes competed and fought for control, even moving each others’ hands away. This was almost never seen with the other installations.

Fig 4. Different family configurations around the Burns Supper Table: Two families, a group of young teenage friends with a mother, and two mothers (standing back) with daughters. At the Burns Supper Table we saw the biggest variation in parental scaffolding, with some parents taking a ‘hands-off’ approach, leaving it to their children to figure things out. This usually resulted in a less engaged experience, with young children (below reading age) often not understanding what to do.


They would press buttons at random times, but not experience the actual gameplay. In reverse, in groups with older children these often were first to figure out how to play and show or explain to their parents. Different to other installations, here parents could play along, and some (who joined the game) clearly enjoyed it, confessing to getting quite competitive. Some families got into competitive mode, shouting out and comparing scores after games. But many also enjoyed playing as a team, encouraging the others to play correctly (e.g. with the Auld Lang Syne game saying ‘lets see if we can get it to light up’). The Supper table overall seemed to support both team play and competitive play, some of the mini-games being more competitive then others, but leaving the style of play open to participants. Initial observations of the final installation in the museum reveal similar patterns of people explaining to each other what to do and playing enthusiastically against or with others, indicating that our attempts at recreating (or rather pre-creating) the social situation of the museum setting were successful.

3.2 The Role of Adult Facilitation and Scaffolding As discussed above, parental supervision and scaffolding is often instrumental in helping children use an installation effectively. Parents will furthermore often add context, point out things the child should notice, and engage the child in conversations that relate the current object to previous experiences (Kelly et al 2004, Sanford et al 2007). Moreover, parents are busy facilitating and moderating interaction between groups of children, in particular if these are siblings, trying to minimize conflict and to ensure that all children get their share. We observed parents reading out aloud for younger children and facilitating sibling interactions. PhotoFit, for example, was highly enjoyed by very young children due to being easy to manipulate, despite considerable amounts of textual description. Even a two year old successfully interacted with it. Parents often tried to orient children towards the educational goal of the installation (emulating Burns’ facial features), and to make them read and understand the descriptions. Parents’ behaviour was carefully modulated reflecting the children’s age and reading ability, reading out aloud, encouraging them to read, or asking questions about what certain phrases meant. Parents further moderated and facilitated turn-taking by siblings. Parental facilitation and scaffolding was also very important for the Poetry game. Here, parents encouraged children to listen to the poem, and, using the pause when it is shown full-screen, asked whether they remembered it from school, or knew what certain words meant. For this installation, without parental intervention, children mainly saw it as a quick-reaction game, almost ignoring the poem. The role of parental guidance observed was encouraging in terms of showing how family interaction can make installations accessible and enjoyable for youngsters who would not be able to do so by themselves. The entry age for most installations turned out to be much lower than anticipated by system designers and ourselves. Also, parents regularly used the opportunity to educate their children, explaining the content, and asking children to remember things. This highlights the importance of setting installations up in a way that provides space for parents and allows them to see what their children are seeing and doing (cf. Hornecker 2010), which in turn enables them to comment, intervene and help. The question left open is whether parents will be as closely engaged in the actual museum situation, or whether they will prefer to have their children entertained with the installations while they browse the showcases of Burns’ letters and belongings, read information panels, and talk with other adults. Parents in our study very likely felt that they needed to supervise their children more closely then they would do without the presence of an observer. The amount of parental engagement may also be influenced by subtle differences in the physical setup, for example the exact height of touchscreens, the floor (many parents sat or kneeled down), and the space available. We anticipate that in the actual museum situation, there will be an even greater variety of group behaviours, including children that are temporarily on their own while adults investigate display cases.

3.3 Fixed Groups in the Lab – More Fluid Constellations in the Wild In our sessions where we invited families, groups came and stayed together, with no distractions that might result from the late arrival of a group member. Our study setup cannot uncover potential interaction issues that arise from more fluid and shifting activity patterns. Previous studies on interactive surfaces in public settings have discovered that the assumption of one activity being done


by a stable group can result in system designs that do not fit with real life situations of groups being in constant flux (Marshall et al 2011), preventing newly arriving members from joining. Moreover, in a museum setting it is quite common for an activity not to be completed, and the next visitor to pick up from this point. This potentially results in interaction problems if the new arrival does not see the initial ‘entry screen’ which introduces the goal of the game and explains how to interact. For many installations, instruction bubbles (help text) are only visible initially, and disappear after the first touch. In addition, having only one family group at a time for our trials, it was not possible to investigate how easy it would be for additional people to join an existing group. We also could not determine the maximum number of people who can interact productively with the installations. We observed that children, teenagers and young adults were likely to use several buttons on the Burns Supper Table at once, working two segments (meant to be for one participant each) or even trying to cover three and more free segments (see figure 4, second from left) if there were not enough players to use all of the buttons. For our current observational studies within the museum, we are interested in whether the game encourages strangers to play together. The design of the games should encourage this, as they can be played without having to discuss and negotiate. Players can join while a game is running by pressing a button (this ‘un-greys’ their segment). But it remains a question whether the tendency to occupy several neighbouring sections of the table might hinder other people from joining, either preventing them from participating or making them hesitate to approach. In our study this did not pose a problem, as group size was fixed, and some parents did not participate. We are also interested to see how different groups or individuals might interact or follow each other at the other installations. 3.4 The Study Setup Influences User Perceptions and Suggestions Testing in a lab setting, with the prototype clearly running on a PC (not physically embedded and afixed), seemed to result in participants perceiving these as standalone systems. Even the library setting within the museum did decontextualize. Participants often compared the games with video games that they might play at home, and this seemed to influence their suggestions for extensions and improvements. Frequently, changes to the installations were suggested that involved ratings (of user performance), competitive versions (played in turns) or playing against a highscore. Children often suggested changes such as difficulty levels that would prolong the activity or make the games very complex. Competitive games can perform well in a museum setting if they allow simultaneous gaming. Having users take turns playing against each other does not work as well, as it prolongs the activity, and can make it difficult for new users to join. Highscores would further complicate the setup. These suggestions are sensible for home video games, but can be detrimental in a museum context where other visitors will need to wait and might be frustrated if they do not get to have a go within reasonable time. We saw that children were determined to complete every bit of the activities, for example in the Spooky Stories to take all figures off the transparencies and place them in the scenery. Even though children liked all of the figures, we recommended to reduce their number to ensure a better throughput. For the Poetry game, where users had to tap in rhythm to the poem being played out, many participants suggested giving penalties for tapping at the wrong time. But for games aimed at engaging different age groups, it could be discouraging to tell the user how well they performed. We saw that very young children could enjoy interaction even if they did not perform well, and older users often took several rounds to figure out what the game asked them to do. Adults often commented that it was hard to connect the games with Robert Burns, and thought that anybody who e.g. did not know the Tam O’Shanter poem well would not recognize it in the Spooky Stories installation. This, again, might be very different in a museum setup which physically contextualizes the installation through its surroundings and, for example, enables parents to draw children’s attention to how the installation relates to surrounding exhibits (cf. Hornecker 2010). This might provide additional learning opportunities that cannot be replicated easily in a lab study – especially if the final layout and arrangement of museum pieces, posters, labels etc. is unknown at this stage. Similarly, participant feedback on aesthetic aspects needs to be taken with caution when soliciting feedback outside of the use context. We repeatedly received comments from parents about the games’ visual design not being up to date, being boring compared to Nintendo games, not colourful enough, etc. despite their children clearly enjoying the same games and wanting to play them again and again.


While a simple and modern visual design, tested in a rather empty room might seem boring, it might stand out within a visually crowded environment, attracting attention through movement, where a more intricate design might be ‘too much of the same’. Here, again, adults tended to compare the installations with common PC-based or video games, and to focus on the graphics as what they believe attracts children.

3.5 Trade-offs and Experiences with our Study Setup Our motivation for running the majority of the user studies in the Kelvingrove museum library was to create a setup that is less intimidating then having to visit researchers in a University, and to have participants experience the installations in a museum-related setting. The question is how far this was successful and whether the effort involved was worthwhile. While our experiences were not wholly negative , arguably the hurdles we had to surmount in conducting the studies in this way slightly outweigh the benefits gained from using this set-up. Having said this, our approach to simulating the museum group situation by inviting young families was very successful. To some extent we believe that the setting helped to make participants feel at ease, although this is impossible to verify. This was more important for the elderly adult user group, whereas children quickly ignored the researchers regardless of setting once they started to play, and parents were simply happy to see their children entertained. Testing fun games was certainly an advantage in this respect and helped recruiting participants. Running the studies in a museum meant that parents could combine our study with a museum visit. More importantly, it enabled us to ad-hoc recruit some participants. Disadvantages of working in the museum library involved disturbances by other visitors and overall noise levels. Moreover, security was an issue, since we could not lock the door behind us, and had to carry equipment across town. In addition, being restricted to museum opening times in combination with being reliant on school-attending participants meant that we often had to run studies over the weekend. In constrast, sessions with the table-based game could take place in the late afternoon and early evenings. Some of our reflection furthermore revealed that due to the by-invitation status of sessions, participants might interpret the installations as stand-alone applications, it being too hard to imagine these being part of a larger museum visit experience. In the ideal case, one would test installations in the museum. Yet this was still being rebuilt and refurbished at the time. Alternatively, one might install prototypes in a children’s activity room within a museum, and enlist participants by having the sessions announced as children’s activities. This would at least partially enable observation of how strangers interact with each other, and also create a relaxed atmosphere. But this requires a much larger group of researchers for running studies. We were able to conduct our study sessions with only one or two researchers being present, going through consent forms, observing game play and conducting interviews sequentially. In a game-lab setup (Isbister and Schaffer 2008), several researchers would be needed to conduct and supervise all these activities going on simultaneously. While this is somewhat disillusioning, it was only by taking the effort that we could come to assess these tradeoffs. The aspect of our study that proved to be most important for its success was the focus on family and group interaction and our decision to invite larger family groups, in combination with the effort taken to recreate core aspects of the installation technology (screen sizes, touch-ability, the circular multi-button table). While there are questions left open as to whether this provides us with insights into all of the potential issues relevant in social situations in the museum, it was essential in order to determine if e.g. the multiplayer games on the table would be fun to play in different group sizes or whether more players would result in confusion, and whether siblings could participate and collaborate in the touchscreen games.

4 Outlook and Conclusion We have discussed some of our experiences with trying to emulate ‘natural’ family interactions in a formative user study of prototypes for museum installations. We have focused on the questions that our study left open, in regard to whether groups will behave similarly in the museum, and whether our setup enables us to predict additional interaction issues that result from the ecological setting in the museum situation. In our future research, we conduct observations in the new RBB museum, and will compare these with the evaluation trials.


Acknowledgements We thank Mary Hudson/Stone from the National Trust for Scotland, the Kelvingrove Art Gallery and Museum in Glasgow, all of our participants, and to Patrick Tobias Fischer and Andrew Smillie for helping to build the table.

References Allen, S. (2004), “Designs for Learning: Studying Science Museum Exhibits That Do More Than Entertain”, Science Education Vol 88(S1), S17-S33. Bell, G. (2002), “Making Sense of Museums: The Museum as 'Cultural Ecology”, Intel Labs. Falk, John H., Lynn D. Dierking and Marianna Adams (2008), "Living in a Learning Society: Museums and Free-choice Learning”, A Companion to Museum Studies. Macdonald, Sharon (ed). Blackwell Publishing, 2006. Blackwell Reference Online. Gutwin, C. and S. Greenberg (1998), “Design for Individuals, Design for Groups. Tradeoffs between Power and Workspace Awareness”, In Proceedings of CSCW'98, 207-216. N.Y.: ACM. Heath, C., and vom Lehn, D. (2002), “Misconstruing interactivity”. Proc. of Interactive Learning in Museums of Art. V&A Museum Hornecker, E. and Stifter, M. (2006), “Learning from Interactive Museum Installations About Interaction Design for Public Settings”. Proceedings of OzCHI 2006. N.Y.:ACM, 135-142 Hornecker, E. (2008), “"I don't understand it either, but it is cool" Visitor Interactions with a MultiTouch Table in a Museum”. Proc. of IEEE Tabletop 2008. 121-128 Hornecker, E (2010), “Interactions Around a Contextually Embedded System”, Proc. of TEI 2010. ACM Press 169-176 Isbister, K., and Schaffer, N. (2008), Game usability: advancing the player experience. Morgan Kaufmann Isbister, K. (2010), “Enabling Social Play. A Framework for Design and Evaluation”. In: Bernhaupt, R. (ed) Evaluating User Experience in Games. Concepts and Methods. Springer Kelly, L., Savage, G., Griffin, J., and Tonkin, S. (2004), Knowledge Quest: Australian Families Visit Museums. Australian Museum and National Museum of Australia Macdonald, Sharon (Ed) (2008), A Companion to Museum Studies. Blackwell Publishing, 2006. Blackwell Reference Online. Marshall, P., Morris, R., Roger, Y., Kreitmayer, S. and Davies, M. (2011), “Re-thinking 'multi-user': an in-the-wild study of how groups approach a walk-up-and-use tabletop interface”. In Proceedings of ACM CHI’11, ACM Press, to appear. Monahan, P. (2005), “Letting Visitors Surprise Us”. Proc. Of International Workshop: Re-Thinking Technology in Museums: Towards a New Understanding of People’s Experience in Museums, Limerick Pemberton L. and R. N. Griffiths. (2003), “Usability Evaluation Techniques for Interactive Television”. in Stephanidis C. (Ed.) Proc. of HCI International 2003. Universal Access in HCI, Volume 4, Lawrence Erlbaum Associates, 882-6. Sanford, C., Knutson, K. and Crowley, K. (2007), “We Always Spend Time Together on Sundays”: How Grandparents and Their Grandchildren Think About and Use Informal Learning Spaces”. Visitor Studies 10:2, 136-151


vom Lehn, D., Hindmarsh, J., Luff, P. and Heath, C. (2007), “Engaging Constable: Revealing art with new technology”, In Proc. of CHI’07. ACM. 1485-1494


Historical Orchestra: An interactive audio/ visual cultural experience Ferhat !en & Lily Díaz Aalto University, School of Art and Design, Media Lab {ferhat.sen, lily.diaz}

Abstract In this paper we describe a case study, Historical Orchestra, about the use of digital technology to enhance the museum experience of exhibiting an ancient illustrated manuscript. The project is based on a 16 th century manuscript, Surname-i Hümayun that is kept in archive of Topkapi Palace Museum in Istanbul, Turkey. The project is an interactive installation that makes use of a physical tangible interface paradigm. In our essay we explain the design of the interface and interaction along with the description of the system. Keywords: Digital Cultural Heritage, Pyhsical Tangible Interfaces, Interaction Design, Illustrated Manuscripts

1. Introduction 1.1 Newer Paradigms for Digital Heritage and Museums As computers increasingly populate and become part of our everyday life they change already existing practices and bring forth new activities and interaction modalities. In these new information ecosystems, traditional graphical user interfaces (GUI) and Window, Icon, Menu and Pointing Device (WIMP) interaction loose their appeal and relevance. Instead, tangible interfaces where the user interacts continuously with the data through objects from the physical environment become more appealing. From initial applications developed for example with Notecards such as Perseus (Crane 1996) or the Virtual Palenque project that already in the late 1980’s made use of interactive panoramas (Wilson 1988) to the latest developments showcased at international forums, the history of digital heritage application development runs parallel to the history of computer interface design and development. More recently tangible user interfaces where the user interacts with the data through objects from the physical environment have become more appealing. This is partly due to the appearance of new, offthe-shelf and open components, such as Arduino and Kinect. The application described in this paper is an example of these new developments.

1.2 A Design Approach for Digital Heritage The approach in designing this digital cultural artifact follows three main principles, artifact translation, concept of specificity and three-level hierarchy of artifacts. Digitizing an artifact means creating a digital copy of the current state of the artifact, mostly using recording media for preserving and displaying purposes. Solely digitizing is no more than transforming the medium of the artifact from real world space to virtual space. Artifact translation, however, is taking the original artifact and creating a derivative of that artifact in digital domain, which includes additional qualities to the original one provided by digital technology. The former provides preservation and accessibility while the latter additionally offers innovative possibilities on how to utilize that artifact.


Fig 1. Medium Transformation and Artifact Translation Diagram In designing for digital cultural heritage, a significant method to create novel and useful solutions is to consider the specific qualities of the artifact in question. That is, it is not only displaying the visual being of the artifact but also representing and conveying the embedded value and/or information that are inherent in that artifact. These inherent qualities include (but are not limited to) the physical qualities, the materials used during the making of, the processes applied to its interpretation after its creation, the ways of use the artifact throughout history, and the details in the story it narrates. Placing a text next to the artifact in the manner that is currently done in most museum exhibitions can have a similar effect. However, we argue that the multi-sensory experience which is active participation rather than passively seeing could contribute to the dissemination of that information in a different way than that of the text. From our perspective of design-as-translation this means comprehending the roles that objects play as mediators of human experience. This understanding is important for the relevancy of the work that we do as designers, since we live in an era where much of our experience relies on computer mediated communication devices. More and more, our interactions with the world unfold within fields comprised of digital representations. We contend that through these it is possible to craft new interfaces and interaction that afford audiences the means to conceptualize and experience traditional media artifacts, such as an illustrated manuscript, in a different way and while still preserving the original. This experiential perspective requires that we consider interface and interaction as involving not only gadgets but also the physical, social, and cultural contexts of use. How will the interaction of a professional musician, or a scholar of illustrated manuscripts with an application such as Historical Orchestra, differ from that one of a member of general audience with no training in these fields but who has an interest in the topic? Is it even possible to design applications that take care of the needs of such polysemous audiences? Among our goals is to observe and record such possible differences and similarities among the experiences of various communities with the objective of deriving a sense of equivalency. In using the concept of equivalency we want to suggest a mapping of the space between the experiences produced through a digital-born artifact as compared to those emerging from of their original counterpart. When and how should the structuring of the interaction implemented in the digital artifact diverge from the one suggested by the original? Are there points of contact between the ancient object and the contemporary digital artifacts that can be further elaborated through the interface? How contingent are these connections on the situatedness of the experience, including the background of the actor interacting with the application? Is there a pattern that can be generalized or is each experience with a digital replica distinct and in a sense possessing its own sense of ‘aura’? (Díaz 2011)

2 Case Study: Historical Orchestra Historical Orchestra is an interactive installation based on the Surname-i-Hümayun, a 16th century Turkish illustrated manuscript. The installation utilizes three tangible musical interfaces and gives realtime visual narrative feedback. The sensor-based musical interfaces enable three users to play three


instruments simultaneously. The visuals are based on an event depicted in the manuscript. The selected illustration is interactively animated according to the actions of the user with the instruments.

2.1 The Context: Surname-i Hümayun The illustrated manuscript used for this case, Surname-i Hümayun (The Book of Imperial Celebration), was written as a text and illustrated between 1583 and 1588 in Istanbul, Turkey. It is a documentation of a fifty-two-day arts and crafts festival organized by Sultan Murad III for the circumcision celebration of his son Prince Mehmed in 1582. The text is written in 16th century Turkish and using Arabic script that follows a right to left layout. The illustrations have a two-page layout that also follows the right-to-left structuring where the illustration on the left page is the continuation of the right one.

Fig 2. The selected illustration pair used in Historical Orchestra, from the Manuscript of Surname-i Hümayun located at the archive of the Topkapı Palace Museum with inventory number H. 1344. A total of 432 pages of text and 472 pages of illustration comprise the volume that presents the events and activities, which took place during the Festival held in the famous Hippodrome Square in Istanbul. In each illustration pair there is a different group of artists/craftsmen depicted in their course of performance-based exhibition and their procession before the audience and the Sultan (Atasoy 1997, 13-17). From these illustrations, the pair selected by the designers for this case study is the one depicting the procession of the musicians shown in Figure 2. This illustration pair was chosen because of the opportunities it affords to create an embodied interface. With its musical instruments and musicians, it provides a narrative and a representation that makes it possible to map the user’s bodily involvement to the characters in the illustrations.

2.2 Aim and Target Audience The main objective of the project has been to research how to use digital technology to create a digital artifact based on a cultural heritage artifact. This artifact is created by considering the content and specific qualities of the original artifact and inspiring the design with those aspects. The experience created is brought forth through embodied interaction with the digital artifact where the item itself


becomes a platform that enables new meanings to emerge. One other aim of the project has been to research ways for enhancing the environment and experience of a museum visit. By creating a playful activity with the content of the illustrations it is aimed to assist the educational value of a museum artifact and dispersion of the institution’s boundaries so it can also be linked with other institutions such as schools and community centers. The target audience is visitors to museums and galleries who might also have an interest in innovative uses of digital technology and media.

2.3 The Concept An initial idea that was considered when developing the concept was that one of poetic interaction. By this we mean that the system’s development focused primarily on the stories and potential aesthetic experience emanating from the manuscript as well and not on the technology. The illustrated manuscript can be regarded as an artifact of mediation (or in the words of Marx Wartofsky a tertiary artifact) with images that represent a space of embodiment and portraying the interaction of diverse communities during the festivities. This interaction is also evidenced in the craftsmanship of the scholars and artists who participated in its making. As a work of art the manuscript also elicits study and contemplation thus conjuring aspects of a world that existed at the time of its making. Though initially lacking in history and tradition, the digitally born artifact can also aspire to become a tertiary artifact: As a new tool for mediation it can foster novel forms of creative performance and interaction in the present.

2.4 The Installation Setup Historical Orchestra is an interactive installation that enables the visitors to have a new experience of heritage based on an ancient illustrated manuscript. Only one projector is employed for the visual display. However, as shown in Figure 3 the projected image is divided into two separate screens since the original artifact is a book with two separate pages. The aim here was to strengthen the equivalence of the digital artifact to the original one. Each page of the selected illustration pair is projected onto a custom-made screen designed to fit the vertical aspect ratio of the illustrations. (See Figure 4.) The interactive animation is visible on these two screens.

Fig 3. The Installation layout

Fig 4. Installation as shown in Demoday 2010 at Media Lab Helsinki

Placed on two tables, that are also used to hide the electronic elements and cables connecting the instruments, are the three sensor-based instruments. In front of each instrument, a red circular rug indicates the standing spots for the players. Two speakers placed next to the projection screens take care of the audio feedback.

2.5 The Interface The system’s interface is based on the evolving physical tangible interface paradigm. The users manipulate the digital data with three sensor-based musical instruments. These three instruments can be considered as the main controlling elements of the interface. Based on their resemblance to actual instruments, they can be categorized as sensor-based instrument-inspired gestural controllers (Miranda & Wanderley 2006, 25). Through these instruments, the users are able to control the digital data, which in this case is composed of audio and visuals.


2.6 The Instruments Although in the original artifact there are fifteen musicians playing various instruments, it was not possible for us to employ all the instruments and their players in this design. Therefore, we decided to pick three of the musicians as the avatars for visuals and their instruments as the interface. The selected instruments are !ehrud, Ney and Zilli Def (Atasoy 1997, 32). !ehrud (Big Oud) is a string instrument, which is a larger variation of Oud. Ney (Reed Flute) is a wind instrument having various sizes and forms. Zilli Def (Tambourine) is a percussive instrument with cymbals. The selected characters and their instruments are shown in Figure 5.

Fig 5. The musicians and their instruments, !ehrud, Ney, Zilli Def, respectively. In designing the sensor-based versions of the instruments, our aim was not to imitate the instrument itself but rather to imitate the experience of the instrument within its historical context. We wanted to achieve this by making use of current affordances of the instruments while decreasing the innate constraints for playing the instruments. By this way it would be possible for a visitor to play the instrument with less effort and skills. For example, one of the affordances of the Ney (Reed Flute) is that the notes are manipulated through the openness and closedness of the holes. We have utilized this affordance in our design and employed photo sensors to detect whether the holes are ‘Open’ or ‘Closed’. According to this data we manipulated the notes played by the system. The strongest constraint of the Ney is that it requires a special training on how to blow the breath. If it is not blown properly and if the lip placement is not accurate, it does not make music. We overcame this constraint by utilizing a microphone and a signal processing system. By processing and filtering the data that comes from the microphone, we were able to distinguish whether someone is blowing or speaking. Therefore, whatever the lip position or blowing posture is, the users can play the instrument. There are two different mapping strategies for playing the instruments; note-based and loop-based. The note-based mode enables users to play the instruments note by note. The instrument does not only play one note but augments the user by playing a number of consecutive harmonic notes based on a reference note selected by the user. The loop-based mode enables users to re-construct a selected song by manipulating loopable fragments of the song. The former introduces the instrument itself and the sound of that instrument while offering a wider musical experience. However, it requires users to have some musical knowledge and skill to create the musical experience that is in accordance with the historical context. The latter introduces the instrument, sound of that instrument as well as the music in the context of the manuscript. Unlike the note-based mode, it enables musically novice users to have an experience of the music that is within the historical context. However, the disadvantage of this mode is to be limited to a set of predefined songs.

2.7 The Interaction The interaction happens in the way that user inputs to system and system outputs according to the user’s input. The user’s input enters the system with the instruments. The system’s feedback occurs in two ways, audio and visual. The interaction is designed in such a way that the narrative can be experienced in a non-linear fashion. Figure 6 summarizes the interaction states and the corresponding changes in the audio-visual space.


Fig 6. Historical Orchestra Interaction Design Matrix. No Interaction (State 0): When there is no user standing on the standing spot, there is no audio feedback. (See D0 in Figure 6). The visual image is the illustration with an empty square (A0), no audience on the balcony (B0) and no Sultan on the palace window (C0). One User Stepping (State 1): When one user comes and steps on the standing spot, the system initializes. Depending on which spot the user steps a corresponding musician appears in the illustration in the form of 2D avatar. (A1). One User Playing (State 1): When one user starts playing the instrument, the digital sound of that instrument comes from the speakers. (D1). At the same time, his avatar starts to play the instrument in an animated fashion and begins his procession. (A1) After two seconds of play a small group of audience in the balcony appears as well (B1). Two Users Playing (State 2): It is basically the same as ‘one user playing’ scenario. A second avatar appears and begins the animated procession and additional audience appears on the balcony such that the balcony is completely occupied. (A2 and B2) Three Users Playing (State 3): When a third user starts playing the instrument simultaneously with other two, his avatar also appears (A3) and then additional audience comes to the square. (B3) And the third avatar starts his animated procession. Once all the three musicians are on the square; the Sultan appears on the palace window in addition to the last group of audience. (C3) Even though a better interaction would be mapping the real audience around the installation to the audience in the animation, due to technological constraints we mapped the number of audience in the animation to the number of users playing the instrument. In this decision we relied on the analogy of street musicians. In a contemporary city scene, when a musician starts playing in the city center, a group of audience gathers around them. So does in Historical Orchestra, when a musician appears, the audience appears as well.


2.8 System Description Historical Orchestra is an interactive system where users are able to affect what they hear and what they see through a set of physical tangible interfaces using sensor technology. The system is composed of three fundamental components; they are sensor, computational and feedback components. A diagram of the system architecture is shown in Figure 7. The Sensor Component is where the computer side of the human-computer interaction begins. The human actions are measured and converted into signals through the use of various sensors. The sensors are connected to an Arduino, or open-source programmable microcontroller that can send signals to a computer via a USB input/output interface (“Arduino,” 2011). The Computational Component is mainly a Mac computer with two open-source software programs, Pure Data (PD) and Animata. Pure Data is a real-time graphical programming environment for audio, video, and graphical processing (“Pure Data,” 2011). Animata is a real-time animation software, designed to create interactive cut-out animations (“Animata,” 2011). An open-source PD patch Pduino, is used to communicate data from Arduino to PD. After being able to read the sensor data in PD, we have written a PD patch which analyses and interprets the sensor data. The interpreted data is mapped separately to each instrument’s notes and the animation with corresponding algorithms. The Feedback Component is what the user sees and hears according to actions that s/he performs with the tangible interfaces. One feedback is the real-time audio from the speakers and which the user hears when s/he plays. The other feedback is visual and includes the images projected on the screen that change as a real-time animation and in response to the user’s actions.

Fig 7. Historical Orchestra System Architecture

3 Questions and Discussions One of the important discussion topics about this project is the issue of interpretation, since the application itself is an interpretation of the original artifact by the design team. An example of this role of interpretation and its influence in the design process decisions is the analogy of street musicians mentioned earlier. We have no clear information on what the 16th century audience’s response was throughout the course of musician’s action. And at the moment, from the historical context of the artifact, we cannot prove that the number of people present in the audience was dependent upon the number of musicians. Our decision to use the audience to indicate feedback is based on our contemporary notions of audience behavior. As a result this is an interpretation of the artifact by the design team. Another issue is the fact that some elements in the original artifact are not included in the digital version. Due to technical limitations, we could only represent three of the musicians in the system


while there are fifteen in the original artifact. This involves a reduction of the semantic space of the artifact during its translation into digital domain. Even though the impact of this reduction can be diminished, it was not possible to do so within the current scope of the project. In our initial user observations, the question of whether the visitors who are playing the instruments can concentrate on the visuals has been raised. Our initial data suggests that the concentration of the users is not solely on the instruments but also encompasses the visuals. We need to do more testing to understand all the different strands that must be factored when designing the user experience.

4 Conclusion and Future Work In this paper we have presented an overview of the current stage of the Historical Orchestra project. The interface and interface related aspects of the system are described. The objective of the project has been to research how the digital technology can be used for creating an experience of out of a cultural artifact. During Winter Demoday 2010 in Media Lab Helsinki, we evaluated the prototype of the system by observing and video documenting the visitor’s interactions and their involvements with the instruments and the animation. According to our initial observations, the system creates a playful activity and engaging experience for the visitors. Users could play the instruments with a very short learning effort due to their resemblance to real instruments. They were also able to perceive what they are controlling in the illustration with the help of accurate visual feedback. We only tested the note-based playing mode. In this mode we observed that users experimented with the instruments and created new musical expressions; however the educational value seemed to be lacking because the music produced were not in accord with the context of the illustration. The Reed Flute and the Big Oud worked technically sound in terms of the interaction. However, the Tambourine did not operate as expected because of the sensor selection. It took longer time for the users to figure out how to hit the Tambourine. Our future work for the system is to make the instruments wireless and movable so that users are enabled to grasp them like the original instruments. It is anticipated that this will increase the sense of embodiment and enhance the experience. Along with the interface, interaction can also be developed further. Adding gesture-based interaction to the system is also expected to enhance the embodiment. Mapping real movements and actions of the users to the animated characters is a possible enhancement. Moreover, additional interaction between the real audience and the audience in the illustration can also be developed. This will enable the real audience to affect the system and hopefully draw them closer to the experience in the scene. Finally some other musicians and the dancer can be implemented into the interface representation perhaps increasing the degree of similarity between the original artifact and the digitally born application.

Acknowledgments We acknowledge the work of other members of the design team, Reha Discioglu, Piibe Piirma in the design and implementation of the project; Ney blower Osman Furkan Abbaso"lu, Oud Player Sinan Bekir Pamuk, Turkish literature researcher Volkan Binici for their contributions to design in cultural and musical aspects of the project. We also thank to Koray Tahiro"lu, Michihito Mizutani for tutoring the projects in terms of physical interaction design and electronics, respectively.


References Animata. (2011). Retrieved February 19, 2011, from Arduino. (2011). Retrieved February 19, 2011, from Atasoy, N. (1997). 1582 Surname-I Hümayun: Dü"ün Kitabı. #stanbul: Koç Kultur. Crane, G. (1996). Building a digital library: the Perseus Project as a case study in the humanities, Digital Libraries '96, ACM Press, New York. Díaz, L. (2011). On the concept of equivalence in digital translation of heritage artifacts. (Manuscript in progress.) Miranda, E. R., & Wanderley, M. M. (2006). New digital musical instruments: control and interaction beyond the keyboard. Middleton, WI: A-R Editions, Inc. Pure Data. (2011). Retrieved February 19, 2011, from Wartosfky, M. (1979). Perception, representation, and the forms of action: towards an historical epistomology. In Models, Representation in Scientific Understanding, D. Reidel Publishing Co., Dordrecht, Holland. Wilson, K. (1988). Palenque, an interactive multimedia digital video interactive prototype for children. In Proceedings of CHI 1988, ACM Press, New York.



Applying Basic Design Principles to Technology in Museums Mark Leslie Martello Media 4 Islington Avenue Sandycove, Dublin [email protected] Abstract Martello Media began in the 1980s as the interactive visualisation arm of an architectural design practice. During the 1990s Martello gradually evolved into a full scale museum and exhibition design company. As ‘narrative architects’ Martello combines script writers, spatial designers, graphic designers, film-makers, multimedia designers and hardware developers in a single design team. This team applies the same five basic design principles to assess the appropriateness and to determine the nature of technological interventions in museums, that they apply to every other aspect of interpretative storytelling and spatial design work. These principles are illustrated with reference to some recent Martello projects, including the multiple award winning Glasnevin Museum and Ireland Pavilion at EXPO 2010.


Martello Media Background


A Multidisciplinary Design Approach

Martello is an exhibition design company founded in 1986. The core team is multi-disciplinary, in an attempt to give equal weight to content and design - the narrative and the architecture - the message and the medium. Inspired by the phrase ‘Pyramids are Tombs’ the creative team is horizontally, not vertically structured. Projects are collaboratively brain stormed. The views of the narrative writer, spatial designers, art director, multimedia designers and programmers, film makers, content researcher and hardware technicians are given equal consideration. Their contributions are not limited to their own specialized fields. Most team members are both qualified and experienced in more than one of the disciplines required for the development of immersive multiple media spaces. 1.2 A Reaction to Outside-In Thinking Martello’s approach evolved as a direct reaction to the frustration of working in the early days of interactive multimedia in museums, as the software developers on ‘outside-in’ projects. An iconic building would be commissioned with little idea of what was to be put inside, and why. With most of the budget expended on the spatial aspects of the building and exhibition fit-out, the all important interactive content was starved of both adequate thought and funding. The traditional hierarchical approach to museum design involved an interpretative master planner handing down an ‘information matrix’ to a spatial designer, and then having little more to do with it. The ‘designer’ was often no more than a project co-ordinator, who subsequently handed down a similarly rigid framework to several separate companies to develop; the graphic look; set dressing; audiovisual production; interactive software; and bespoke audio-visual/multimedia hardware installations, and so forth, independently of each other. These consultants are effectively precluded from collaboratively contributing their valuable skills and experience to the overall conceptual ‘highlevel’ thinking.


1.3 Inside-Out Spatial Design By contrast Martello’s approach is to try to integrate the development of all aspects of interpretative design, right from the start, ideally in close collaboration with the client. Martello team members are primarily storytellers, with the capacity to research and develop basic thematic concepts, and then script and produce all aspects of a scheme. Martello trained in the basic ‘modernist’ architectural philosophy that ‘form follows function’ and believe that wherever possible narrative spaces should be flexibly designed, working from the inside outwards. As architects, Martello have designed fit-for-purpose shells to house their narrative concepts at Stowe Landscaped Gardens in the UK, and the Lartigue Railway Museum in County Kerry, for instance, as well as collaborating harmoniously with the architects of buildings such as the Glasnevin Cemetery Museum, Dublin, and the Ireland Pavilions at EXPO 2005 in Japan and EXPO 2010 in China.


Martello’s Design Philosophy

Five Clear Design Principles

Fig 1. The Ireland Pavilion at EXPO 2010, Shanghai, China Martello uses consistent criteria to assess the layout, interpretative story-lines, look-and-feel and appropriate uses of technology on all their projects. Martello try to ensure that all their projects and exhibitions have: Clarity of Purpose Layering Engagement Authenticity Resonance


The hierarchy of intended messages and target audiences is clear. The content caters for different levels of interest and knowledge. There is meaningful and innovative interactivity. The Exhibits are apposite to their setting. The content resonates with the needs of the intended audience.

The examples below illustrate Martello’s effort to stay true to these principles:


Principle One - Clarity of Purpose

2.1.1 Target Audiences Clarity of purpose involves, both being clear as to what the key messages to be imparted are, and exactly who the target audience is. Understanding who the audience is, and what motivates them is critical to shaping the appropriate experiences and outcomes.. John Falk argues that traditional demographic measurement of museum users isn’t helpful. He has identified five universal “experience types” that describe basic human needs regardless of demographic category. They are:


Explorers – Browsers motivated by personal curiosity Facilitators – Motivated by other people and their needs (parents etc) Experience Seekers – Motivated by the desire to see and experience a place (tourists etc) Professional/Hobbyists – Motivated by specific knowledge-related goals (scholar, researchers) Rechargers – Motivated by a desire for a contemplative or restorative experience Clarity on what combination of these types of visitor are being targeted by an exhibit is particularly useful when developing role playing and problem solving displays, as well as considering the appropriate layering of complex informational displays. 2.1.2 Clarity of Message - Cliffs of Moher Visitor Centre The importance of absolute clarity about the ‘main message’ of an exhibit was highlighted by the original brief for the Atlantic Edge Exhibition at the Cliffs of Moher in Co Clare. This asked for proposals for an attraction that would ‘bring visitors back to the cliffs time and again’. And yet the visitor centre was only required because too many people were visiting the site already. Martello argued that the cliffs themselves were the attractor, and that what was required was a ‘visitor deflector’ that would absorb pressure on peak days. The exhibition needed to be a ‘Clare Portal’. The multimedia displays needed to emphasize all the other wonders of the region, and not the only location in Clare that most tourists had heard of. Success can be measured by visitor surveys revealing that only 10% of visitors to Moher intended spending a single night in Clare before the centre opened. After Atlantic Edge opened, 40% of visitors to Moher now express a desire to spend several nights in Clare.

Fig 2. The Atlantic Edge Exhibition at the Cliffs of Moher - A Visitor Deflector The brief for Martello’s ‘Irish Linen Story’ in the Louth County Museum in 1995, for instance, was to produce a pictorial database of historic linen artefacts made in Louth. These were all to be seen in virtuous reality in the Ulster Folk Museum in Cultra. Martello pointed out that the covert message of any such display would read ‘Get back in your car! You are in the wrong museum!’ Martello produced


instead, a role-playing game involving growing linseed, operating a linen mill, making linen garments, and selecting combination for fashion models on a catwalk, and so forth. 2.1.3 Overt and Covert Messages at Cosmos Museum displays can usefully carry both overt, and covert messages. The Cork Institute of Technology’s research centre at Blackrock Castle Observatory is a landmark on the banks of the River Lee. Martello designed the interactive science exhibition ‘Cosmos at the Castle’ which highlights recent discoveries of extreme life forms on Earth and their implications for life in space. It invites interactive debate on mankind’s ultimate place in the Universe. In the Forum area, visitors can compose their own multimedia impression of life on Earth, and use a Pan-Galactic Email Station to pinpoint and transmit to an exo-planet orbiting a distant star. The overt message of the display is that it forces visitors to think carefully about what it is about Earth that might possibly induce ‘ET’ to respond. The animation that shows their messages streaking through space, highlights milestones along the way, such as: Planet Neptune,4 hours; Proxima Centauri, 4 years; your target planet, 96 years; and so forth. Visitors can use the Cosmos website to monitor their message’s progress in the decades that follow. The covert message being ‘The Cosmos may, or may not, be teeming with intelligent life, but either way, don’t expect a reply anytime soon! The Cosmos is so inconceivably vast.’ 2.2 Principle Two - Layered Information Multimedia touch screens have provided a very effective and convenient means of fitting vast amounts of information into very small museum spaces. Virtual presence and role playing games have also allowed exhibition to become far less collections orientated. Many contemporary ‘museums’ being in reality interactive visitor centres with few, if any, authentic objects on display. This has sometimes allowed curators be less than discriminating in the amount, and the structure of the information embedded within the displays. The hierarchy of messages is not always properly considered. A database-like approach forgets that, perhaps only John Falk’s ’Professional’ visitor category wants to surf the net whilst standing in a gallery space. The majority of visitors need to be grabbed by a touch screen in the first 10 seconds. and want to spend no more than 1.5 minutes at it. Most visitors have the internet at home. They have no more patience with didactic book on a screen, than they did with the old museum approach of printing out that book on numerous tedious, and densely written printed panels. Properly considered multi layered filtering with: thematic headlines; topical key points; body texts; supporting references; etc need to be applied on all aspects of exhibit design. This includes the spatial, graphical, audiovisual and technological aspects. In this way, hopefully both tourists and academics can get the experience they expect from within the same museum space.

Fig 3. Pan Galactic Email at Cosmos

Fig 4. Covert Message - Space is sparse.


Fig 5. Displays for spiritual re-charging

Fig 6. Information in depth for academics

2.2.1 The Life and Work of WB Yeats National Library of Ireland The National Library’s exhibition commemorating WB Yeats is intended to draw scholars and the public’s attention to their pre-eminent collection of Yeats source material, and to make Yeats’ life and work accessible to the widest possible audience. It is an example of an object rich exhibition, where the technology is integral. This allows a vast amount of scholarly information to be embedded, but in a considered layered way, so as not to overwhelm the casual visitor. Yeats’ Creative Phases The exhibition wraps the succession key creative phases of Yeats’ working life around the edge of two gallery spaces. These phases are grouped as Yeats saw them, into the early ‘Lunar’ Period and the latter phases controlled by the Sun. The layout also marks major turning points, such as meeting Lady Gregory and the shock of the 1916 Rising. In each creative phase, wall displays chart the events political, literary and artistic influences and personal contacts. His literary output and manuscripts are displayed in table cases towards the middle.

Interactive Object Labelling All of the display cabinets benefit from a touch screen based multi-layered labelling system. Each screen shows the objects laid out exactly as they are within the adjacent cabinet. However, on the screens the viewer, can zoom in to a Turner print, say, compare it with others in the back of house collection; open a notebook and turn the pages; call up additional background text on every object that interests, and so on. Furthermore every object can be linked to every other object, event and/or person in the exhibition, via a timeline database accessible from all of the screens. The Permanent Cyber Exhibition The whole exhibition space is replicated in virtual reality on the Library’s ‘Yeats Online’ site. All of the AVs, and interactive screens are fully accessible in the virtual exhibition space.

Fig 8. The Story so Far at Cosmos

Fig 9. Moving pictures are worth a 1000 words


Fig 7. The front of house exhibition, and the back of house collection can be visited at Yeats On-line This is the permanent public portal to the entire Yeats collection, that will continue on-line, long after the physical exhibition closes. The real exhibition was a ‘place holder’ in effect, for the home page of the permanent virtual exhibition. 2.2.2 A Layered Universe at Cosmos In contrast to the Yeats exhibition the Cosmos exhibition at Blackrock Castle Observatory has space for very few artefacts. Interactive media does all the heavy lifting. In the ‘Story So Far’ gallery, visitors interact with four cinema sized HD video screens via proximity sensors. The key points of cosmology, astronomy, and the evolution of life have been summarised in sixteen c 1.5 minute video animations. Visitors can absorb complicated ideas and processes such as the Big Bang, Divergence of Species, and so forth merely by watching the animations. Further layers of information are provided by screen captions and narration, however, gesturing over the colour coded sensors zooms viewers into deeper layers of supporting information. 2.2.3

Irish Lifelines at Glasnevin Museum

Glasnevin Cemetery is Ireland’s national necropolis. It combines the roles of Paris’s Pere Lachais, and Washington’s Arlington Cemetery. It was founded as a non-denominational cemetery open to people of ‘all religions and none’ by Daniel O’Connell in 1832. It keeps meticulous multiple entry records of the more than one million people buried there. Their diversity and importance makes it a place of national pilgrimage. The Glasnevin Museum opened in 2010 in a striking new building. Digital innovations that won the museum the DMA (Digital Media Association) 2011 Grand Prix include a Memory Wall, recording the names of every person buried there. ‘Milestone Lives’ is a 10-metre long multi-touch timeline table which interweaves the lives of 200 of the most interesting people buried at Glasnevin. Visitors can search by name, category or profession. Visitors with no knowledge of Ireland can choose a person at random and browse Irish history by following personal connection lines highlighting an individual’s friends, lovers, collaborators, etc. The information is headlined with a summary paragraph on each person, so that users can judge whether they wish to drill down to, either a potted pictorial biography, or a full written one. A map then shows how to find each person’s grave. Information relating to significant people buried in the future will be added on a continuing basis.



Principle Three - Meaningful Engagement

People are perpetually fascinated and engaged by other people. This is a point sometimes missed by the ‘pot shard’ school of museum. Visitors of all kinds are only ever interested in pots to the extent that they shed light on the cooks. The best form of classroom learning, for instance, is the inspired teacher who knows how to get pupils interested in a subject they wrongly perceived as boring or irrelevant by ‘educating’ ie drawing information and conclusions from within the pupil, and not by bombarding them with information that they cannot absorb. The learning process is an interactive problem solving treasure hunt in which the teacher helps with clues and hints. Interactive media in museum spaces, and on the internet, present marvellous, and often missed opportunities to democratise this form of inspired teaching. As a result the enthusiastic and well informed tour guide often remains much the most user-friendly form of interpretation at a historic site, say. In 1999 Martello were asked, as multimedia consultants, to propose a whole series of technological interventions to the Dunbrody Famine Ship in Enniscorthy, County Wexford. This was a perfect replica of an 1845 emigrant sailing vessel. We insisted, instead, in recreating the interior exactly as it would have been in 1845, (with not an interpretative text panel in sight.) The entire interactive engagement came from interacting with the captain , the crew and other passengers, dressed and speaking in the manner of the time. 2.3.1 The Value of Human Engagement at Kilmainham Gaol An upgrade of the audio-visual presentation in the Chapel at Kilmainham Gaol in Dublin in 2004 presented an interesting opportunity to use technology to enhance human interaction rather than replace it. Visitor exit surveys had indicated the gaol’s key interpretative strength was direct interpretation by

Fig 10. Milestone Lives is a 10-metre long multitouch display weaving together over 200 significant lives

Fig 11. A million names on the Memory Wall

Fig 12. 140 years of meticulous record keeping

the guides. But the guides complained that the 20 minute audio-visual presentation put pressure on their capacity to move visitors efficiently at peak visitor times. They also complained that immersion in


an audio-visual experience at the mid-point of every tour broke the critical engagement with the guide. It was very hard to recreate that personal connection in the second half of the tour.

The solution was to merge the audio-visual presentation with the guide’s performance in a ‘SuperPowerpoint’ presentation. Guides now use a touch screen pulpit in the film theatre to call up and talk over a mix of still and video images on a large-scale screen. They can edit their presentation on the fly to cater for the interests gleaned from the visitors during the first half of the tour. They can overlay foreign language sub-titles as required. The human factor turns what was an impersonal, automatic show into a live, immediate, personal and flexible performance. 2.3.2 Virtual Curators at Discover Your National Library A new exhibition ‘Discover Your National Library’ engages visitors with its Discovery Tables. These are mini versions of Glasnevin’s Milestone Lives. The subject matter is the Library’s entire print collection digitized at high resolution. The multi-touch tables perform like giant iPads allowing visitors to drag, zoom, search and shuffle copious amounts of documents at will.

Fig 13. The Dunbrody- a technological famine

Fig 14. Editing on the fly at Kilmainham Gaol

Human engagement is provided at ‘Discover’ by large format flat screens adjacent to the document display cases. Sensor beams trigger videos of relevant conservators and curators talking about their relationship with key objects in the cases. The videos change as artefacts are replaced on a regular basis for conservation reasons. 2.3.3 Opinion Monitor at W5 The Opinion Monitor display in the W5 Science Centre in Belfast takes visitor engagement to the level of an exercise in local democracy. Opinion Monitor is an interactive cinema that records visitors reactions to a provocative range of expert opinions, and to controversial proposals to deal with topical health and environmental issues. The installation tabulates the results and emails them to local government, so as to influence its political decision


Fig 15. The Discovery tables - giant i-Pads

Fig 17. Opinion Monitor -an exercise in democracy

Fig 16. Sharing curatorial passions

2.3.4 Group Interactivity - The Comet Chaser Space Mission at Cosmos Technology in Museums can also offer human engagement to the social media generation with role playing activities and quests for large groups. At Cosmos, for instance, audiences can collaborate in a simulation of the European Space Agency’s Rosetta Mission in Ireland’s first immersive digital cinema. They can work together to track and divert a comet threatening the Earth. They can consult with Mike McKay in ESA mission control, and others, at critical stages. The material composition and track of the Comet is different every time. Audiences must decide whether to gently nudge the Comet into different orbit, or melt it with a thermal explosion, and so forth.

Fig 18. Save Blackrock Castle Observatory in Cork from an incoming Comet


Fig 19. Museums can use technology to provide human engagement in problem solving groups


Engagement on the move - The Ireland Pavilion at EXPO 2010

Fig 20. Ireland Pavilion- A mirrored video wall pushed Irish creative endeavour to infinity.

In 2010 Shanghai hosted 70 million visitors at the World’s largest EXPO. The theme was ‘Better Cities – Better Life’. The Ireland Pavilion, a lantern-like building embedded into a grassy mound, symbolised the relationship between urban and rural, manmade and natural.


The exhibition brief called for a presentation of the beautiful landscape, ancient history, vibrant culture, and technical innovation of Ireland. The challenge was that 18,000 visitors per day would have to be kept continuously moving through the 1,600 sqm spiral of narrow ramped exhibition corridors. 90% of the visitors were Asian. The ‘Messages’ would have to be absorbed without spoken commentary. Written picture captions, traditional AV presentations, and interactive screens would cause people to stop in their tracks. This would causes mayhem. Digital projections, symbolic doorways, large format images, dressed sets, driving sound and music were all used to draw visitors in, and keep them moving forward. The atmospheric light of Ireland was evoked by projections of wind, clouds, falling rain and flowing water. Large banners announced headline information. Detailed information and extended picture captions in both English and Chinese could be obtained on the move from the Pavilion’s mobile website, and by downloadable Apps, tailored for mobile phones popular in China. The Paris based BIE (International Exhibition Bureau) - awarded the Ireland Pavilion the BIE Bronze Medal for: ‘enhancing the EXPO message’ with ‘innovative visions’ and ‘environmental solutions’ presented with ‘clarity and intelligibility’.


Principle Four – Authenticity

Fig 21. Projections evoked the Irish climate

Fig 22. The Pavilion hosted 3 million visitors

The principle of using technology appropriately by designing exhibits and exhibitions that resonate with their surroundings is often overlooked. The hi-tech Cosmos exhibition was threaded with great difficulty through the narrow awkward spaces and thick walls of a 16th Century castle. This was appropriate because the Castle houses the astronomical research unit of the Cork Institute of Technology. Real scientists doing genuine space research within the Castle gives the Cosmos its authenticity. The exhibits reflect the aims and importance of their work.


2.4.1 Oriel Culture Centre at Dundalk Gaol When Comhaltas Ceoltoiri Eireann converted one wing of Dundalk into a culture centre and performance venue for traditional Irish music, all of the prison cells were converted into offices, recording spaces, and sound archives. Comhaltas was determined to use the central gallery to celebrate the fascinating social and political history of the gaol. Technology was used to restore the authentic atmosphere to the experience of a prison with no cells. The projection systems for the dance stage features moving mirrors so that random moving silhouettes of warders and inmates, provide a ‘ghostly’ atmosphere. Patterns of feet on the dance stage convey to visitors the Irish dancing that goes on at night. 75 mm format video screen are mounted behind the ‘warder’s peepholes’ in the blocked cell doors. These allow visitors to watch mini-dramas involving a range of historically researched and authentic virtual inmates incidents that include: a Famine-era beggar; a Land League agitator; an Independence War escaper, a Civil War condemned leader, and so on. 2.4.2 The Proclamation Display in the General Post Office in Dublin The General Post Office in O’Connell Street is one of Dublin’s iconic buildings. An Post wanted a small museum to stimulate an appreciation of the history of postal services in Ireland. They also wanted an authentic evocation of the building’s role at the centre of the 1916 Rising. Digital elements of the ‘Letters, Lives and Liberty’ exhibition include: a touch screen that allows

Fig 23. Dancing feet

Fig 24. Ramdom ghosts

Fig 25. The Warder’s spy holes

Fig 26. A range of virtual prisoners

visitors to design their own stamps, and then email them to their own computer; a comprehensive digital database of Irish stamps; a touch screen, embedded into a writing desk, displaying a variety of historic letters; an original telephone exchange that invites visitors to connect the lines and listen; and a Travelling Post Office. The train window presents the passing countryside as a screensaver and visitors can select various digital AVs.


The climax is the Proclamation which uses a digital projection onto an invisible stretched film to create a 3D ‘Peppers Ghost’ spatial effect to bring to life real events during 1916. This room is set dressed with rubble, scorch marks and sandbags to show its state immediately after the 1916 Rising. The projection magically restores the Telegraph Room to its pristine state on the first day of the Rising. The 3D costume drama depicts postal workers persevering amidst the bloody mayhem of the insurrection.

Fig 28. A travelling Post Office

Fig 29. Design and email your own stamps

Fig 27. Holographic figures from the GPO’s violent past appear to re-inhabit the space


Principle Five - Resonating with Visitor Needs

Martello’s fifth principle is that, whatever the brief, full attention must be paid to the visitor’s agenda. A trip to any museum, or visitor centre is only a small part of a tourist’s holiday trip, and/or, a family day out. What are their requirements? Is there something fun to distract children while their parents engage with serious content, or visa versa? Is there somewhere comfortable to park Granny with a nice cup of tea? What do the visitors need to know, independently of the client message?


2.5.1 The Ledge at Moher The re-landscaping of the Cliffs of Moher site necessitated barring resentful visitors from the popular, but highly dangerous overhanging rock ledge. Equally, tour buses roll up to the site, on schedule, regardless of weather conditions and visibility. Many tourists gain not one glimpse of Ireland’s most popular natural wonder. Martello felt that both these disgruntled groups deserved a consolation prize. Something that they would never experience, even in good weather. The Ledge Experience acknowledges that the cliffs belong the sea birds that live on its ledges. A black box 4-way projection space turns visitors into a Gannet. Starting on the famous ledge they can soar the cliffs in search of prey, before diving to the seabed for close encounters with basking sharks and humpback whales, at one to one scale. 2.5.2

New Entry and Exit at the Guinness Storehouse

The Guinness Storehouse is Dublin’s leading attraction. But it did not match many visitors expectations of a historical site. The ticket queues distracted from appreciation of the central Atrium’s giant pint glass shape. Escalators whisk visitors to the rooftop Gravity Bar where unrivalled panoramas make this Dublin’s Eiffel Tower. The exit experience, by contrast, was anti-climatic - a fire exit door, at the foot of a long enclosed fire stair. The Storehouse’s new entry and exit arrangements are a direct response to visitor needs and suggestions. Shadow silhouettes of early 1900s street scenes are projected on the Entrance Hall windows. The atriums pint shape was enhanced by the addition of a projection of a continually settling pint of Guinness. In the new Exit Hall visitors are thanked by Guinness surging on 72 plasma screens on twelve, 3-sided towers.


Conclusion As the power of information networks, and new digital technologies, expands in exponential inverse ratio to the costs, the potential for exciting new applications in museums, and the radical and improvement of existing ones, can bewilder. However, the basic aims and principles of good exhibition design are as relevant to these new technological opportunities, as they were to every previous innovation in interpretative display and storytelling. Martello continues to analyse every situation, and design opportunity, in terms of : ! ! ! !

Clarity Layering Engagement Authenticity 85



Martello has found these five basic concerns an invaluable compass for thinking about communications technology in museums in productive ways, that relate to both client, and user needs. Clarity on the basic aims and needs of every project lessons the danger of using new technology for its own sake, and increases the likelihood of developing interesting and useful applications that will vastly expand the role and scope of museums. both within their own spaces. and, in their global outreach.

Acknowledgements To all my colleagues in Martello whose imagination, skill, and dedication pushed the boundaries of the possible far beyond my own expectation, on all of the above, and so many other projects. Thank you Evelyn Fitzpatrick, Rob Reid, Clare Kavanagh, Rob Molenaar, Laura Murtagh, Peter Whittaker, Joanne Byrne, Maryanne Jardine, and Michelle Dempsey.

References Falk, J. (2009), Identity and the Museum Visitor Experience,: Left Coast Press Inc.


Promising beginnings? Evaluating museum mobile phone apps Maria Economou & Elpiniki Meintani Museology Laboratory, Department of Cultural Technology & Communication, University of the Aegean, Greece [email protected], [email protected] Abstract Since 2009 museums have started introducing mobile apps in their range of interpretative media and visitor services. As mobile technology continues to develop and permeate all aspects of our life, and the capabilities of smart phones increase while they become more accessible and popular, new possibilities arise for cultural institutions to exploit these tools for communicating in new ways and promoting their exhibitions and programmes. The use of mobile apps opens up new channels of communication between the cultural institution and the user, which extent to his or her personal space and go beyond the boundaries of the museum’s walls. The paper presents a survey carried out of mobile apps designed by art or cultural historical museums and analyses the wider issues which are raised by the findings. It discusses, among others, the kind of use these apps were designed to fulfil (e.g. the majority are guided tours to the permanent collections or to temporary exhibitions), the layering of content,and the type of user interaction and involvement they support.

1 Introduction The last decades have seen museums undergo important changes, moving towards the model of a more democratic cultural organisation which places great emphasis in communicating with different types of audiences in different ways. As Stephen Weil (2007) put it, museums have shifted the balance ‘from being about something to being for somebody’. Challenging the unique authority of the curator, they invited visitors to actively create their own meaning from the collections, encouraging existing audiences to interact in new ways with the objects, as well as reaching out to new audiences. In this process, museums have been experimenting with different strategies and practices, exploring, among others, the use of new technologies which were developing very fast, permeating every aspect of social life. After the earlier introduction of audio tours, in the 1990s museums started experimenting with digital mobile guides (e.g. the Minneapolis Institute of Art in the USA in 1994, and the HIPS/HIPPIE project in Europe in 1997). These allowed visitors to receive related information as they wandered around in the galleries independently and to their own will and developed from simple audio systems to fullyfledged multimedia ones which were location-aware. Several museums were attracted by the idea of allowing visitors to access information related to the particular context and the surrounding space, particularly as these devices evolved, offering an increasing number of options for colour presentation, incorporation of sound and video, memory capacity, long battery life, and wireless communication with a central system controlled by museum staff. The other attractive feature that mobile devices could offer with increasing sophistication and effectiveness was the ability to personalise the presentation of the content according to the users’ needs (Tallon 2008). The Handscape initiative of CIMI (Computer Interchange of Museum Information) which studied the existing and possible applications of mobile computing in museums, identified in 2002 as possible scenarios of use the role of mobile applications as: virtual guides, electronic maps, guides to the museum’s website, communication channels, ways of accessing the museum shop, and personal diaries for recording visitors’ impressions (Gay, Spinazze and Stefanone, 2002). Another of the early projects, the Electronic Guidebook of the Exploratorium in San Francisco, started initially with the idea of creating a pocket travel guide, to which visitors would be able to add personal notes (Semper and Spasojevic, 2002). This initial metaphor was extended to the role of a ‘mobile learning companion’, aiming to assist in the exploration of the museum exhibits before, during, and after the visit. The possibility to transfer texts, images, sounds and video through the mobile devices was used to offer information about the displays, but also to encourage visitors to extend their relationship with them by participating in taking readings, collecting data and other experiments for the


better understanding of the phenomena to which the exhibits referred. The evaluation of the Electronic Guidebook showed that the portable devices were successful in encouraging users to think about the exhibits and to look at them in new ways. Users often combined information that was given in both the real and the virtual environment. On the other hand, it also showed that it gave users a feeling of isolation. It is interesting that the teachers who participated in the evaluation thought that the content of the mobile devices and of the whole network would have been more useful for educational activities before and after the visit to the museum, rather than during (Hsi, 2002). Since then, several projects have been experimenting with some of these scenarios of use, as well as the potential of museum handheld devices as study guides and tools to assist learning in different forms. In 2005 Nancy Proctor listed 101 handheld and wireless cultural tour projects and the number has rapidly increased since then moving into uses beyong the museum tour to include games, storytelling, and other means of actively engaging visitors. Since 2009 we also started seeing the release of museum-related applications for mobile phones, known as mobile apps, the large majority of which were designed for Apple’s iPhone. Mobile telephony is one of the technologies with the greatest degree of permeation in our every day lives1. The last few years the technology, as well as the culture related to the use of mobile phones has evolved to such a degree that today the mobile phone, particularly in the case of smartphones, is not just a communication medium, but also a popular tool for social networking (e.g. allowing users to send SMS, participate in chat rooms, make use of acquaintance services), as well as a way of accessing information (e.g. news) and services (e.g. financial ones). The creation of mobile apps with museum content is a rapidly expanding area with several instutitions around the world experimenting with their potential, particularly their advanced computing abilities and connectivity. For museums which are continuously exploring new strategies for communicating with current and potential audiences, one of its most attractive features is that it opens the possibility for reaching new audiences through a personal device they have chosen and are familiar with, not only during their museum visit, but also before and after the visit, wherever the user chooses to be. This ability to reach users in conditions and at an environment of their choice opens up new possibilities for the communication of cultural content for life-long learning and edutainment, apart from the potential for cultural marketing. Additionally, the fact that these users are connected in a wide network offers possibilities not only for one-to-one communication between the cultural organisation and the user, but also for social networking and creating communities of users interested in cultural content, incorporating Web 2.0 capabilities. There is steady increase in smartphone sales in the last few years (e.g. the Nielsen Company (2011) reported that at the end of 2010 31% of U.S. mobile phone owners had a smartphone and expected smartphones to become the majority by the end of 2011, while the IDC Worldwide Quarterly Mobile Phone Tracker (2011) reported 74% increase in smartphone sales for 2010 compared to 2009). The literature (e.g. Tallon & Walker 2008, Proctor 2010, the June 2010 issue of the Museum Practice), related conferences (such as the Tate handheld conference, held annually since 2008) and online discussions on the use of mobile apps in museums is growing, but still focuses largely on the technical issues and challenges. But as their use steadily increases in the cultural sector, it is important to also examine their effect on the experience of the museum visit, the visitors’ perception of the cultural organisation, and museum and lifelong learning. There is also a need to evaluate the effect these tools have on museum staff and the internal policies and working practices of cultural organisations.

2 Survey of museum mobile apps 2.1 Methodolody In this context, we carried out a survey which recorded the mobile apps related to museum content, focusing more on arts and humanities collections (apart from three applications from a natural history museum which included an application of augmented reality, not yet employed in the apps of the other types of museums). The selection of the sample for the survey was undertaken in November 2010 (did 1

The International Telecommunication Union (ITU) estimated that by the end of 2010 there would be 5.3 billion mobile cellular subscriptions worldwide (more than double the number of subscriptions for 2005), while access to mobile networks is now available to 90% of the world population. In developed countries there were on average 116 subscriptions per 100 inhabitants, and in developing countries 73% at the end of 2010 (ITU 2010).


not include any museum apps released later) and was based on internet and bibliographic research (e.g. journal and conference papers, the 2010 Horizon Report: Museum Edition) on the topic, looking for specific case studies. The internet research was based on the following sources: • Online shops providing applications for smart mobile phones (iTunes, Android Market, Nokia Ovi, Samsung Apps, etc.), • Museum websites (for a more systematic access to these we used the following portals: Virtual Museum Pages-Vlmp2, MICHAEL Web Portal-Europe3, and the list of accredited museums from the American Association of Museums website4) • Online social media, such as blogs and wikis. In the survey we only included applications developed by the museums themselves or by the institutions where museums belong according to their organisational structure, as we wanted to investigate how the cultural organisations themselves are directly responding to these technological and social developments.

2.2 Findings 2.2.1 Number of museum mobile apps We identified 71 museum mobile apps with interactive and multimedia features (not just mobile phone acoustic tours), three of which are not related to a museum of art or culture5. Apart from these 71 applications, we identified another seven offering Augmented Reality (AR) features, four of which were using the Layar mobile browser6. Six of these seven AR apps were only accessible onsite and were therefore, not included in the survey7. One of the 68 art and social sciences apps with interactive and multimedia features identified was not accessible due to the researchers’ lack of related language skills8, two were only accessible at the museum premises9 that we were not able to visit at this stage of the research and one was not able to function due to technical problems10, so the final sample of the apps examined in greater depth was 64. 2.2.2 Year developed From the 69 mobile apps for which we could collect information about the year of development, the majority (60) were developed in 2010 (or had a latest version released in 2010), with only nine developed in 2009, showing clearly the recent spread of this type of technology and the museums’ related interest in investigating its potential.

2 4 5 Those of the American Museum of Natural History in New York. 6 The browser, first developed in 2009 by a Dutch company, allows users to find various items based upon augmented reality technology. It makes use of an in-built camera, compass, GPS, and accelerometer to identify the user’s location and field of view. From the geographical position, the various forms of data are laid over the camera view like additional layers. 7 The four apps using the Layar mobile browser are: i) The Layar: Augmented Reality Browsing of Powerhouse Museum around Sydney, Australia (, ii) the one offered during the Lowlands Festival in August 2010 by the Stedelijk Museum, Amsterdam, iii) the Urban Augmented Reality Amsterdam by the Netherlands Architecture Institute showing Amsterdam ‘as it will be, was, or might have been’ (, and iv) the Layer for Layar Reality Browser of the Andy Warhol Museum, USA. The three non-layar AR apps are: i) the Streetmuseum of the Museum of London, UK superimposing old images and historic photographs of London on parts of the contemporary city (, ii) Walking Through Time, a JISC-funded iPhone app that lets visitors to Edinburgh walk over historical maps provided by Landmark Information Group and the National Library of Scotland (, and iii) the Meanderthal, of the Smithsonian National Museum of Natural History, which is the only one accessible online, allowing users to transform their portraits into some form of early human 3

( The Incheon Museum app developed by the Incheon Metropolitan City Museum in South Korea available only in Korean.

8 9

The TAP application of the Indianapolis Museum and app of the Guggenheim Museum, Bilbao. The Brooklyn Museum Collection app was crashing in iOS4 and was withdrawn from iTunes by the Museum in December 2010 in order to launch a fixed version soon. 10


2.2.3 Countries The majority of the museum mobile apps we identified were developed by museums in the U.S.A., and in a european context, by museums in France, the U.K. and the Netherlands.

Country USA France United Kingdom Italy The Netherlands South Korea Spain Australia Austria Brazil Canada Colombia Germany Total

Number of museum mobile apps 27 19 9 3 3 2 2 1 1 1 1 1 1 71

Table 1: Number of museum mobile apps recorded per country 2.2.4 Distribution platform The large majority of these applications (63 out of 71) were designed for functioning exclusively on Apple’s iPhone smartphone (and its iPod touch and iPad devices). Of the remaining eight, four11 were also designed for mobiles running Google’s Android operating system, as well as the Apple devices, and one12 was designed for both iPhone and Blackberry. Only one13 of the 71 applications was designed for most mobile platforms apart from Apple. Two of the apps identified were web-based, operating on the user’s mobile web browser (on any type of smartphone) but were also available on the museums’ website, the app of the Dallas Museum of Art, and that of the Nelson-Atkins Museum of Art in Kansas City14. One of the reasons for this preference for iPhone apps might be related to museums’ limited finances which lead to the development of apps for only one platform. Although iPhones do not have the largest share of the smartphone market (about 16% worldwide in 2010 according to Gartner (2011), 26% in the U.S.A. according to Nielsen (2011), and 20% in the U.K. according to Stephens (2010b)), they did revolutionize the smartphone and popularise mobile apps, being the first to have a special mobile apps store, which is still the largest and more popular online store of its type (in October 2010, this included about 225.000 apps). Although most art museums typically do not have the in-house expertise, budget and staff to create iPhone apps, there are a lot of developers for iPhone apps that museums can use. 2.2.5 Fee The majority of the mobile apps examined (56 out of 71) were being distributed for free, indicating that these are considered part of the free provision of visitor services that the cultural organisations are providing. The fee for the remaining 14 apps ranged from 0.79 Euros to 5 dollars (the average fee was approximately 2.5 Euros, 2.7 Pounds and 3.8 dollars for the respective regions).


Two developed by the Brooklyn Museum, one by the Albert Kahn Museum and Gardens in France, and one by the Art Institute of Chicago. 12 From the Canadian Museum of Civilization. 13 The app of the Guggenheim Museum in Bilbao. 14 The TAP application of the Indianapolis Museum designed for iPhone and iPod touch can be downloaded onsite and was not included in the in-depth survey as we were not able to visit the Museum at this stage of the research.


2.2.6 Notification of availability of mobile app on website For only a small part of the mobile apps examined (16 apps from 11 related museums), there is a notification and a direct link to a related page from the museum’s homepage. For most of the apps examined (52 apps from 42 museums), there is no indication of their existence on the institution’s homepage. This raises issues of promotion and marketing of the apps, but also of wider integration in the organisation, particularly its communication and ICT strategy. 2.2.7 Type of application in relation to museum content From the 64 apps examined in greater depth, we identified the following six categories15:

Type of application Presentations – guided tours of permanent exhibitions and the museum in general Presentations – guided tours of temporary exhibitions and practical information about the museum visit Combination of the two above Apps devoted to a single object or artwork from the collection Content creation or manipulation from the user, inspired by artists’ work Games based on the exhibits

Number 29 20 5 5 3 2 64 Presentations - guided tours of permanent and temporary exhibitions Most of the mobile apps examined in greater depth (54 out of 64) take the form of a guided tour to specific exhibitions, collection highlights, or the whole museum through touchscreen icons or/and virtual pads. A different approach in this type of application is followed by Rijkswidget, the app of the Rijksmuseum in Amsterdam (Fig. 1). This presents a different object every day from the museum’s permanent collections, with accompanying information about the work, its creator, and the museum, with the possibility to zoom in on its image and to connect with the museum’s website for additional material. This encourages repeat visits to view other objects, unlike the other apps which cover a large number of objects but usually offering only basic information.

Fig 1. Rijkswidget, the app of the Rijksmuseum in Amsterdam, showing a different object from the collections every day Apps devoted to a singe object or artwork In this category we identified five apps. Their approach varied from presenting an object from the permanent displays (Tipu’s Tiger-V&A) or artwork temporarily on display (How It Is-Tate Modern, Brion Gysin’s Dream Machine-New Museum, NY), to a recent acquisition (Urban Light-LACMA), and an electronic publication related to an artwork from the permanent collection (Vincent Van Gogh’s Starry Night- MoMA).


The full list of mobile apps under each category can be found in the Appendix.


MoMA’s Van Gogh’s Starry Night is an e-book, the short version of a longer printed book published by the museum in 200816. In this case, the app is related to the distribution of pre-existent traditional interpretation material through a new digital medium (smartphone) and its release appears more like a marketing decision than a special user engagement programme based on the specific technology. The remaining apps in this category have a strong promotional and entertaining character, acting as ‘teasers’ by offering users a taste of the museum content in order to attract them to the whole display onsite. In these cases, the purpose is more to promote the museum’s work in an enjoyable way, rather than to engage users in depth with the content itself. One of the most interesting and promising examples in this section is the app How It Is of Tate Modern (Fig. 2), related to Mirslaw Balka’s commissioned work with the same title which was shown at the Turbine Hall in 2009-10. It is an augmented reality audio tour of the specific artwork with a highly interactive17 and explorative character. Its approach is more experiential than interpretational, since it is based on the user’s personal experience of and ‘immersion’ in a virtual environment, than a fixed presentation of the artwork. The fact that the app takes the form of an immersive game, quite similar to popular video games, can contribute even more to attracting young visitors to the exhibition. The interactive character of the application, as well as its entertaining features (with a secret game unlocking to users when they open their mobiles when they visit the work in the gallery) can contribute to audience development.

Fig. 2 Tate Modern’s How It Is app

16 17

It can be purchased with a fee of $3.99, while the rest in this category are all distributed freely. Based on 3D animation, 3D sound and a virtual joystick.

92 Content creation or manipulation from the user, inspired by artists’ work Three applications were identified in this category, Muybridgizer (Tate Britain), In Still-Life 20012010 (LACMA), Art-Me (MASP). Tate Britain’s Muybridgizer (Fig.3) is an app developed for the temporary exhibition on early photographer Eadweard Mybridgizer (08/09/2010-16/01/2011). It gives users the opportunity to take photographs and then use the app to create images in the style of Muybridge’s work, sepia toned ‘freeze frames’ of moving subjects, which they can store and share through flickr.

Fig 3 Tate Britain’s Muybridgizer app

Fig 4. LACMA’s app John Baltessari: In Still-Life 2001-2010 where the user can position the elements on the painting Los Angeles Museum’s of County Art app, In Still-Life 2001-2010 (Fig.4), designed by contemporary artist John Baldessari, has a highly interactive character inviting users to re-compose onscreen a Dutch painting of 17th century by arranging through the touch interface the 38 objects in the artwork (each of which has symbolic meaning). Both these apps offer users a more active role, encouraging them to create their own works, inspired by the specific artists. Both apps are quite entertaining (especially Muybridgizer) and offer opportunities for interaction with the museum content, through


experimentation, creation, user’s self-expression, and some learning about the history of photography and 17th-century Dutch painting respectively. They are also serving the promotion of museum work since they were launched on the occasion of related temporary exhibitions and their distribution is free. The Art-Me app of the Museum of Modern Art of Sao Paolo in Columbia allows the transformation of photographs taken by the users to works of art in the style of Rembrandt and Van Gogh. This is a promotional and entertaining app with no extra interpretation material. Apart from encouraging the creation of personal artworks based on museum work, all three apps store these in the user’s own device and allow their sharing through email or social networks18, thus promoting the feeling of familiarity and ownership of the cultural material and related social interaction. Games based on the exhibits We identified two applications in this category. The British Museum’s Book of the Dead is an interactive knowledge game related to the temporary exhibition ‘Journey through the afterlife: ancient Egyptian Book of the Dead’ (04/11/2010-6/3/2011). Through this app the users are led to the Underworld by Anubis, the ancient Egyptian god of embalming and are asked questions, which, if answered correctly, will unlock one of the Books of the Dead presented in the exhibition and lead them to the Underworld. The Tate Trumps (Fig. 5) app of Tate Modern is a game with digital cards during which users explore the exhibits in the museum, ‘collect’ some of them, and win points. The game can be played by up to three players or groups of players. This application encourages users to explore the works on show and interact socially with their co-participants in the physical environment of the museum.

Fig 5. Tate Modern’s Trumps app – a digital cards game 2.2.8 Use of the apps in relation to the museum visit All the apps, except the ones designed for exclusive use in the museum, can be used before, during, and after the visit to the museum or independently from the visit, enriching and assisting the museum experience but also extending this experience beyong the museum walls. Although there is no obstacle to all these different ways of using the apps, the way of distributing them, their design and content appear to encourage specific type of use. • Use before the visit to the museum None of the apps studied was designed for exclusive use before the museum visit, but most of them offered practical information which encourages the use of the app for preparing the museum visit, such as opening hours, admissions, access maps, etc. There is potential for exploring this further in the future, particularly for assisting life-long learning, school projects, etc.


Muybridgizer: Fickr, Email; In Still Life 2001-2010: Flickr, Facebook, Twitter, Email; Art-Me: Facebook, Email.


• Use during the museum visit Some of the applications can only be downloaded once the user enters the museum space (e.g. the TAP app from the Indianapolis Art Museum) and aim to enrich the experience of the visit. But even in the case of apps which can also be downloaded off-site, the majority are best suited for use during the visit, due to their design and content. These apps have navigation structured according to the spatial arrangement of the exhibits in the museum, include interactive or simple floor plans of the museum’s exhibition spaces with the exhibits marked, or offer activities for enriching the museum visit such as Gallery Tag! of Brooklyn Museum. Furthermore, in some cases the use of these apps is encouraged in the museum by orientation services which take advantage of users’ location tracking technologies19, and the incorporation of a keypad for selective access to audio narratives about particular exhibits20. In many apps the whole approach which offers simple visual material (images) with audio narrative, refers to the traditional audio tour. • Use after the museum visit Even though no app we studied was designed for exclusive use after the visit, a lot of them included features which offer opportunities for studying and processing the museum material in this way (such as the tagging of favourite material and the possibility to store content such as photographs on the user’s mobile). 2.2.9 User’s interaction with the content Layering of content is important for applications that are shown on very small screens and are addressing a varied audience with different levels of interest in the collections and the exhibition themes. In the applications examined (which follow the guided tour paradigm), this aspect ranges from minimal, with apps offering cards with images and interpretative text or narration (e.g. the app of the Musée Jacquemart Andre, Paris on 18th century painters) to adequately layered (e.g. the SFMoMA app with a special “Go Deeper” interface) (Fig 6).

Fig 6. The SFMOMA’s app provides a “Go Deeper” button which assists the layering of content Additional features that also assist in the provision of different layers of information are the various ways of navigating (e.g. spatial, chronological), the availability of images of high resolution and magnification, the linking and correlation of the system’s content (e.g. MoMA) and any link to additional online material (e.g. Brooklyn-Wikipedia, MoMA podcasts, MoMA i-Tunes U, related material at the organisation’s website). Almost all the tour type of apps examined take advantage of the capabilities of smart mobile phones for high level of zooming via touch, offering images of high resolution and magnification suitable for examining details of an object. This gives the impression of bringing the works closer to the user, 19

Brooklyn !useum and Explorer-AMNH. SFMoMA; Asian Art Museum; Brooklyn Muse-Brooklyn; De Cordova Sculpture Park and Museum; Houdini; Monet- Grand Pallet; Baba Bling-Musée de Quai de Branly; MdbK Kunst Begleiter-Museum der bildenden kunste Leipzig; Portland Museum; Rubens, Poussin et les peintres du XVIIeme siècle- Musée Jacquemart-Andre; Phillips Collection, USA. 20


offering views which are often not possible to acquire in other cases. The access to details of the objects gives added value to the apps and enriches the user’s experience encouraging personal exploration. We should note here, however, that the magnification capabilities offered vary between the different applications, ranging from small magnification (e.g. Musée National Marc Chagall) to large one (e.g. Uffizi). Another feature offered to users to assist their interaction with the content is the tagging of material as ‘favourite’ or the use of bookmarks and the creation of related personal collections. We identified eight apps with related features21. The specific feature of the MoMA’s Van Gogh’s Starry Night app is enriched with the possibility of attaching to the favourite objects, notes by the user, reinforcing in this way users’ interaction with the material. These features include several possibilities for enriching the user’s experience and for personalised learning through the personal and emotional linking of the user with the museum material. They also promote the use of the app after the museum visit, offering users the opportunity for further exploration and study of the bookmarked content, e.g. at home, extending the museum experience beyond the museum walls. The applications MoMA Snaps and Share of the Rooftop Garden-SFMoMA allow users to take photographs in the museum spaces, store them, and distribute them via email. This feature promotes user’s active exploration of the museum content and their creative expression, as well as creating a feeling of familiarity and intimacy about the museum visit. Another feature which allows users to interact with the museum content at the museum spaces is Gallery Tag! of the Brooklyn Museum app. This is an interactive game where users tag museum objects aiming to win points and awards. It is based on exploration and discovery of exhibits which, according to the players, fit to a series of predetermined tags. The game includes roaming services (Roam!), encouraging visitors to find and tag objects from different floors of the museum and rewarding them with additional points, linking to the mobile version of the museum’s website (Crossover!), and finally, convergence of the game’s tags with the online collection (Covergence!), thus bridging the physical with the virtual collection22. This game app has the potential to contribute to experiential learning and the personal construction of meaning through the active participation of visitors, while strengthening the entertainment potential of the museum visit. 2.2.10 Integration of multiple perspectives Only a few of the tour type of apps investigated strove to integrate multiple perspectives, apart from the curator’s or the official museum view. Seven of the apps attempted this, some to a greater23 and others to a lesser extent 24, incorporating comments from conservators, the artists themselves, directors, family members of the creators, and in one case (SFMOMA’s Rooftop Garden), a musical piece inspired by the museum’s artwork. 2.2.11 Social interaction From the 64 mobile apps examined in greater depth, 19 offered features supporting social interaction. Most of these (16) relate to the sharing of museum content, commenting, evaluating and tagging material as ‘favourite’. Sharing is related to content of either the app itself (video, images, etc.) or that has been created by the users (photographs, comments, evaluations, lists of favourite material, messages, etc.), or both. Thirteen apps use as sharing platform the online social network websites (facebook, Twitter, Flickr) (!"!#-YouTube) and email25, while three museum mobile apps use ‘walls’ for posting messages26. 21

The Asian Art Museum smartour; Brooklyn Museum’s Like This !: feature allowing visitors to share favourite exhibits; Vincent Van Gogh’s Starry Night-MoMA; Monet-Grand Palais; Houdini-Jewish Museum; Uffizi; Galleria del’ Academia; SD2010-Sydney Design-Powerhouse Museum. 22 The description was based on information from the museum’s blog, as access to this part of the application is only possible in the museum: [15/11/2010] 23 E.g. Rooftop Garden- SFMOMA; Yours Vincent, The Letters of Vincent Van Gogh-Van Gogh Museum; MoMA’s main app; Gauguin: Maker of myth-Tate Modern. 24 E.g. Love Art-London National Gallery; Quilts:1700-2010-V&A Museum 25 Rooftop Garden-SFMoMA; Asian Art Museum Acoustic Smartour; Brooklyn Museum Mobile; Houdini-The Jewish Museum; In Still Life 2001-2010, An Interactive Artwork-LACMA; French Impressionism-Art Institute of Chicago; Monet-Le Grand Palais; Musée de Quai de Branly; Muybridgizer-Tate Britain; Hunterian Museum; Art Me-Museu de Arte de Sao Paulo; Art Museums in Seoul; SD2010-Powerhouse Museum. 26 Graphic Design Museum-Breda, the Netherlands; Wim Delvoye au Musee Rodin; Sala Parpallo, Valencia, Spain.


Additionally, we found two apps with an interface for connecting the user with the museum’s social networks27. We also found one app (Tate Trumps) which supports social interaction through group collaboration, dialogue and social contact28. Finally, another feature of Brooklyn Museum’s Gallery Tag!, which allows users’ to tag exhibits from the online museum collection, has also possibilities for social tagging of the museum content which should also be studied further in the future.

3 Discussion Because of the early and experimental stage of integrating these technologies in the museum work, a number of issues have risen related to the content, design and use of these apps, but also of their overall impact on the image of the museum and the whole experience of the visit and/or the use of the cultural material (in the case of remote only access). As with every new technology, also in the case of mobile apps, complex issues emerge about the development and maintenance of a new business model by cultural organisations, especially in this case where the mobile telephone environment changes rapidly. The fragmentation of ‘mobile internet’, with multiple platforms for access and navigation is feeding a heated discussion about the selection of distribution platforms for mobile apps and is discouraging smaller or less technically-savvy museums from exploring the possibilities of mobile apps. The new technologies of mobile telephony have the potential for supporting museum communication as a holistic and interactive cultural process. However, our survey showed that this is not always fulfilled. The majority of the apps examined relate to guided tours and presentations of permanent or temporary exhibitions with the following basic characteristics: • Navigation according to the spatial layout of the exhibits or their chronological or alphabetical order, • Minimal exploitation of technologies of location tracking and guidance, • Limited connectivity with the museum’s online presence • Satisfactory layering of content • Limited effort to incorporate different and interdisciplinary perspectives • Interaction focused mainly on great magnification of images • Social interaction almost exclusively through the sharing of content on online social networks and email, • Use mainly during the museum visit. The majority of the museum apps developed so far have the form of enriched audio tours, in some cases following the model of traditional tours (linear exploration, use of reference images of the work with audio commentary). We observe here the transition from the portable audio tour device to the multimedia device (PDA, iPod, tablet, smartphone), with intermediary stage the distribution of audio tours on mobile phones (cell phone acoustic tours), resulting in the emergence of new challenges in the context of the automated museum experience (Smith 2009). Although the technology has changed, the mentality related to the design of the experience appears in many cases to have remained the same to the one which produced the acoustic tours. It is necessary to shift from the approach that the device determines the content, to a new approach, where the device is a medium in the process of creating content, but not the decisive factor (Smith 2009). One of the reasons there is still scepticism about the use of smartphone apps in museums is that it carries the risk of overshadowing the exhibits and the museum content itself. Because of their strong mobile and multimedia character and the fact that they are usually displayed on the visitors’ own, familiar devices, this is often stronger than with other ICT interpretational means (e.g. infokiosks). This risk raises questions of design and content of these systems and their integration with the surrounding space and exhibits, which also need to be further investigated in the future. As the technology evolves, but also our thinking about its potential in a cultural environment, the discussions about the adoption of mobile devices by museums need to relate less to the technical issues and more to the variety of content and services that these can offer, and particularly the ways in which these apps can encourage multiple and meaningful connections between people, organisations and 27

MOMA-YouTube and Hunterian-Wikipedia. Also, the feature Gallery Tag! of the Brooklyn Museum app mentioned above, even though it supports individual exploration of the museum exhibits, might also support group interaction, but the limited off-site access to its content does now allow us to test this further at this initial stage of the research. 28


collections (Johnson et al 2010). As attitudes of both museums and visitors are changing, “people are becoming used to getting information on a museum whenever and wherever they want” (Nancy Proctor quoted in Stephens 2010a). Proctor highlights as very important the changes in the nature of the mobile content and the experiences being designed, which she identifies as far more dialectical than earlier generations of mobile narrowcasting (Stephens 2010a). A series of further questions which need to be examined relate to the manipulation of content by the user and its integration in the application, the nature of the interaction between user-exhibit which is mediated by the mobile apps, the understanding of the different models of use for constructing different museum experiences29. Few of the apps we examined were found to support social interaction, a fundamental aspect of contemporary museum communication but also of cognitive development. Most of these accommodate it through content sharing on online social networks or/and via email. The design and content of most apps supports individual and not group use. This fact supports the fear of museum professionals that the use of these applications can isolate users and degrade the social dimension of the visit. However, this might have to do more with the way these apps have been designed until now and not with the inherent limitations of the technology. It is therefore, imperative to experiment with the design of museum apps for groups, such as families, school parties, etc. investigating the various levels and types of interaction that these systems can support for different visitor groups. Our study carried out at this first stage of the research an initial mapping of the field, identifying apps by art and cultural historical museums and examining them off-site. Further work is needed involving the analysis of the use of the apps on-site, but also recording the views of both museum staff and users. In order to study the complex set of parameters involved, a wide range of methodological tools would be required, and in some cases, innovative approaches, while taking into account ethical issues (such as the need to respect user’s privacy). The development of these applications by museums is undoubtedly a development in the exploitation of new technologies, with positive results in the promotion and understanding of their collections. However, in our days the discussion about the museological exploitation of these apps, both from the side of museums as well as that of users, includes more questions than answers because of their recent integration in museum practice. Further exploration in different directions, some of which we highlighted here, is necessary for the formulation of a wider interpretative strategy, where these apps would be included in the museum’s portfolio of available interpretative media in a constructive way and would be used according to their potential and special characteristics, supporting the museums’ mission and enriching the use of cultural material.

Appendix –Museum mobile apps examined listed by type of application Presentations – guided tours of permanent exhibitions and the museum in general AUSTRIA CANADA COLOMBIA FRANCE

1. 2. 3. 4. 5. 6. 7. 8.

Kunsthistoriches Museum Wien, Austria Canadian Museum of Civilization AMBO, Museo de Arte Moderno de Bogota Chateau de Versailles, Versailles LaM, Lille Metropole Musée d'Art Moderne, d'Art Contemporain et d'Art Brut Le Grand Palais, Paris Musée Cluny, Musée National du Moyen Age, Paris Musée de Quai de Branly, Paris


An interesting example of an app that integrates user-generated content in itself, seems to be the custom-built iPhone app ‘Scapes’, which we were not able to examine in depth as it is designed to work fully online. This is designed by H. Burgund, a musician and sound artist, specifically for DeCordova Sculpture Park and Museum, Lincolc, USA ( [07/04/2011]). Scapes is based on two-way communication that encourages visitors not only to listen, but to make their own recordings as they wander through the Park, that are then incorporated into the sound-part for everyone to hear. As visitors navigate the Sculpture Park with the phones, they can hear location-specific voices, music, other visitors’ thoughts, and sounds that change as they move. In this case, the app has the potential of keeping the visitors’ experience sociable, constructive and engaging.



9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29.

Musée des Baux Arts de Cambrai, France Musée du Louvre, Paris Musée National Fernand Leger de Biot, France Musée National Marc Chagall, Nice The Official Guide, Accademia Gallery, Florence The Official Guide, Uffizi, Florence Art Museums in Seoul, Seoul Sala Parpallo, Valencia GDM, Graphic Design Museum, Breda, The Netherlands Rijkswidget iPhone, Rijksmuseum, Amsterdam Hunterian Museum, University of Glasgow Love Art, National Gallery, London Acoustic Smartour, Asian Art Museum, San Fransisco Dallas Museum of Art (web app) De Cordova Sculpture Park and Museum, Lincoln Infinity of Nations, National Museum of American Indian, Smithsonian Fowler Museum, Fowler Museum at UCLA, Los Angeles French Impressionism, Art Institute of Chicago MOAS, Museum of Arts and Science, Daytona Beach Rooftop Garden, MoMA San Francisco The Nelson-Atkins Museum of Art, Kansas City (web app)

Presentations – guided tours to temporary exhibitions and practical information about the museum visit AUSTRALIA



2. 3. 4. 5. 6. 7. 8. 9. 10.


11. 12. 13. 14. 15.


16. 17. 18. 19. 20.

SD2010, Powerhouse Museum, Sydney, ‘SD2010-Sydney Design 2010’, 13/07/2010 – 15/08/2010 Baba Bling, Musée de Quai de Branly, Paris, ‘Baba Bling’, 05/10/2010 – 30/01/2011 Bruegel, Memling, Van Eyck, Musée Jacquemart-Andre, Paris: Exposition 'Bruegel, Memling, Van Eyck’, 11/09/2009 – 11/01/2010 Clichés Japonais, Albert Kahn, Musée et Jardins, Boulogne-Billancourt, Clichés Japonais, 09/11/2010 – 28/08/2011 L’Or des Incas, Pinacotheque de Paris, ‘L' Or des Incas’, 10/09/2010 – 06/02/2011 Monet, la Visite: expo Grand Palais, Paris, ‘Monet’, 21/09/2010 – 24/01/2011 Musée Jacquemart-Andre, ‘Rubens, Poussin et les peintres du XVIIeme siècle’, 24/09/2011 – 24/01/2011# Pinacotheque de Paris, retrospective of all the temporary exhibitions Une ville pour l’ Impressionisme, Musée des Beaux-Arts de Rouen, ‘Une ville pour l' Impressionisme: Monet, Pissaro et Gauguin a Rouen’, 04/06/2010 – 26/09/2010 Une visite comme si vous y etiez, Centre Pompidou-Metz, exhibition: ‘Chefsd’oeuvres’, until 04/07/2011 Wim Delvoye au Musée Rodin, Paris, ‘Wim Delvoye’, 16/04/2010 – 22/08/2010 Le Scuderie del Quirinale, Rome, ‘Caravaggio Exhibition’, 20/02/2010 -13/06/2010 Yours Vincent, Van Gogh Museum, Amsterdam, ‘Yours Vincent, The Letters of Vincent Van Gogh’, 09/10/2009 – 03/01/2010 Gauguin, $#$% !odern, London, ‘Gauguin: Maker of myth’, 30/09/2010 – 16/01/2011 Quilts 1700–2010, Victoria and Albert Museum, ‘Quilts 1700–2010: A close-up, stitch-by-stitch look at British quilting’, 20/03/2010 – 07/07/2010 Dutch Utopia, Telfair Museums, Savvanah, exhibition ‘Dutch Utopia, American Artists in Holland, 1880-1914’, 01/10/2009 - 10/01/2010 Houdini, The Jewish Museum, NY, exhibition ‘Houdini: Art and Magic’, 29/10/2010 – 27/03/2011 iAfrica, Minneapolis Institute of Arts, exhibition ‘iAfrica: Connecting with SubSaharan Art’, 3/10/ 2009 – 18/04/ 2010 MoMA San Francisco, exhibition ‘Calder to Warhol: Introducing the Fisher Collection’, 25/06/2010 - 19/09/2010 Yves Klein, Hirshhorn Museum and Sculpture Garden, Smithsonian, Washington, D.C., ‘Yves Klein: With the Void, Full Powers’, 20/05/2010-12/09/2010


Combination of the two above GERMANY USA

1. 2. 3. 4. 5.

MdbK Kunst Begleiter, Museum der bildenden kunste Leipzig Brooklyn Museum Mobile Phillips Collection, Washington, D.C. Portland Art Museum !o!#, NY

Apps devoted to a single object or artwork UK

1. 2.



4. 5.

The How it Is by Polish artist Miroslaw Balka, Unilever Series Commission for Tate’s Modern Turbine Hall, 13/10/2009 – 05/04/2010 Tipu’s iTiger, V&A, London: object from permanent collections, part of exhibition ‘Maharaja, The Splendour of India’s Royal Courts’, 10/10/2009 – 17/01/2010 Brion Gysin’s Dream Machine inspired by Gysin’s artwork Dreamachine (1961), launched on occasion of Gysin retrospective exhibition in New Museum, NY 03/07/2010 – 03/10/2010 Urban Light app: developed on occasion of first anniversary of installation of Chris Burden’s work Urban Light at LACMA, (February 2008) Vincent Van Gogh’s Starry Night-MoMA: based on the museum’s homonym book, by the art historian Richard Thomson

Content creation or manipulation from the user, inspired by artists’ work BRASIL UK

1. 2.



Art-Me, Museu de Arte de Sao Paulo Muybridgizer, Tate Britain, London, exhibition ‘Eadweard Muybridge’, 08/06/ 2010 – 16/01 2011 John Baldessari, In Still-Life 2001-2010, Los Angeles County Museum of Art, exhibition ‘Baldessari: Pure Beauty’, 27/06/ 2010–12/09/ 2010

Games based on the exhibits UK

1. 2.

Book of the Dead, British Museum, London, exhibition ‘Ancient Egyptian Book of the Dead’ 04/11/2010 – 06/03/2011 Tate Trumps, Tate Modern, London

References Gartner (2011) “Gartner Says Worldwide Mobile Device Sales to End Users Reached 1.6 Billion Units in 2010”, 9 February 2011, available online at, last accessed 20/2/2011. Gay, G., Spinazze, A. and Stefanone, M. (2002), “Handscape: Exploring potential use scenarios for mobile computing in museums” Cultivate Interactive, 8:15, November 2002, available on line at, last accessed 28/10/20010. Hsi, S. 2002, “Evaluation of Electronic Guidebook Mobile Web Resources, Executive Summary”, 29 July 2002, available on line at, last accessed 10/2/2011. IDC Worldwide Quarterly Mobile Phone Tracker, (2011), 27 January 2011, available online at§ionId=null&elementI d=null&pageType=SYNOPSIS, last accessed 03/10/2010 ITU (International Telecommunication Union) (2010), “The World in 2010. ICT facts and figures”, Geneva: ITU, available online at, last accessed 20 February 2011. Johnson, L. Witchey, H., Smith, R., Levine, A. and Haywood, K., (2010), “The 2010 Horizon Report: Museum Edition, Austin, Texas: The New Media Consortium”, available on line at, last accessed 20/2/11.


Museum Practice (2010), journal published by the UK Museum Association, June issue, available online at:, last accessed 16/3/2011. The Nielsen Company (2011), “Among mobile phone users, Hispanics, Asians are most-likely smartphone owners in the U.S.”, 1 February 2011, available online at, last accessed 20/2/2011. Proctor, N. (2005), “Off base or On Target? Pros and Cons of wireless and location aware applications in the museums”, in J. Trant and D. Bearman (eds) ICHIM (International Cultural Heritage Informatics Meeting) 2005, Paris, France, available online at, last accessed 20/2/2011. Proctor, N., (2010), “The Museum Is Mobile: Cross-platform Content Design for Audiences on the Go”, in J. Trant and D. Bearman (eds) Museums and the Web 2010: Proceedings. Toronto: Archives & Museum Informatics. Published March 31, 2010, available online at, last accessed 20/2/2011. Smith, K., (2009), “The Future of Mobile Interpretation”, in J. Trant and D. Bearman (eds) Museums and the Web 2009: Proceedings, Toronto: Archives & Museum Informatics, available online at, last accessed 20/2/2011. Stephens, S. (2010a). “The growth of mobiles app”, Museum Practice, 15 June 2010, available online at:, last accessed 15/3/2011. Stephens, S. (2010b). “Building an app”, Museum Practice, 15 June 2010, available online at:, last accessed 15/3/2011. Tallon, L., (2008), “Introduction: Mobile, Digital and Personal” in Tallon, L. & Walker (eds), Digital Technologies and Museum Experience, Handheld Guides and Other Media, AltaMira Press, UK, xiiixxv. Tallon, L. & Walker (eds), Digital Technologies and Museum Experience, Handheld Guides and Other Media, AltaMira Press, UK. Tate Handheld Conference 2008, 2009, 2010, [last accessed 20/2/2010.




Weil, S. (2007). “From Being about Something to Being for Somebody: The ongoing transformation of the American museum”, in Sandel, R. And Janes, R. (eds), Museum Management and Marketing, Routledge: London, 30-48.



Museum Guide 2.0 – An Eye-Tracking based Personal Assistant for Museums and Exhibits Takumi Toyama, Thomas Kieninger, Faisal Shafait, Andreas Dengel German Research Center for Artificial Intelligence

Abstract This paper describes a new prototypical application that is based on a head mounted mobile eye tracker in combination with content based image retrieval technology. The application, named “Museum Guide 2.0”, acts like an unintrusive personal guide of a visitor in a museum. When it detects that the user is watching a specific art object, it will provide audio information on that specific object via earphones. The mobile eye tracker thereby observes the visitors eye movements and synchronizes the images of the scene camera with the detected eye fixations. The built in image retrieval subsystem recognizes which of the art objects in the exhibition is currently fixated by the users eyes (if any). Challenges that had to be faced during our research are the modifications of the retrieval process utilizing a given fixation for better accuracy, the detection of consciousness when looking at one specific object as trigger event for information delivery and to distinguish from noise (unconscious fixations). This paper focuses on the application aspect of Museum Guide 2.0. It describes how a database of given art objects is created from scratch and how the runtime application is to be used. We end with a user study that has been conducted to evaluate the acceptance of the system, specifically in contrast to conventional audioplayer based approaches.

1 Introduction When tourists visit a Museum or a historical site, they need more information about the exhibits they are visiting. This information is usually provided by a trained professional (tourist guide). However, professional guides cannot cater the needs of all tourists. Therefore, automated personal guides are usually provided to visitors in Museums or archaelogical sites to aid them in getting more information about their exhibits of interest. Traditionally, these guides are provided as pre-recorded audio tapes, where the user can select and listen to an audio track corresponding to the exhibit of interest. Recent advances in the fields of object recognition, augmented reality, and virtual reality have led to the development of many interesting ideas for enhancing user experiences when visiting a Museum. The CINeSPACE project [Santos et al. 2007] aimed at designing and implementing a mobile rich media collaborative information exchange platform. The main idea was to enable users to interact with location-based multimedia contents while navigating a city. Audiovisual information was delivered through a portable low-cost wireless high definition near-to-the-eye display and audio phones. The audio-visual information focused on the cities' culture, history, tourism and art accessed through film heritage played out in a mobile virtual environment. The EU project AGAMEMNON aims at providing a visitor to a site of historical interest with personalised, information enriched experience through 3G cellphones and at the same time contributing to the preservation of cultural heritage [Agamemnon]. In the recent years, eye tracking and image based object recognition technologies have reached a certain degree of maturity which encouraged us to develop a new application on top of the technological basis of. Museum Guide 2.0 (also MG2.0 - the “2.0” like to express a next generation of museum guides) is an application that enhances your experiences in a museum. This application integrates eye tracking technologies and object recognition technologies to detect user's gaze on specific exhibits which can be observed when the user watches these exhibits with interests. In our scenario, a visitor to a museum wears a head mounted eye tracker whilst strolling through the exhibition (see Figure 1). As soon as gaze on a specific exhibit is detected, the application plays an audio file that provides additional information to the user about the exhibit.


Fig 1: Sample scenario of Museum Guide 2.0 MG2.0 communicates with the eye tracker system called “iViewXTM HED” [SMI] using a lean communication protocol called “UDP”. A simplyfied model of this application is shown in Figure 2. Images from the scene camera and eye tracking data are sent from eye tracking software iViewXTM to the MG2.0 application and our built in object recogtion algorithm then judges whether the user's gaze exists on a specific object. If it is the case, MG2.0 starts to play prerecorded data of the specific exhibit.

Fig 2: Brief model of Museum Guide 2.0

2 Museum Guide 2.0 Runtime System The human eye is characterized by very frequent jumps from one point to another and only short timespans for which the eyes rest on a specific point. This resting of the eyes is what we refer to as fixations and the fast movements from one fixation to another are called saccades. The main purpose of the eye tracking software is to detect fixations and to synchronize the viewing direction with the images that are recorded from the scene camera. Therefore the eye tracking software needs to be calibrated for each new user first. Then, the runtime application of MG2.0 must be started. After succesfully loading the database file, the system prompts “Ready” in its MessageBox (see Figure 3). The eye tracking software and MG2.0 communicate via IP. After starting the application (“Start” button), MG2.0 begins to receive image- and fixation data from the eye tracker. The runtime application continuously shows the images of the scene camera and indicates the user’s gaze direction with a blue rectanglar box (its centroid is the fixation point). The cropped image of that area is then piped to the SIFT feature based object recognition framework [Lowe 2004]. SIFT (Scaleinvariant feature transform) is an algorithm in computer vision to detect and describe local features in images and is well suited for content based indexing and retrieval of images. If a known object is recognized, this subsystem returns its the symbolic name that the operator has assigned to identify this object – also refered to as “label”. However, this label does not directly reflect


the object of interest, as many fixations are unconscious and may be considered as noise in our application context. The image recognition results will rather be used to label the frames of the video stream with the label of the recognized object. All frames that belong to the same fixation will get the same label. Some frames however, will not be labeled: This is the case, if no fixation was registered a that point of time or if the confidence of the retrieval system for a known object was too low. As MG2.0 runs in a realtime scenario, some fixations might furthermore not have been analyzed as the average processing time to analyze one image exceeds the average time between two fixations.

Fig 3: Screenshot of Museum Guide 2.0 monitor application The sequence of labels of the frames is then analyzed to distinguish noise from non noise and to recognize the event of consious gaze (the trigger for MG2.0). The event of such a trigger is also monitored in the application as the name of the object is displayed in the MessageBox. For details about these analysis steps please refer to [Toyama2011]. As soon as the application detects gaze on a particular object, it plays the prerecorded voice data with information about that specific object to the user. Gaze detection is mainly driven by two system parameters which we tuned in experiment to best suit all test persons. With these parameter values, the AR starts with a delay of one or two seconds. The operator interface also allows the modification of a set of system parameters. The individual values are preset to defaults that have been determined by a series of experiments that we conducted during the development of MG2.0. Most mentionable here are the values for Attentive duration and Attentive noise. Attentive duration specifies for how long an object has to be watched before the users interest is anticipated and audio data is displayed. Attentive noise specifies the maximum duration for which the user may fixate another object so that this can be considered as noise. If the value of attentive duration (number of frames) is chosen to small, MG2.0 will likely present audio data for an object that the user does not watch consciuosly. If it is to large, the presentation starts with longer delay or not at all. The suitable value has been evaluated in experiments with different users and the value 22 is set as the default value. Similarly, if attentive noise if chosen to small, the system resets the counter too often, that has to reach the attentive duration value. It might thus happen, that the system does not recognize gaze at all or only if the user acitvely concentrates to one object in an unnatural way. If that parameter value is chosen to large, even long fixations to one object are treated as noise and the system reacts more lazy. The experimentally evaluated default value is 18. Whereas the user interface of MG2.0 presents audio data when a gaze event is recognized, the console window of the application presents much more information that is helpful for the developers and operators of the system to analyze and debug any kind of malfunction and to tune the system to achieve best performance. Figure 4 shows a screenshot of the console with annotations to the kind of information displayed.


Fig 4: Screenshot of the console of MG2.0

3 Populating the Database with Exhibits To demonstrate MG2.0 under most realistic conditions, one also needs a real world environment that at least simulates a museum with its exhibits. With our intended application in mind and the requirement to recognize objects based on their visual appearance, we had to find decisions about the type of exhibits that we like to recognize in two dimensions: 2D- versus 3D-objects and small versus large objects. 2D-objects like paintings or photographs would certainly be the easier choice. However, if our approach manages to deal with 3D-objects (e.g. sculptures), we can right away recognize 2D objects as well. 3D is more challenging, as the user looks at these objects from many different perspectives and the captured camera image of the object, as taken by our camera, looks much different from each direction. For our application we decided to face the more challenging of 3D-objects. To overcome the mentioned variety of object appearance, pictures of each object need to be taken from many perspectives. All images of the same object need to be indexed with the same unique label for that exhibit. Slightly similar to this are the implications of the decision for either small or large objects (no matter whether 2D or 3D). When facing a large object, the visitor will typically stand close and his eyes scan the objects different regions step by step. When we like to index such a large object, we need to point the camera to the many different areas and label all images just as we did for the different perspectives. In our simulated museum scenario however, we limited the sample exhibits to small 3D objects (see Figure 1). Ideally, these objects in the museum should be arranged in a way that neighboring objects are not too close to each other so that objects do not overlay in the camera image for most perspectives. For each object in our exhibition we perform the following steps: 1. 2. 3. 4. 5.

Record a video file of the object that should be added to the database. We use the camera of the eye tracker to obtain the most similar images (wrt. e.g. brightness, contrast, resolution) as during runtime. Place the object to its intended place in the museum A person wearing the eye tracker walks around the table, thereby directing the scene camera to the object (Eye tracking is not required here). Record the video data with the iViewXTM software. Save the video with a filename matching the label of the object (e.g. “speaker”)

If the exhibits are exposed to natural daylight (not purely artificial light), it is recommended to also record the video under different lights to extract sufficient variation of SIFT features for recognition. After that step we have one video for each exhibit in our museum. To populate our database with the SIFT features of the art objects, we have to process each video with MG2.0 Indexing Tool. A screenshot of this application is shown in Figure 5.


Fig 5: Screenshot of the MG2.0 Indexing Tool With this tool, the operator can navigate through the video (in different speed, frame by frame as well as forward and backward) to search and select specific still images that appear to be ideal for the indexing. Rather than using the eye tracking information as done at runtime, during the indexing phase the operator has to manually select a point in the still image. The application now extracts SIFT features from the local area whose center is the clicked position. The operator can define the size of the local area with a slider. The extracted features within the image are shown as white arrows in the smaller image box at the right. These features can then be saved to the database. The database itself is represented by a defined path in the filesystem, following certain naming conventions. Just like the label of the exhibit defined the basename of the video, this name is also reflected in the database as it contains subdirectories with that name. All features for this object as well as the still images are stored in that subdirectory. To achieve a good performance, the total number of features should be even among objects. As a rule of thumb, the largest number of features from one object should be double to the smallest one. Objects that have lesser visual structure and thus fewer features should therefore be indexed with more images. The last step to prepare our system for a specific museum is the definition of the audio files that are played to the user if gaze to an object is detected. The recordings can be done by an external tool (e.g. voice recorder). These files simply need to follow the same naming convention for the basename as the video file, but with the extension “wav”. That file then has to be moved to a folder which contains all the audio files. Different such audio folders can be defined for multiple languages.

4 Evaluation – User Study To evaluate the usability of the complete system, we conducted a user study with 23 users. The users were recruited from the staff of the DFKI knowledge management department, and most of them were students in the age of their mid 20-ies. They are familiar with ICT technology and have experienced the established AR systems in real museums already, but they have no specific affinity to museums. As the object recognition and anticipation of user interest based on the eye-gazing information have been the central aspect of our research, no real museum was involved in the studies. Instead, we rather built “our own museum” out of daily life objects. The users were asked to stroll in our museum with two different guide system. One is our Museum Guide 2.0 and the other is an audio player based guide system. Audio player based museum guides are currently used in most of the museums and therefore provide a good basis of comparison with existing technology. Usually exhibits have a tag number in front of them and the users have to select the corresponding audio track from the audio guide to get more information about that exhibit. The same setup was used in our experiment by assigning a tag to each of the twelve objects in our museum and storing the corresponding audio information with the same tag in the audio player. The users were


asked to freely move in the museum and get information about the object they are interested in with the help of the audio player. After the users finished their round with the audio guide, they were introduced to the eye tracker and the eye tracker was configured and calibrated for each user. This whole setup process for each user took 10 seconds in the best case, but also 5 minutes in the worst case (e.g. if a user had a different dominant eye than his predecessor and the eye camera thus had to be mounted to the other side or if the pupil was not recognized immediately). If the calibration was not done properly, the system could not detect the accurate gaze position in the scene, which then causes wrong trigger events for MG2.0. When this setup step was done, the users were asked to take another round in the museum wearing the eye tracker. Whenever the users gazed at an exhibit and gaze on exhibits was detected, Museum Guide 2.0 played a pre-recorded audio file to provide more information about the gazed exhibit. When the users finished their round with Museum Guide 2.0, they were given a questionnaire to assess different aspects of the system. A summary of user responses to the questions comparing the gaze based interface with the traditional audio player interface is shown in Figures 6-10. Since the eye tracker used in the study has several hardware constraints (such as uncomfortable helmet, chin rest, etc.), we referred only to a “gaze based interface (device)” in the questionnaire to judge the real potential of gaze based information provision. The results show that most of the users would prefer to use a gaze based device as compared to an audio player when they go to a museum. Another interesting result was that although many users were satisfied with the traditional audio player, the mean opinion score (MOS) for Museum Guide 2.0 was 4.3 as compared to 3.2 for an audio player.

Fig 6: Responses in the user study for the question: How much do you like a gaze based interface (or a traditional audio player) for getting information?

Fig 7: Responses in the user study for the question: How much was the calibration process acceptable for you? We must admit, that the users of our study have to some degree been biased in a positive way: The study has been conducted by the developer himself and the users have been colleagues or even friends. Also, we argued that the users shall abstract from the need to wear the helmet with the built-in eyetracking device. But we did at least not register negative tendencies which would have been taken as a knockout for the idea of MG2.0. Nonetheless, the questions reflected in Figure 8 and 9 were of higher importance to us, because the more objective answers reflect the central aspects of our research: the anticipation of the users interest to some specific object based on gaze information.


Fig 8: Responses in the user study for the question: Did you get the information against the object you want to know?

Fig 9: Responses in the user study for the question: How often did you get the information against the object which you were NOT interested in?

Fig 10: Responses in the user study for the question: What would you like to use when you go to a museum (Ignoring the hardware constraints)?

5 Conclusion and Outlook This paper presents a novel approach to provide Augmented Reality to users based on gaze information. The technologies involved are eye tracking devices, image based object recognition and own approach towards anticipation of the users’ interest. Our aim was to demonstrate the feasibility of such technology in practical applications (without considering location information – similar to Google Goggles – which might further boost recognition performance). And we selected the domain of museums, specifically inside exhibits of static objects, as these environments provide the neccesary settings to achive best object recognition results. However, we are aware, that a system like MG2.0 would increase the costs for setup and maintenance in comparison to state of the art AR technologies. At the other hand, MG2.0 might specifically serve elderly people or people with motorical disabilities, as they do not require any explicit manual interaction. MG2.0 thereby resembles a personal human guide. Pros and Cons of a system like MG2.0 thus need to be balanced carefully. 109

Today, eye-tracking devices are still rather expensive and MG2.0 is hosted on a laptop PC that needs to be carried along. But technology evolves quickly. With an increased demand for those devices – as it might be raised by applications like MG2.0 – larger production series will lead to reduces prices and the hosting computers will soon have the size of a smartphone. In parallel, the eye trackers become smaller as demonstrated by SMIs new Mii device. Rather than being mounted to a helmet, this device integrates its components (including scene camera and eye-tracking cameras for both eyes) in the frame of almost ordinary glasses. As announced by SMI, that new device also reduces the mandatory calibration step from today five points to just one point at which the user has to look during this step.

Acknowledgements This work was partially funded by the BMBF (German Federal Ministry of Education and Research), project Perspecting (01 IW 08002). We also wish to thank SensoMotoric Instruments (SMI) GmbH for providing us an iView XTM HED mobile eye tracker for building the application and conducting all experiments.

References Agamemnon, EU co-funded undert the IST 6th Framework Program of the European Commission, information available online: Lowe, D. G., “Distinctive image features from scale-invariant keypoints”, International Journal of Computer Vision 60, 2, 91–110, 2004. Santos, Pedro and Stork, André and Linaza, Maria and Machui, Oliver and McIntyre, Don and Jorge, Elisabeth. “CINeSPACE: Interactive Access to Cultural Heritage While On-The-Move”, Online Communities and Social Computing, LNCS Volume 4564, pages 435-444, 2007 SMI - SensoMotoric Instruments, “iView X™ HED – The mobile, head-mounted eye- and gaze tracker”, Takumi Toyama, “Gaze Guided Object Recognition Using a Head-Mounted Eye Tracker”, Master Thesis, Graduate School of Engineering, Osaka Prefecture University, March 2011


Exploring Exhibitions in Virtual Worlds: Case studies in using open technologies Alan Miller, Sarah Kennedy, Lisa Dow and Colin Allison University of St Andrews [email protected] Abstract This paper discusses 3D Virtual Worlds it reports on our experience in developing; in world exhibitions, historical reconstructions and virtual laboratories. A virtual archaeological excavation based upon the Acropolis Basilica Sparta, a reconstruction of St Andrews Cathedral, a Wireless Island and a Network Island are all described. These spaces were developed in the first instance for use within formal educational courses. However, a central design aim was to support exploratory learning. This has made them useful in less formal settings. We believe the specific sites, technologies and lessons learnt are relevant to both virtual exhibition spaces and real museums. Most virtual world museums have been built using the proprietary Second Life service, here experiences using both the open source virtual world server OpenSim and Second Life are reported on. This paper concludes by identifying challenges that need to be met before the potential of OpenSim is realised.

1 Introduction For decades the spread of 3D virtual technology has been limited by the capabilities of commodity technology and the cost of specialised technology. In the last year the popularity and distribution of 3D movies has been taking off. This has been accompanied by the reasonably priced availability of 3D cards and monitors. In the next few years 3D technology will become ubiquitous. Meanwhile 3D televisions, players and films can be bought on the high street. The first 3D mobile phones are becoming available. It can be expected that in the future 3D will be part and parcel of everyday life. Social media will cease to be limited to asynchronous, 2D technologies and will be augmented by integrated synchronous communication and 3D environments. How these emergent technologies may be integrated into established domains such as museums, exhibitions spaces and education is a question of relevance to all practitioners in these areas. Virtual Worlds and Metaverses provide a 3D environment that users inhabit through the proxy of avatars (Kumar 2008). Through avatars users interact both with each other and with the virtual environment. Virtual Worlds differ from games primarily in that their users can create their own content and shape the world. Users do not have a pre–ordained set of goals that they are expected to achieve, as they would in a computer game. Rather they are free to think up their own objectives and goals. They may collaborate with others in defining an in world space. This might be a town frequented by virtual steam punks, a racing circuit or a virtual museum. Metaverse users are able to define geography, build architectural structures, fill their space with crafted objects, and upholster their buildings using diverse images. They may also establish communication between the virtual world and other parts of cyberspace or through sensors and actuators with the real world. Avatars may be fashioned to a myriad of different shapes and sizes. They may be clothed appropriately for cultures past, present and imagined. Robots that will respond to presence, movement and conversation may populate the landscapes. The whole environment, landscapes, buildings, objects and “bots” may be programmed to interact with Avatars in a myriad of different ways. The combination of the flexibility of metaverse environments combined with their availability and cost effectiveness that makes them an appropriate platform for developing virtual spaces that may in turn be filled with exhibitions, interactive exhibits, visitors’ centres and museums. These spaces may be standalone virtual spaces or may be integrated to enhance physical exhibitions. An excellent survey of Museums within Second Life can be found in (Urban 2007). The body of the work discussed in this paper comes out of collaborative projects primarily between academics. Our motivation was to develop resources that would overcome barriers of time, place and cost to allow students to engage in exploratory learning. We believe that the resources described here have relevance within less formal education settings as well as to those who are simply interested in the


subject area. Consequently, the work here has a relevance to those interested in utilising these technologies in the context of exhibitions, exhibits and museums. This paper reports on our experience in using 3D virtual environments within contexts relevant to museums and exhibitions. It is structured into a number of sections. First, the state of the art for 3D virtual worlds is discussed, in this we focus on Second Life the most popular virtual world and OpenSim. OpenSim offers a more flexible service model, which empowers developers and users alike. It overcomes many of the limitations inherent in Second Life. Three case studies are discussed. The first is a virtual excavation. This allows visitors to engage in a hybrid web and virtual world game focussed around the virtual excavation of a Spartan Basilica. The second is a virtual reconstruction of St Andrews Cathedral, as it would have been at the time of consecration by Robert the Bruce in 1318. The third is a site containing technical, historical and interactive exhibits connected with Internet routing and wireless networking technologies. Having discussed the three case studies this paper moves on and draws some inferences about the opportunities virtual world technologies offer and some of the challenges that need to be overcome for these to be met.

2 Virtual Worlds, Technology and Exhibitions Virtual Worlds are a new type of Internet application. They offer the potential of extending the engagement and realism of 3D games into the domains of computer based and computer enhanced exhibitions. They have the advantage of being cost effective and flexible. In a virtual world a user is represented through the proxy of an avatar. The avatar can navigate the world through walking, flying and teleportation. Users may communicate through an avatar's gestures and movement as well as textual and voice chat. A physics engine imposes an approximation of the laws of physics. These virtual worlds can also act as a multi-media portal bringing together and organising, interactive exhibits, video shows, audio podcasts, graphical displays and documents in various formats. They provide a framework from which a myriad of different applications can be built. These include interactive simulations of Internet routing, role playing environments for steam punks, racing games and exhibitions of special collections of ancient books. They have also been used to develop exhibitions, art galleries and virtual laboratories. There are numerous virtual world technologies however, the genre is currently dominated by Second Life a commercial virtual world which is offered as a service by Linden Labs. Second Life has been the most successful virtual world. It has had millions of users and provides an environment where tens of thousands of users are online simultaneously. With Second Life a single virtual world is provided as a proprietary service. Users may connect to it for free. However, if they want to develop a presence in the world, they need to rent a space from Linden Labs. Second Life also has an internal economy mediated through the currency of Linden Dollars. These are in turn exchangeable for real dollars. There are a number of limitations on the functionality supported by Second Life, which are not inherent in Virtual Worlds themselves but rather flow from the business model adopted by Linden Labs. This has in turn shaped the service that they provide. • • • •

To protect the value of virtual commodities within Second Life, the ability to import and export objects is restricted. There are limitations on the functionality of user programs. For example the rate at which objects can be created and the frequency of communication are both restricted. The ability to customise the environment to meet specific needs is limited by the lack of administrative control available to users. The cost of virtual real estate restricts the usefulness of the environment.

Recent times have seen a harsher economic climate. This combined with the limitations mentioned above have contributed to a decline and halting of the growth of Second Life's user base. At the same time virtual worlds based upon OpenSim have been gaining in popularity. This has been given a significant spur in the education sector by Linden Labs decision to remove its educational subsidy, effectively doubling the price of maintaining an educational presence in SL. 112

OpenSim is an open source toolkit for building virtual worlds available under GPL. There are a number of similarities between OpenSim and SL. They share the same clients, the SL client can be used to access OpenSim and clients designed for OpenSim such as Hippo and Meercat can also be used to access SL. Consequently, they share a common user interface. If you know how to control the movement and communication of your avatar in SL you will be able to manage in OpenSim as well. Geography, architecture, artefacts and clothing are created in the same way. Interactivity can also be added to both environments through the use of Linden Scripting Language and its interpretation by an inbuilt script engine. The above similarities are a consequence of the origins of OpenSim, which can be traced to the release of an open source Second Life client. The SL server code base remains proprietary. However, it was possible to reverse engineer the compatible open source virtual world server OpenSim. Although it has been possible to purchase SL servers to run locally the cost of such a set-up has been prohibitively expensive for many. Consequently, if an institution wants to run its own Virtual World service OpenSim offers an attractive option. Being able to run a local virtual world server opens up a number of opportunities. First it means the restrictions on the programmability of the system are lifted. Second, it means that local work stations can be situated within a physical exhibition or indeed integrated with physical exhibits and thirdly, it reduces the marginal cost of virtual world space.

3 Recovering a Spartan Basilica The Laconia Virtual Archaeology project (LAVA) is a collaborative project (SGM09) for the development of digital exhibitions, simulations and reconstructions based upon the Acropolis Basilica Sparta (Sweetman2000-2001). The project is based upon a real site which has undergone recent excavation activity. A virtual excavation based on a Byzantine basilica excavated by the British School of Athens during 2000-1 has been created. It aims to organise and make accessible the results of that excavation along with contextual material, by digitising artefacts and maps as well as developing 3D resources. In this way the results of the excavation are made available to students, scholars and other interested parties, without them having to physically visit the site in Sparta.

Fig 1. The Excavated Spartan Basilica The project is made up of the following resources: 1. 2.

There is a web page, containing movies, maps and textual information about the site and its history. This provides an introduction to LAVA and the resources associated with the project which is accessible to anyone with a web browser. It also contains links to related resources. A 3D model of the excavation site and its surrounding area as it is today has been developed. This shows the remains of the basilica that are visible, for example parts of the Basilica walls and doorways. The reconstruction covers almost a square kilometre. It allows users to situate





the basilica within its geographic context, to gain a feel for the surrounding fauna and flora and to explore other sites that are physically close, for example the remains of a theatre. A reconstruction of the Basilica as it would have been in the 10th Century has been developed. This includes the church and the baptistery. There is a full architectural reconstruction of walls, windows, doors, stairways and roofs. In addition the reconstruction is authentically decorated internally and externally. Through the proxy of Avatars visitors are able to explore the grounds, to enter the church and to climb the tower. They are able to view the Basilica from a range of perspectives and to gain a spatial awareness of the key aspects of the site. A visitor’s centre provides social and historical context for the Basilica, in much the same way that a real world visitor’s centre might provide context for a physical site. Visitors are able to read about the history of the Basilica, to see scale 3D reconstructions from different eras and to inspect life sized models of historical artefacts. There is an interactive web application that allows users to manage an excavation, discover artefacts and document their progress.

The reconstruction of the Basilica was based upon the results of recent excavation activity. The process involved four stages: materials including extensive, maps, photographs and textual documentation were collected and organised. From these a detailed mock-up of the model was created using 2D technical drawings. This was followed by the construction of a prototype 3D computer model using 3D max.

Fig 2. Reconstruction of Acropolis Basilica Sparta A 3D model of the Basilica was then created in Second Life and later ported to OpenSim. The main motivation for moving the resource from Second Life to OpenSim was cost. To rent the area required for the in world resources, (basilica, visitors centre and excavation site) would require an annual rent of some £20,000, however it can be run from a single server with marginal costs of between £500 and £1000. The LAVA project was intended primarily as a co-operative exploratory learning environment that supports engagement with archaeological excavation scenarios. By leveraging the immersive nature of Game Technologies and 3D Multi-User Virtual Environments (MUVEs), LAVA facilitates the adoption of exploratory learning practices in environments which have previously been inaccessible due to barriers of space, time or cost. Students benefit from being able to collaboratively manage and participate in a virtual excavation. of the basilica. Real world findings have been incorporated to provide an authentic virtual excavation experience. The infrastructure supports a group-based exploratory approach which integrates 3D technologies into an existing web learning management system to enable location independent, self-paced access. A research issue addressed by the excavation was to provide evidence for a working-hypothesis; that contrary to traditional belief dome-roofs were constructed during the 7th century AD. The creation of a staged 3-D model of the Basilica is of significant research benefit to the study of Byzantine architecture. Since the roof of the Basilica no longer survives, the nature of the roof has been postulated on the basis of the load bearing walls in the eastern section of the church and on the remains of fallen chunks of masonry, which lie outside the church proper. Previously, Sweetman had successfully argued that it is possible for this dome to have existed as early as the 7th century and has


found further parallel examples to support the hypothesis (Sweetman 2006). Reconstructing the church, using the architectural survey completed in 2001 allows the theory to be visualized and for further permutations to be tested. At the same time, the creation of a 3-D model allows the students to further interact with the learning game and thus answers one of the key suggestions raised following evaluation. The resource is also appropriate for less formal educational settings. The ability to access LAVA through the Internet makes it globally accessible. The spread of digital literacy, widespread access to the Internet and growing familiarity with 3D navigation, gained through playing computer games, means that many users will be able to explore LAVA resources without having to undergo a steep learning curve. The LAVA project was conceived primarily to address educational needs within a structured degree program. This approach also has value within a research context and in less formal educational settings such as outreach work.

4 Reconstructing Scotland’s Cathedral The core of the Phoenix project is a 3D virtual model of St Andrews Cathedral. Both innovative and traditional technologies were used to develop a set of resources that enable the virtual exploration of St Andrews Cathedral and thereby enrich our understanding of its appearance both externally and within. The 3-D reconstruction improves accessible as it can be reached through a standard Internet connection and may therefore be visited without users having to make a physical journey to the Cathedral site. The virtual reconstruction of this august building is of benefit to those interested in architecture and history in general as well as those with a specific interest in religious buildings. It is not only open to University students and academics but also to the residents of and visitors to St Andrews. This project makes use of advanced computer technologies to bring to life an important national asset in a way that enhances access to a shared cultural heritage.

Fig 3. St Andrews Cathedral St Andrews Cathedral is a striking landmark of the town and a fascinating building for scholars and visitors alike. At present there are few visual guides available (one of the best is that by Prof Richard Fawcett) that allow the visitor to reconstruct in their imagination the former glorious, fully vaulted and roofed extent of this major medieval monument. Work on the Cathedral began around 1160 and was completed 150 years later (the west façade and parts of the nave collapsed in a storm around 1270) it was finally consecrated in 1318 in the presence of Robert the Bruce. When St Andrews Cathedral was in its prime, it was Scotland’s largest and most magnificent church. Its presence was the catalyst for the foundation of a University at St Andrews in the 15th century. This was Scotland’s first university.


The project aims can be summarised as: 1. 2. 3. 4.


The development of a historically accurate 3D model of St Andrews Cathedral. The provision of a service so that users can explore the model remotely. The development of a virtual tour, with commentary provided at important locations throughout the virtual site. The design of cross reality exhibits, so that visitors experience of the physical site is enhanced through access to the virtual reconstruction. The provision of an online visitor’s center, which provides historical and social context for the Cathedral.

Fig 4. Reconstruction of Cathedral Cloister In order to overcome the technical challenges and to ensure that the work was historically accurate the Phoenix project involves collaboration between two of the oldest disciplines, Classics and History, on the one hand and one of the newest, Computer Science, on the other. Plans for the reconstruction were developed using pre-existing resources relating to the cathedral. For example, architectural fragments located in the museum and those reused throughout the town, along with the existing work of medieval architectural historians. It drew upon architectural drawings and artistic impressions of the cathedral’s original appearance. These were used in constructing a set of architectural drawings and prototypes of the Cathedral and surrounding buildings including the earlier church of St Regulus. A key innovative aspect of this project was to situate the reconstruction within the immersive 3D virtual world OpenSim. This allows visitors to explore the reconstruction through the proxy of an avatar. It provides interfaces which allow user to create, shape and combine objects to construct complicated structures. Images may be imported and applied to the surfaces of objects to give an authentic look to the structures created. Coordinates for the placement, orientation and location of objects may be specified thereby allowing consistency of scale between the real and virtual worlds to be established. This project was the subject of six successful computer science and information technology MSc. dissertations in the summer of 2010. Technical support was available in regular laboratory sessions; in addition students had input from experts in the history and architecture of the cathedral and direct access to the physical site. Each project had a specific set of goals: one was developing a web site which would link to a welcome area in the 3D world, one was developing a visitor’s centre, three were developing specific areas of the site and the six was to develop historically accurate clothes so that visiting avatars would look the part. Upon completion of the MSc’s, work has continued integrating and developing the models and associated software.


Use of the specific virtual world OpenSim had a number of benefits for a medium sized development project:

Fig 5. Reconstruction of Cathedral Choir Stalls 1. 2. 3. 4. 5. 6.

The 3D virtual space provides realism and engagement. OpenSim provides tools which support the creation of 3D content. A server and a client may be run on a laptop, providing cost effective portable sandbox development environment. Entire regions in OpenSim may be saved to “oar” files which may then be loaded onto a different server. This facility allowed work developed independently to be easily loaded onto a shared server prior to being integrated with the other projects. A quad core server with four gigabytes of ram and 100 Gbytes of memory provides a cost effective solution. It can provide 30 to 40 simultaneous users with the 1 Km2 of virtual space required for the cathedral, induction area and visitor’s centre. Support for pictures, photographs, film, music and animations, provides for a rich multimedia environment for the visitor’s centre. Accessibility through the Internet facilitate widespread access and allow integration with social networking.

The Phoenix project is an example of a medium sized project which involves significant multidisciplinary collaboration with multiple actors involved in the development process. The use of OpenSim as a platform provides appropriate levels of support for collaborative working. Its interfaces also provide sufficient scope for the development of cross reality systems which allow the virtual world to enhance visitors’ experience of the physical site.

5 Interactive Exhibits: Virtual Networking Spaces Computer networks in general and wireless communication in particular are becoming increasingly central to the way that we live our lives. Since the 60s and 70s they have been important for the military and the academia. Since the 90s they have been increasingly part of the business scene. The growth of social networking in recent years has seen their expansion into the social lives of millions. The rebellions in 2011 across the Middle East and the Maghreb have been heralded by many as Twitter and Facebook revolutions. Without wanting to overstate the case it does appear that wireless technologies now also play an important part in the political sphere. This project focuses on the creation of a virtual world space which allows visitors to explore and experiment with a variety of exhibits and exhibitions connected with the technical world of wireless networking (Sturgeon2006).


Fig 6. A WiFi Virtual Laboratory At the core of the experience is the ability to interact with simulations of WiFi 802.11 protocols within a WiFi Virtual Laboratory. The WiFi Virtual Laboratory (Sturgeon 2006/2009) uses ns2 (ns2 1995) for generating realistic simulations. The simulator provides realistic results and has been central to the development evolution and testing of core Internet protocols. It generates a discrete event simulation, based upon the topology of a virtual wireless network set up by students on the island. A trace is generated for each simulation which gives the source, start time, type destination, end time and amount of data for each packet transmitted. Wireless Island is a 3D virtual space which can be hosted by OpenSim servers. A visitor interacts with the simulator through the proxy of an avatar. The avatar may define the number of nodes and the topology of the network. Nodes may be distributed across the Island using an intuitive interface. Data flows across the network may then be defined either through pointing and clicking on network elements or through a simple form interface. The network structure and data flows are then communicated to an instance of the network simulator and the results of the simulation returned to the virtual world. The events in the simulation are then played back to the avatar. There are standard controls which allow the speed of playback to be controlled and for the user to skip forwards or backwards in the simulation. The transmission of packets being represented by expanding colour coded translucent spheres. Thus users can observe; the source of a packet, the range of its transmission, the type of packet, how long the transmission takes, whether a collision occurs with another packet and how the packet relates temporally to data flows within the simulation. The WiFi Island is more than a single interactive exhibit. It leverages Opensim’s programmability, communication interfaces and support for multiple media to provide comprehensive support for somebody interested in the history, technology and operations of wireless networks. There is a central area to the island where avatars arrive. This contains a space for socialising, an introduction to the geography of the island, signage and teleport stations, which allow avatars to quickly transport to an area they are interested in. 1.

2. 3. 4.

There is a museum building which contains a timeline of the history of wireless computing, which is illustrated by reference to important personalities and significant events. Signage guides an avatar through the museum. As the avatar progresses they move chronologically through the history of wireless networking. The museum also contains a small video lounge. Open courseware, standards documents and research papers related to wireless networking are maintained on a web server. These may be viewed in world through portals or downloaded for printing. A movie theatre allows visitors to select from videos addressing different aspects of wireless networks. These include lectures made available through the open coursework initiative. There are a series of interactive exhibits that are pre canned simulation scenarios. These 118


illustrate “classic” wireless network issues such as illustrating how the early Aloha protocol worked and demonstrating the important “hidden node problem”. There is a lecture theatre, which may be connected to a live lecture or allow visitors to view a range of slides and listen to their associated audio.

The first implementation of the WiFi Virtual Island was in Second Life, however re-implementing it in OpenSim lead to several improvements. The removal of land-space/cost limitations of SL allowed WiFiSL to be expanded into the full blown Wireless Island. Wireless Island has been used for two consecutive years by final year undergraduate students. The set up encouraged students to take control of their own learning, they could navigate between resources as they pleased, rather than being expected to follow a linear path. The students found it highly engaging and educationally valuable. A similar resource Network Island is an interactive simulation of routing algorithms. As with the WiFi Island the virtual world acts as a 3D portal providing access to videos, sound clips, animations, presentations, lecture notes, research papers and standards. At the core of this resource is a 3D interactive routing simulation. This allows visitors to construct network topologies, and observe how network routing algorithms work in determining shortest paths and routing tables. The flow of packets across the network can be observed and the effect of topological changes experimented with. In both of the above cases the sense of presence, support for multiple media and programmability of virtual worlds enrich the specific subject matter. This use of a virtual world as a portal into a collection of interactive exhibits is in many ways generic. Many subjects would benefit from such portals as standalone resources or as enhancements to real world facilities. This is particularly the case with the growth of open course ware there are multi-media learning resources available for deployment within portals. Whilst virtual worlds are valuable for building virtual spaces that allow a subject to be explored there are technical challenges that need to be met if their use is to move from the experimental edges of society into the mainstream of technical and cultural life.

6 Reflections and Conclusions Developing, deploying and running a local virtual world service using OpenSim allows 3D applications to be developed in a cost effective manner. There are challenges in the areas of content creation, application development, service provision and system development. However, if these challenges are met virtual worlds offer significant potential for use within the context of both virtual and physical exhibitions and museums. Virtual worlds provide a framework which supports the development of a wide portfolio of 3D applications. Virtual exhibitions will be able to leverage on the presence users project into the virtual world the richness of available communication modes and support for multimedia content. 1.



Content Creation: Content may be created in world using simply 3D modelling tools. Shapes may be created moulded and combined. Textures may be applied to surfaces. Images and other media may be imported. Geographic features may be simply drawn in an external application and imported. Creating content within OpenSim takes a similar effort to a reasonably sophisticated web site. Support for Content Creation: Having administrative control over a virtual world allows the import and export of objects and groups of objects to be easily managed. Object Archive Repositories (OAR) and Inventory Archive Repositories (IAR) allow the entire contents of an Island or all the objects that an avatar possesses to be saved or loaded. This enables the creation of libraries of geographic features, buildings, textures, artefacts and clothes to be developed, shared and reused. The will in turn allow domain experts to quickly utilise generic content before developing content specific to their subject area. Application Development: Administrative control of a local virtual world opens the opportunity to enable powerful programming paradigms. OpenSim contains support for three programming modes. Mini Region Modules, Region Modules and Plug Ins. These offer


4. 5.


increasing control over the environment and integration with the code base. There is a trade off between control available to the programmer and the complexity of development. The possibility of developing complex animations has been demonstrated by the simulation of Internet routing. The ability to interface with external objects, whether they be in cyberspace or the physical world was demonstrated by the Wireless Island. Service Provision: Running a local service means that sufficient in-world space can be made available to develop impressive virtual exhibitions. Furthermore rich models of access control to the virtual world are supported. These range from open access to tightly regulated systems. System development: The existence of an open source server for virtual worlds allows experimentation with system aspects of virtual worlds to be addressed. For example dual reality systems to be developed where the virtual and real world are overlaid. Cross reality systems may be developed where information from the real world is used to inform the virtual world and interfaces in the virtual world are used to control virtual objects in the real world. Persistence of presence: Virtual worlds facilitate synchronous communication through the proxy of avatars. In Second Life there is one world and your avatar remains forever within it. With OpenSim there are multiple worlds. How to allow an avatar to move seamlessly through these worlds whilst maintaining its personality, appearance and belongings is an important challenge.

In the future it is possible that distributed virtual worlds, which are as accessibility as the web, as ubiquitous as social networking and as engaging as 3D games, will be part of our electronic landscape. Today they are an important technology, which has significant potential for delivering cost effective virtual exhibition spaces and for enriching real world exhibitions. This paper has discussed the use of virtual worlds in four settings. As a part of a virtual archaeological simulation, as a reconstruction of a significant historical building, as space for exploring wireless networking and as space for learning about Internet routing. In each case the virtual resource allowed barriers of time, place or cost to be overcome to allow visitors to explore the subject area of choice. As 3D technologies become ubiquitous, the general IT literacy of the population will improve making virtual worlds more accessible and therefore more valuable.

References Kumar, S.; Chhugani, J.; Kim, C.; Kim, D.; Nguyen, A.; Dubey, P.; Bienia, C. & Kim, Y. Second Life and the New Generation of Virtual Worlds Computer, IEEE Computer Society Press, 2008, 41, 46-53 Sweetman, R.The Sparta Basilica Project British School at Athens, 2000-2001 Getchell, K, Miller, A, Allison, C. and Sweetman., R. (2009) “ Exploring the Second Life of a Byzantine Basillica”, Serious games on the move. Springer Wien 2009 Sturgeon, T.; Allison, C. & Miller, A. 802.11 Wireless Experiments in a Virtual World ACM SIGCSE Bull., ACM, 2009, 41, 85-89 Urban, R. et al., A Second Life for Your Museum: 3D Multi-User Virtual Environments and Museums, in J. Trand and D. Bearman (eds.) Museums and the Web 2007: Proceedings, Toronto: Archive & Museumes Informaties, published March 1, 2007 Conculted April 23, 2011.


Living with a Virtual Agent: Seven Years with an Embodied Conversational Agent at the Heinz Nixdorf Museums Forum Thies Pfeiffer, Christian Liguda, Ipke Wachsmuth A.I. Group, Faculty of Technology, Bielefeld University, Universitätsstraße 25, 33615 Bielefeld, Germany {tpfeiffe,cliguda,ipke}

Stefan Stein HNF Heinz Nixdorf MuseumsForum GmbH, Fürstenallee 7, 33102 Paderborn, Germany [email protected] Abstract Since 2004 the virtual agent Max is living at the Heinz Nixdorf MuseumsForum – a computer science museum. He is welcoming and entertaining visitors ten hours a day, six days a week, for seven years. This article brings together the experiences made by the staff of the museum, the scientists who created and maintained the installation, the visitors and the agent himself. It provides insights about the installation’s hard- and software and presents highlights of the agent’s ontogenesis in terms of the features he has gained. A special focus is on the means Max uses to engage with visitors and the features which make him attractive.

1 Introduction Today, virtual agents are much more common than ten years ago. There are, however, little of them continuously in action – not to say alive – for longer periods. One of the exceptions is the virtual agent Max who has a full-time job as a central exhibit at the Heinz Nixdorf MuseumsForum since 2004.

1.1 The Virtual Agent Max The virtual agent Max marked the cutting edge of research in artificial intelligence and artificial life at the turn of the millennium. His ontogenesis has, however, never stopped and he is continuously updated. This kind of research can rarely be experienced in everyday life. As a layman it is hardly possible to appreciate or even use such technology. With Max the visitors can have lifelike experiences with an anthropomorphic “virtual life form” and interact with it using simple natural language input via a text console (see Fig. 1). During the dialogue with the agent many questions arise which go far beyond the exhibit itself: How manlike is the artificial human? Can his intelligence and his knowledge be compared to that of humans? How well are his mimics and gestures evolved? Interacting with the agent is like running a small Turing test, explicitly or implicitly run by every visitor – even by the naïve. Doing so, visitors soon realize that in contrast to humans, artificial systems require a much more explicit use of language. Beyond this, the agent is the most colorful information system of the museum: similar to textual descriptions and multimedia terminals, the agent offers explanations and comments about the exhibits in his surroundings, about the museum, the city, Germany or about himself. Max is self-explanatory in the true sense of the word; no manual is needed. He can elaborate on himself and the technologies which have been used to create him. When interacting with the agent, the visitor experiences individual aspects of computer science, such as logic, natural language processing or computer vision, in concert. This is a unique feature of this exhibit. The agent has been developed by the A.I. (Artificial Intelligence) group at Bielefeld University starting in 1999 (Kopp and Jung, 2000). In 2004, the museum renewed the state-of-the-art area of its permanent exhibition. In this progress, the department “A.I. and robotics” was created and integrated into the exhibition. At that time, the agent already lived at the department of the A.I. group for five years (Jung and Kopp, 2003). Soon, the contact was made, the two groups met and the agent moved – or better: copied – to his new place (see Fig. 1). Since then, the agent’s ontogenesis has been driven by a triangle


of three forces: the scientific progress, the vision of the curator of the museum, Stefan Stein, and the needs of the visitors. A contract between the museum and Bielefeld University’s A.I. group ensures regular updates and the maintenance of the system, including security updates and new operating systems. In addition to that, the A.I. group also takes care of updating the knowledge base and provides new features.

Fig 1. The agent is a continuous attraction for the public and the local and national media.

1.2 The Heinz Nixdorf MuseumsForum The Heinz Nixdorf MuseumsForum (HNF) in Paderborn/Westphalia, Germany, has been initiated in 1996 by the Westfalen Stiftung, a foundation established by the German computing pioneer Heinz Nixdorf. It is dedicated to the history of communication technology (Heinz Nixdorf MuseumsForum, 1999) and has been recognized by the Guinness Book of Records as the biggest computer museum in the world. Not just a museum for computer science, the HNF shows communication technology from the early beginnings to modern digital technology and its possible futures. The agent Max is located on the second floor towards the end of the permanent exhibition and the course “A.I. and robotics”. This location also marks the end of the full course through the museum: At the beginning, the visitors can play with a variety of intelligent systems, such as the best artificial chess player, systems for recognizing coins by their noise, logic games, or artificially generated cantata of Bach. The highlight at the end is then the small talk with a “real” virtual agent. The agent Max stands for one of the museum’s main principles: to show historical and concurrent technologies that can also be tested. The visitor can really interact with technology. In addition to the permanent exhibition, this principle is also motivation for several special exhibitions, for example the highly acclaimed Computer.Gehirn (Computer.Brain) exhibition in 2001 (Heinz Nixdorf MuseumsForum, 2001).

1.3 The A.I. Group at Bielefeld University The A.I. group’s research on virtual embodied agents began in 1995, when an agent was used to establish a frame of reference for spatial tasks in a virtual environment (Jörding and Wachsmuth, 1996). While at that time users already could interact with the virtual environment via natural language, the agent itself was used like a marionette. This changed in 1999 when work started on Max who later moved to the museum in 2004 (Kopp et al., 2005). 122

The A.I. group envisions virtual embodied agents as an important metaphor in human-machine interaction. When users are interacting with computers, one finds some anthropomorphism, but overall, the interaction often is in the fashion of a command-and-control interface. If done properly, a virtual agent of human-like appearance paired with adequate behaviors can elicit more natural human interactions. At the same time, creating such an agent requires very explicit models of human behavior, interaction skills and intelligence (Cassell et al., 2000). That is why understanding through construction is the driving scientific method of the group. Scientific research in A.I. does not often leave the protecting walls of the laboratory. Also, the dominant user group testing research prototypes consists of students and scientists of computer science, which might not represent typical users. Consequently, putting a complex A.I. system such as an interactive virtual agent into a museum is a great opportunity: the system’s environment is dynamically changing, there is no caring scientist around to provide immediate support, and there are many curious visitors testing the system to the limit. There is a lot to learn and observe for A.I. researchers in such a context, and various studies have been conducted with the agent at the museum (Kopp et al., 2005; Kramer et al., 2007; Von der Putten et al., 2008). So far, it has been argued that the opportunity to show the agent as an exhibit in a museum is very interesting to scientists. Yet there is an additional challenge: The visitors have to be interested in the agent, too. Otherwise there would not be much interaction to observe. The agent thus has to be both intelligent and interesting. One advantage is that the museum’s theme is computer science and the agent – as a piece of advanced computer science – is interesting per se in such a context. To further provoke the curiosity of the visitors and please their needs, the agent has been designed to have several features: • he has interesting content to tell • he is a character (see description in 2.5) • he proactively engages with visitors, and • he shows emotions In addition, the agent is continuously evolving, so that recurring visitors will have something new to discover on a regular basis. The following section provides a description of the strategies which are followed to attract the attention of the visitors. Subsequently, Section 3 goes into details about the set-up of the installation, the agent’s interaction capabilities and the necessary hardware. Section 4 then is dedicated to the visitor experiences and provides feedback from the museum staff. Finally, Section 5 concludes and discusses future work, which may involve the extension of the interaction capabilities of the agent to social media.

2 Strategies to Attract Attention and Facilitate Active Interaction 2.1 Be Engaging The first thing to do is to get the attention of the visitors. The agent thereby is not just waiting for a visitor to start typing on the console. He is equipped with a video camera targeted at the hall just in front of him and monitors the area for passers-by. Using computer vision techniques, he continuously tries to detect human faces. As soon as he has recognized a human face, he turns his head and starts to address the visitor with a greeting and an invitation to talk to him. Sometimes, he also just states that he is bored and wants to play. This strategy is maintained throughout the dialog, i.e., the agent does not only react to sentences entered by the visitor, but also introduces new issues.

2.2 Be Informative Once the agent has gotten the attention of the visitor, he starts with a formal greeting and an exchange of names. Right after that, he offers to give some presentations about the museum or the exhibition and demonstrates his willingness to provide answers to the visitor’s questions. To this ends, the agent has been equipped with a knowledge base consisting of facts about the museum, the exhibition, the local area, a coarse knowledge about national politics and extensive knowledge about him and the technologies used to create him.


2.3 Be Witty Joseph Weizenbaum’s Eliza has led the way on how to create the illusion of an understanding interlocutor and to keep the dialog flowing (Weizenbaum, 1996). The agent adopts this approach by using a pattern matching system to detect specific keywords, but also to identify questions, requests or statements. Once a match is found, an associated rule specifies the reaction of the agent in terms of sentences to answer, sounds to play or animations to show. These reactions are designed in such a way that they are partly humorous, partly thoughtful and whenever possible, they are open, so that the flow of dialog can continue. These verbal reactions can also be accompanied by special effects. For example, when asked whether he is cool, the agent draws black sunglasses out of his pocket and keeps them up for some time. At one time or another he will later put the glasses down again, independently of the current topic of the conversation.

2.4 Be Surprising – Logic and Emotions When interacting with an artificial intelligence, one would expect to be confronted with logics and very formal behavior. One probably would not anticipate perceiving a virtual agent displaying emotions. In a certain sense, the agent can be happy or be annoyed. Every perception registered by his sensors, the video camera or the console, is given a certain emotional value, which is sent as an impulse to an artificial emotion system (Becker et al., 2004). The same holds for internal events, such as registered failures in parsing or interpreting the input given by the visitors. If, for example, the agent is insulted, he gets angry and his voice gets tense. Also, his visual appearance changes and his mimics reflect his anger. Over time or when the visitors apologize, he calms down and his facial features get soft again. If he does not get any impulses over a long time, he even gets bored and it could be that he falls asleep.

2.5 Be a Character The agent is not a shapeless piece of software. He is designed to be a plausible, interesting character. This can be best described by the following characterization, given by one of the museum’s staff members: The agent has been brought to live in 1999. He is non-smoker, single, knows little about his developers but nothing about family. He knows about national politicians, but nothing about local ones. He is into 30 music except for pop, loves Kraftwerk , and knows at least one poem from Goethe. The agent knows how to sing and dance – at least occasionally. His favorite movie is “Star Wars”. He prefers watching sports over being sporty. He knows that his eyes are blue, likes guessing games, and does not go to the 31 movies but likes watching DVD. He is instantly offended when hearing the word “Pfannekuchen” (pancake) and knows more or less that he does not need to eat or drink. He absolutely loves his job at the museum, where he also celebrated his 10th birthday (see Fig. 2).

Fig 2. The agent’s tenth anniversary has been celebrated with attention from several media. The people in the picture are Max, Stefan Stein, Stefan Kopp and Ipke Wachsmuth (from left to right). 30 31

Kraftwerk is a famous German electric music band which is active since 1970. “Pfannekuchen“ is used as a keyword to explicitly trigger emotional reactions for show during tours.


2.6 Be Current Every year, the agent is provided with a major update. While there have been purely technical updates, such as a migration from single-core to multi-core technology, most of them offer new functionalities that can be interactively explored by the visitors. This way, recurring visitors have always something new to find and thus have an additional reason to interact with the agent over and over again. As the world is continuously changing, the knowledge of the agent is aging. This is particularly true for his knowledge about the weather, the exhibition or national politics. It is thus necessary to provide near-term knowledge updates. In case of the weather, the agent is enabled to query a WWW-weather forecast and verbalize the answer in natural language to the visitor. All other knowledge, including facts about the exhibition is updated manually (so far). In 2007, the agent learned about mathematics and demonstrated his knowledge in the year of mathematics 2008. Since then, he not only knows about famous mathematicians or number theory, but also how to do calculations and to check prime numbers. He also learned some logic tricks and enjoyed playing games with the visitors. This made him so special, that a new instance of the agent had been created to be part of the temporal exhibition on mathematics. This special agent was at first distinct from the permanent agent – also, e.g., in terms of appearance to accentuate the difference visually. After the year of mathematics, however, most features were merged into the permanent agent. Also in 2007, the former standalone system was migrated to a system driven by a Live-DVD and the agent became mobile. This feature had also been used for creating the special agent. This special agent was so successful that, starting in 2009, it toured through several museums as part of a smaller installation, only comprising the main system unit, a computer display, a sound system and a keyboard. In 2010, the agent has been trained in photography and is now happy to take pictures of visitors, compose an electronic postcard and send this postcard with some greetings to an email address given by the user. This process is fully interactive and handled in a natural dialog. Considering the demonstrated interaction capabilities, this is the first time where the agent can interact with the visitors outside of the museum, beyond the direct face-to-camera interaction at the exhibit.

3 The Installation 3.1 Presentation and Interaction The exhibit is located on the second floor of the museum, right at the end of the main course through the permanent exhibition. It is the area where many high-level computer systems, A.I. systems and robots are located. The main component of the installation is the large canvas where the agent is displayed using a front-projector mounted to the ceiling. The agent is target of many group interactions. This is considered by the installation by exposing the agent above ground and oversizing him a bit, so that everyone in the group has a good experience. Voice and other sounds are made audible by a set of speakers attached to the mount of the canvas. Interaction with the agent is supported by two modalities: typing on a keyboard and visual interaction. The keyboard console is located right in front of the canvas, at about 1m distance. The console and keyboard is ruggedized and well suited for frequent visitor interactions. Special care has been taken to disallow the visitors to interact with the underlying operating system, thus special keys, key combinations and functions such as task switching have been deactivated. A mouse or touchpad interface is not required. Visual interaction is supported by a video camera which is mounted right beside the canvas to the left of the agent. It is focused on the area right in front of the console to take a close-up of the visitor interacting with the keyboard.

3.2 Software and Computer System At the time being, the agent itself is hosted on a 2.66 GHz Quadcore system with 3 GB RAM, a NVIDIA Quadro FX 1700 and a Brooktree Bt878 framegrabber. This system is located in a server facility next to the exhibit. The computer system is running on Ubuntu Linux. The graphics are handled by SGI OpenGL Performer. Since 2004, the hardware has been completely exchanged once, in 2009, to migrate to the modern multi-core architecture, allowing for advanced computer vision techniques, which are the basis for recent feature additions.


The software pattern used to create the agent is a multi-agent system. Thus, the overall experience provided by the agent emerges from the collaboration of many software experts, which have been constructed to cooperate on this task. There are software experts to receive keyboard input, to parse natural language, to manage knowledge and the progress in dialog, to realize the visual appearance, to plan synchronized speech and gestures or to verbalize the generated utterances. The A.I. group has used this architecture for over 15 years – but it is more than ever up-to-date, because it scales well with the multi-core systems that are state of the art today. 3.3 Maintenance Running a complex installation such as the virtual agent requires a robust strategy for update and maintenance. In the beginning, the success of the installation had not been foreseen and long-term maintenance had not been an issue in the design of the installation. Considering the fast progress in computer hardware and the high iterations of updates of operating systems, seven years are a long time in terms of computer technology. Thus, up to today, major changes happened regarding computing power, starting with single core systems and nearly non-programmable computer graphics cards in 2004, and are now definitely not in a finite state in 2011, where there are multi-core CPU architectures and graphic cards with hundreds of programmable cores. At the same time, the operating systems version number has changed from version 4.10 to 10.10. The changes in the operating system software included major changes in compiler technology, which directly affected the overall software installation. In addition to these problems, over the last seven years several functionalities have been implemented to support maintenance regarding the functional part, the interaction with the visitors: - A verbose logging system has been implemented to assess the interactive sessions after a failure has been detected. - A log-file rotation scheme has been realized. Log-files are normally stored on the hard drive of the computer system. The insertion of a USB stick in the securely stored system triggers a process that copies the current log-files to the stick. This simplifies the on-site administration process. This feature was especially useful for the mobile installations, as they did not have a permanent connection to the internet. - A remote administration service has been set-up, which allows the scientists from the A.I. group to access the system on-line in case of problems. This is the second alternative to copying the log-files as the most relevant source of analyzing system problems. - Since 2010, the system also automatically composes emails with excerpts of the recent log-file activities in the case of unexpected failures and reboots. This simplifies the reaction scheme in the case of incidents and provides just-in-time reporting. The staff at the museum does no longer need to report every incident, but can provide elaborate feedback on the situation at the exhibit if unknown problems are encountered. - The Live DVD system which has been installed in 2008 increases the robustness of the installation. By swapping the Live DVD, the staff at the museum is now able to exchange the hardware or to revert to a previous version of the agent known to work if a new feature shows teething problems. With all of the aforementioned maintenance functions in place, controlling and administering the agent has become a lot easier for the scientists, who can now focus on solving problems and implementing new features.

4 Acceptance and Feedback 4.1 Qualitative Feedback of the Museum Guides The museum employs freelancers as museum guides. They complete a specific training offered by the museum and successfully pass a control tour. Many of them have already given several hundred tours. For the following collection of qualitative feedback, the six most active guides have been interviewed and asked for their comments along a collection of questions regarding the acceptance of the system, the typical questions asked, anecdotes or the kind of visitors that are interested in Max. (Interestingly, one of the six reported that he deliberately does not include Max in his tour, but refrained to disclose his reasons.) The overall acceptance of the system as perceived by the other museum guides is excellent. The agent is seen as one of the most popular attractions of the exhibition and receives a lot of attention, both from the young and the young at heart. The demand to challenge the artificial intelligence is great. As the


agent is in most cases the final highlight of the guided tour, visitors often remain at the console over the end of the tour. Some were even reported to have missed their bus. The guides are reporting that especially younger visitors approach the agent without reservations. They even address him via direct speech although they had been explained that he can only be talked to via the console just right beforehand. This can be interpreted as an immersion of younger visitors into the illusion that the agent is a real conversational partner and underlines his inherently consistent appearance. When talking to the agent, especially young pupils tend to ask embarrassing questions and use awkward formulations they would never try on their parents. The agent has been adjusted to this observed behavior and retorts accordingly. When the situation escalates, he protests and leaves the screen, only returning after a proper apology by the visitor. Overall, visitors are perceived to be excited about the intelligence and the friendliness of the agent. However, the tour guides of the museum noticed a difference in the attitude towards the agent between adults and children. Children like chatting with the agent very much. The adults appear to be concerned about robots and agents replacing and surpassing humans. When interacting with the agent, though, they notice his bounded intelligence: he often gives wrong answers if he does not recognize words or has no knowledge about a topic. On the other hand, he is perfect at German grammar, which most visitors – be they national or international – are not. Yet if something is spelled incorrectly, he would not recognize it. In the end, visitors are either surprised by the intelligence displayed by the agent, or they leave with a smile. Whether this is due to the witty character of the agent or to their relief that agents will not take over the world tomorrow the museum guides cannot tell. The museum guides also report on many repeated visits. Often, some members of a group of visitors already know the agent and are happy to meet him again. There are also always some visitors who return to the agent at the end of a tour or walk there repeatedly during their stay. This is especially true for groups of seniors. 4.2 Questions Asked The range of questions asked by visitors covers topics such as personal background, physical attributes, up to philosophical issues. Typical questions, which have been compiled by a museum guide, are: - Do you have a girlfriend? - How old are you? - How tall you are? - Do you like soccer? - What’s your mother’s name? - Where are you from? - Why are you here? - Are you a victim? - Can we dance? - What is the answer to the universe and all the rest? - How’s the weather tomorrow? - Will you be a robot some day? Interestingly, there are also questions about the price of such a system and whether it will be available for sale. In addition, visitors are interested in the use of such a system. One museum guide remembers a young pupil asking whether the agent is also good at doing homework. And sometimes, they even ask the agent to be their friend. These are the questions posed by visitors which are aware of being observed by a museum guide or which are part of a group tour. The log-files, however, can tell a different story by revealing the questions asked when visitors are feeling unobserved. Groups or individuals which are not part of a tour often pose quite explicit questions, use vulgar expressions or exhibit a crude behavior towards the agent. Unfortunately the percentage of visitors showing this behavior is comparably high. Some are even undeterred by other visitors or officially guided tours.

4.3 Some Statistics from the Logfiles The logfiles between November 2010 and January 2011 have been analyzed to generate some statistics about the interactions with Max. During the time of regard, the museum has been open on 75 days. On 127

average, about 21 direct interactions were started by visitors a day by using the console. Of them, 53% (11.36) were conversations of appropriate form, including a formal greeting, exchange of names, several exchanges of small talk, game playing or presentations, and a farewell. The remaining interactions consisted only of one or two sentences. The mean number of inputs made during such conversation was about 23 sentences, the mean duration about 13 minutes. Note that typically a group of visitors has a conversation with the agent with interleaving inputs, so the number of visitors that interacted with Max is greater than the number of conversations. These numbers have to be taken relatively to the numbers of visitors per day, which is on average about 390 (estimated on data from 2004 to 2010). And they have to be taken relatively to the number of about 2000 exhibits concurrently being presented in the permanent part of the exhibition, not counting temporary special exhibitions. Each day, Max responded to about 387 sentences. In 75% (290) of the cases the agent was able to make an interpretation of the sentence, i.e. Max had an explicit rule matching the input. In conversations (852 total in 75 days), Max made a direct offer to give a brief presentation about the exhibition in 61% (523 total) of the dialogues, which was accepted in 21% (110 total) of these cases. On average, he asked visitors to play a game with him in 32% (269 total) of the dialogues. This offer was accepted in 62% (168 total) of the cases. Regarding the emotional system of Max, in about 6% (1806 total) of the cases the sentence entered had been an insult which provoked a negative emotional reaction. Only a bit less than 1% (281 total) of the sentences were interpreted as a compliment, which led to a positive emotional reaction. To compare, De Angeli and Branham (2008) found in a similar analysis of logfiles from the online chatterbot Jabberwacky a typical conversation length of about 41 inputs. This is nearly twice as many as the 22.57 inputs found in conversations with Max. Interaction with Jabberwacky, however, happened online and interlocutors were probably sitting in front of their computers having a private chat. Also, interaction is purely based on text and answers from Jabberwacky are quick and short. In the museum, the users are standing in front of Max and his answers are spoken out loud. They are elaborate and span several sentences, as Max is primarily an information-giving agent. It can thus be assumed that the overall length of conversations in terms of time actually may have been longer with Max than with Jabberwacky, but timespans are not reported in their analysis. Regarding the insults and abuse of language, they come to similar, but slightly higher results, which might be attributed to the more private interaction with an online system as opposed to an interaction in public space where some responses from Max could have been overheard.

4.4 Anecdotes The agent has seen some famous visitors, such as in 2005 the federal chancellor Gerhard Schröder (see Fig 3, right). In 2006, for another example, the federal president of Germany, Horst Köhler, visited the museum during summer (see Fig 3, left). Mrs. Köhler, who was joining him, addressed the agent with the question: “Is it not too hot for you?” and the agent answered: “Going to the swimming pool would be just the right thing to do today.” Given the formal circumstances, this bold answer caused quite a laughter. As already mentioned, the agent’s 10th birthday has also been celebrated officially at the museum. There has been a short speech, coffee and a marzipan cake for all of the participants of the A.I. conference held at the very same day and place. This event even has been taken up by the local media and it could have been the first official birthday party for a virtual agent.


Fig 3. The agent is well-known and has been visited by famous German politicians, such as Horst Köhler (federal president of Germany 2004 – 2010, to the left) with his wife Eva Köhler on July 11th, 2006 and Gerhard Schröder (federal chancellor of Germany 1998 – 2005, to the right) on July 5th 2005.

5 Discussion 5.1 Lessons Learned Managing and maintaining a large software architecture such as that of the agent Max (see Section 3) is a challenge for a research group. In science, especially in A.I., one is primarily concerned with developing theories and creating software prototypes as a proof of concept. There normally is only a restricted group of users interacting with the systems and the scientists are always around to help and fix. This is quite different from the situation at the museum. The installation has to be up and running six days a week for up to ten hours. There is no expert around if something breaks. The reason for a broken system is often found in unexpected clever or just lucky user input of visitors which are no longer around to ask what they have done. Consequently, the first two years have seen many incidents and some tense interactions between the staff of the museum and the A.I. group. However, both sides and especially the agent have learned a lot during this phase. Reviewing the results of the logfile analysis, it is interesting to see that there are a lot of insults targeted at the agent. This could be explained as follows: The visitors are explicitly made aware of the fact that


the agent shows emotion. It is thus quite natural that they try to test him by throwing insults at him, just to elicit his reactions and, for example, make him leave the screen or show some other interesting behaviors. Visitors could just be checking out his boundaries, just as children do. However, looking at the logfiles, one finds that often dialogs later leading to insults had started quite naturally. In most of the cases, it is only when the agent got something completely wrong that visitors turned to insulting him. This is a further motivation for the A.I. group to improve the dialog capabilities of the agent. Still, this observation also shows the strengths of the strategies implemented in the agent: If it was not for the emotions, the visitors would probably just have dropped out of the dialog, as soon as they had lost the immersion and started to disbelieve in the intelligence of the agent. With the dynamics of the agent’s emotional system, some myths still remain and keep up their interest.

5.2 Future Vision and Plans Having a virtual agent living in a museum for seven years, one vision that repeatedly comes up is that of allowing the agent to make one step beyond his nice but narrow quarters. With the photo-postcards one first approach is taken were the agent is enabled to contact the external world. But this approach is very restricted. More interactive examples could, e.g., be used to enrich the presence of the museum enabling visitors to get a first glimpse of the system on the museums webpages. This would, however, require support for real-time 3D graphics and sounds on the client-side – technologies which are just in the progress of being taken up by the general public. As the virtual agent is accepted as a human-like communication partner, he could also make use of other communication channels. One example is Facebook, where he could keep in contact with the visitors or announce upcoming events. As Max is an emotional being, he could also report on the visitor interactions he had each day: if there would have been a special event and many visitors would have interacted with him, he could, e.g., comment on the fun or the stress he had that day. Finally, modeling the agent’s real-world knowledge manually is a very time consuming task. The agent is therefore only equipped with a very restricted knowledge, as has been explained before. An interesting resource for common knowledge is collected by thousands of people in their work on wikipedia. Current research of the A.I. group is thus focussing on accessing this huge ressource and thus broadening the range of topics the agent is able to talk about.

Acknowledgements The work on Max has been supported by the Deutsche Forschungsgesellschaft (DFG) and the Heinz Nixdorf MuseumsForum (HNF). The authors would also like to give thanks to the reviewers for their constructive feedback.


References Becker, C., Kopp, S., and Wachsmuth, I. (2004), “Simulating the emotion dynamics of a multimodal conversational agent”, in E. André et al. (Eds.): Affective Dialogue Systems (pp. 154-165). Berlin: Springer (LNAI 3068). Cassell, J., Sullivan, J. and Churchill, E. (2000), Embodied Conversational Agents, MIT Press. De Angeli, A. and Branham, S. (2008), “I hate you! Disinhibition with virtual partners”, in Interacting with Computers Volume 20 Issue 3 (pp. 302 – 310). Heinz







Heinz Nixdorf MuseumsForum (1999), Museum Guide, Paderborn: Verlag HNF – Heinz Nixdorf MuseumsForum. Jörding, T. and Wachsmuth, I. (1996), “An Anthropomorphic Agent for the Use of Spatial Language”, Proceedings of ECAI'96-Workshop Representation and Processing of Spatial Expressions (pp. 41-53), Budapest. Jung, B. and Kopp, S. (2003), “FlurMax: An Interactive Virtual Agent for Entertaining Visitors in a Hallway”, in T. Rist et al (eds.): Intelligent Virtual Agents. 4th International Workshop, Springer, LNCS 2792. Kramer, N., Simons, N. and Kopp, S. (2007), "The effects of an embodied conversational agent´s nonverbal behavior on user´s evaluation and behavioral mimicry", in Proc. of Intelligent Virtual Agents 2007 (pp. 238-251), Berlin/Heidelberg: Springer-Verlag. Kopp, S. and Jung, B. (2000), “An Anthropomorphic Assistant for Virtual Assembly: Max”, Working Notes Workshop Communicative Agents in Intelligent Virtual Environments, Autonomous Agents 2000. Kopp, S., Gesellensetter, L., Krämer, N., and Wachsmuth, I. (2005), “A conversational agent as museum guide – design and evaluation of a real-world application”, in Panayiotopoulos et al. (eds.): Intelligent Virtual Agents (pp. 329-343), Berlin: Springer-Verlag (LNAI 3661). Von der Putten, A., Reipen, C., Wiedmann, A., Kopp, S. and Kramer, N. (2008), "Comparing emotional vs. envelope feedback for ECAs", in Proc. of Intelligent Virtual Agents 2008 (pp. 550-551), Berlin/Heidelberg: Springer-Verlag. Weizenbaum, J. (1996), “ELIZA: a computer program for the study of natural language communication between men and machines”, Communications of the ACM, vol.9.



Touching the Stars Niall Smith, Alan Giltinan, Paul Walsh, Clair McSweeney, Frances McCarthy Cork Institute of Technology, Blackrock Castle Observatory [email protected] Abstract Blackrock Castle in Cork, Ireland, houses an interactive science centre and astronomical observatory. The exhibition within the castle is heavily dependent on multi-media technology, which has many positive aspects but which needs constant updating as technology and content changes. We have re-engineered a popular exhibit that involves sending a live message towards a distant planet using an inexpensive radiotelescope at the Castle so that it is portable and re-deployable without major expense or time expenditure. We report on our experiences with the exhibit at the Castle and its first redeployment at the BT Young Scientist exhibition in 2011. We comment on the reaction of over 1000 end-users to the concept. We argue that deploying technology outside a science centre can be used to link more people back into the science centre, raise its awareness amongst the public, and potentially help to build communities of users on a global scale.

1 Blackrock Castle Observatory (BCO) Blackrock Castle is situated on the shores of the river Lee in Cork and dates back to approximately 1593. Today it houses an interactive science centre and professional astronomical observatory which is operated through a partnership between Cork Institute of Technology and Cork City Council. The purpose and vision of the facility is “to affect positive attitudes toward science, engineering and technology in Ireland and be recognised and respected as a centre of excellence in scientific research, education and outreach32”. Since opening its doors to the public in 2007 the facility has welcomed over 100,000 visitors and has won three national awards and been shortlisted for two others. BCO received a THEA ‘Award for Outstanding Achievement’ from TEA, the world association for themed attractions in Los Angeles in 2008.

1.1 The Exhibition and BCOLabs Central to the BCO experience is a state-of-the-art exhibition that invites interactive debate on mankind’s ultimate place in the Universe33. A gallery of cinema sized high-definition digital video screens with proximity sensors allows visitors to select the content they wish to view, taking them on a journey back to the beginning of the universe and forward to the challenges that will be faced by those potentially colonising other planets. In an exhibit that is unique to BCO, visitors to the interactive theatre use scientific data, which they themselves gather, to launch a virtual space mission to save the earth from possible destruction by a comet. Primary level aged children can explore educational issues in a fun and accessible format in a room called “The Forum” with Cosmo, our friendly virtual astronomer. A laboratory-styled classroom allows groups to prepare and debrief before and after the exhibition experience. An onsite Education Officer facilitates all visits and workshops. BCO houses an astronomical observatory (BCOLabs), staffed by researchers from CIT engaged in developing and applying technologies to search for planets around distant stars using telescopes at the castle and abroad. The close integration of the public exhibition and the astronomical observatory – facilitated by the display of research content which is updated regularly, and by a host of regular public events – makes BCO a rather unique location nationally. Having the research team onsite means the facility is not just a place where information and experiences are recalled, but where new ideas are generated.

32 33

BCO Mission Statement Document Installed by Martello Media, Dublin.


1.2 Multimedia Technology The footprint of BCO is relatively small and the rooms are irregular in shape, so the decision was made at the exhibit concept stage to install an exhibition that is dynamic and flexible, using configurable multimedia technology. Since all our content is multimedia driven it is possible for us to change the content as required for specialist visiting groups, or to develop new content to keep the experience “fresh” for returning visitors. It is also possible for us to customise every display screen to meet the needs of corporate events. A crucial point to note here is that whilst BCO is a science centre, many of the events that take place there would more accurately fit into the definitions of cultural, musical, theatrical or artistic. This is intentional and encourages activities that bridge the perceived (and often actual) barriers between different disciplines.

2 Re-thinking the Technology at BCO The design paradigm for the exhibition assumed that technology, appropriately used, would provide us with cost-effective opportunities into the future and help to keep us relevant and interesting to visitors. It was clear when the initial design began in 2004 that future developments of the Internet, sensor, and wireless technologies would have a profound effect on the way people interacted with the facility. What we didn't envisage was the explosion in the use of social media by all ages and the ubiquity of digital media content in all our lives. How then, do we as a science centre that relies heavily on technology, maintain and grow the interest of the public and enhance their engagement, in a world where technology is becoming more common in people’s everyday lives?

2.1 BCO 'competitors'. Assuming you wish to encourage people to visit your exhibition, a key question to consider is who your 'competitors' are. Our experiences in the past 3 years, based on informal feedback from visitors supplemented with data from focus groups, have taught us some key lessons about using digital media technology: 6. You cannot easily compete with the likes of the Discovery Channel. They can spend millions of euro on a programme that lasts 30 minutes and they can distribute this to millions of viewers. Thus, even the most impressive content, on its own, will not be enough to bring visitors back, especially if the delivery medium is similar to that which your visitors are regularly exposed to (e.g., plasma screen TV’s). 7. You can impress visitors with cinema-sized screens and sound, assuming the content is of a very high standard. Such an environment cannot be created at home, hence it has an element of wow factor associated with it. 8. Content which is up-to-date and which includes a local flavour (not always easy to incorporate, depending on the theme of the facility) is very well received by visitors. The corollary is that content which is out-of-date is very poorly received. 9. The personal touch remains very important, even if the digital content is high grade and interactive. Enthusiastic and knowledgeable guides add significantly to the visitor experience. A personal interaction that lasts for a even a short while has a big impact on how visitors feel about their visit. 10. Location matters. At BCO we find that most first-time visitors are completely surprised by what they find inside the castle, even though our website and marketing material attempts to capture this in advance of their visit. In general, this surprised reaction is positive and the marriage of “21st century technology in a 16th century castle” works extremely well. By providing tours aimed at the history of the castle we also capture the attention of visitors who, despite our best efforts, are less interested in engaging with the main interactive experience. 11. Introducing an element of gaming adds to the excitement and fun of a visit and increases the potential for the experinece to be educational. However, with the ubiquity of commercial technologies such as Microsoft Xbox and Kinect, the gaming experience needs to be different to the norm if it is to be a highlight of a visit. At BCO our “Comet Chaser” cinema has a storyline on the big screen that is actually altered depending on what the audience uncovers on their individual screens as they are guided through the science needed to save the earth from a comet on a potential collision course with earth. Each member of the audience gets a score, depending on how well they perform, and the added element of competition is very well received, often acting as a “talking point” subsequently. Interestingly, the gaming element is as popular with older groups as with younger ones.


2.2 BCO target audience. Whilst workshops are targeted at specific age groups, the target audience for the exhibition is very broad. It includes: • People of all ages, national and international, who can walk into the facility at any given time. The motivation for visiting the facility is varied, but the objective from the centre's point of view must be for them to have a fun, educational and rewarding experience. Ideally, we would like them to recommend us to their friends and family and to consider purchasing one of the options for membership. • Teachers and their classes, for whom curriculum relevancy is very important. Teachers have very limited time to take their students out of the classroom environment, and the clear message is that without curriculum relevancy in the experience at the science centre they are less likely to choose it as a destination. • Corporate and other specialised groups for whom the exhibition needs to be customised to most accurately reflect their needs, interests and target market. • People who interact with BCO by (possibly repeatedly) visiting the website and people who engage with our social media initiatives (notably facebook and twitter).

2.3 Using technology to expand the experience We regularly assess the response of visitors to the various elements of the exhibition and noticed the popularity of our send-a-message to space exhibit (described in section 3). This gave us the idea to extend the basic concept so that it could be used by remote users, initially in the classroom environment, and to introduce the possibility of developing educational modules with curriculum relevancy to go with it. The idea was extended by creating a website that allows users to track their messages. While the exhibit has been running at BCO for 3 years now, here we describe its evolution and deployment at the BT Young Scientist exhibition in Dublin in January 2011.

3 'Send a message to Space' The basic idea behind this exhibit is simple. Using a touchscreen interface, users get the opportunity to learn about sending radio signals from a radiotelescope and then get to select a star which is known to have a planet orbiting it (called an exoplanet), and to send a radio message towards that planet in realtime.

3.1 Methodology 3.1.1 Software architecture The application is composed of multiple hardware and software components as shown in Figure 1. These components can be classified under the following headings: Client 7. User Interface – allow users to easily understand star properties (notably the distance) and select a target star which has a known exoplanet(s) orbiting it. 8. Star position calculation – reads the local time and calculates the positions of stars with known exoplanets that are currently overhead of the radio telescope at BCO. 9. Webcam and keyboard capture – Capture an image of the user(s) and allow them to compose a message with a virtual touch screen keyboard. 10. Store and Forward - Write the message and image to a file and send the message to the server ftp software controlling the radio telescope. 11. Radio Telescope – A webcam viewer that shows the radio telescope slewing to the selected star. A series of flashing LED's show the moment the radio signal is transmitted. Server 7. FTP Server – Receives star co-ordinates and message to send to space. 8. Radio Telescope Controller – controls telescope servo mechanisms to target the correct star and its exoplanet. 9. Webcam Server – Transmits live pictures of the radio telescope to the web client at external sites.


Figure 1 Software architecture. The design of the exhibit requires that the software is easily deployed to personal computers at external sites and that it is able to interact with the local file system and both local and remote webcams. A number of rich internet application (RIA) technologies were reviewed and assessed for their suitability to the project. These included Adobe Flash, Microsoft Silverlight and the open standard HTML5. (Other possibilities include Javascript, Python and PHP and if a particular design team is more comfortable or expert in these development domains they may chose to use them.) While Flash is a mature technology it has been geared more towards designers rather then programmers and we consider it to have an overly complex object model. HTML5 has the potential to be a powerful cross platform technology, but the standard is still in development and is not yet at the point where it can be deployed to live sites. For us, this leaves Silverlight as the best choice for this application because it: 6. supports a local webcam, 7. allows data to be written to the local file system, 8. has a sophisticated programming model, 9. is easy to deploy to remote sites, in our experience. 3.1.2 Hardware architecture and deployment at BCO The hardware required to enable this project is a re-deployment of existing technology. In essence it is the reverse of receiving a radio signal. One needs the same technology in the form of an Antenna (dish), a transceiver, and a decoder/encoder. The hardware list can be broken into 6 main components: 6. 7. 8. 9. 10. 11.

Antenna XY positional Mount (to point the dish at the star chosen by the user) Transceiver (a transmitter-receiver in the form of a standard WiFi dongle) Encoder (PC bitmap encoding) Wireless router (transmission) Server (to hold the local copy of the bitmap picture that accompanies the message)

For an antenna we use a standard 0.9m satellite dish commonly used in household and business for both satellite TV or Satellite internet solutions on an industrial mount usually used in the security industry for outdoor cameras. This mount was controlled via the PC using Labview software and a serial port. The transmitter is a Wifi dongle, which avoids possible ComReg issues, and is a reliable and inexpensive off-the-shelf solution. The message encoding was done in bitmap format and was performed on a PC. The encoding was performed by the software, transferred to a laptop at BCO and saved in bitmap format waiting for transmission. 3.1.3 Hardware deployment at BT Young Scientist The hardware physically deployed at the BT Young Scientist included: • 2 x PC's • VGA splitter • touchscreen • sound system • network switch 136

• 2 x large Plasma screens • autoFTP client The PC's were two standard Dell machines loaded with Windows XP Pro. PC1 was connected to the touchscreen and PC2 was connected to 2 Lancams (webcams that connect straight to a network port) situated in BCO. These showed the users the radio antenna at BCO as it moved to its required destination and also showed a number of pulsed LED's which coincided with the moment the wifi signal was sent. (The wifi signal itself is invisible to the eye, so we devised a mechanism which would at least allow the user to know when the message was being sent.) A touchscreen was used to interface with the audience. It dispenses with the need for keyboards and mouse, both of which are awkward to use in public areas. The sound system – a low-cost system readily available for PC's - relayed the audio commentary that accompanied the interactive software. However,we found that in the noisy environment of the BT Young Scientist the audio was difficult to hear above the general din. In quieter environments, such as BCO or classrooms, the audio works well. All of the hardware (with the exception of the rented 42'' plamsas) was transported using a single car. Total hardware setup time was of the order of 1 hour. 3.1.4 Tracking the message Users who sent a message can track their signal by connecting to a URL on the BCO website. They are asked to input (i) the planet to which they sent the message, (ii) the time/date they sent their message. The website then displays (i) the time since the message was sent, (ii) the distance the message has travelled and (iii) the time to reach the target exoplanet. This is an exciting interactive element to the exhibit as it encourages continued interest and interaction between the student and the exhibit, long after they have left the BT Young Scientist. We are further developing the concept to include additional information on the webpage, for example upcoming events at BCO and breaking science news; the list of potential options is extensive. 3.1.5 Content design Content was produced for the exhibit that would maintain the learning outcome of the exhibit, while at the same time presenting the information in an engaging manner. The main scientific concepts that we wish to relay to users: 12. Stars with one or more planets have been discovered beyond our solar system. 13. Radio messages can reach these planets at the speed of light, but it still takes years because of the vast size of the universe. 14. Radio messages degrade when they travel long distances, so choosing planets closer to the earth will result in a stronger signal reaching its destination. Graphics and voice-over were used to convey these points and were also consolidated into a dynamic animation that presented them in a meaningful and engaging way as shown in Figure 2. This animation shows the relative position of a selection of stars that are currently overhead, scaled to the available screen size. The size of the star represents how far it is from Earth in light years. Animated planets orbit each star to reflect the number of actual planets that orbit the star. Users can also touch each star to get more precise details. Anecdotal feedback from the BT Young Scientist indicates that this latter feature was particularly well-received. 3.1.6 The science underlying the exhibit – not intimidating your audience There is a good deal of scientific concepts that underlie the exhibit, not all of which need to be introduced to the end-user for their session as too much information can be intimidating and the exhibit loses its impact. We have been looking at different interfaces which are customised for different age groups and/or different levels of background knowledge (e.g., different school groups). For most groups the full rigour of the science underlying the exhibit is not necessary, but for some we can introduce challenging concepts for them to discuss subsequently, perhaps back in school. For example, although the message is beamed towards a target star, the radio signal must “compete” with the general radiofrequency noise that permeates the universe. At some distance the signal strength is equal to that of the background noise and at farther distances it is less. While this does not make it impossible to detect34, it does pose challenges for the “person” receiving the signal to extract it from the background 34

By including a modulation key into the data sent by the radiotelescope it is possible to make it detectable against a very noisy background. This is a standard technique that allows signals which are one thousandth billionth of the background noise to be detected.


noise, and this opens up the very important area of extracting signals from noisy environments that has applications in many industries, not just space-related ones.

Figure 2: Screen shot of application showing stars overhead with planets.

4 Response of users 4.1 At BCO Users at BCO interface with the exhibit in a relatively controlled environment. At any given time there are many exhibits to see, so visitors can take time to work through the send-a-message-to-space exhibit at their own pace. There is no time limit and they can use the “back” button as they wish. The degree of learning that goes with completing the exhibit varies widely. Young children (from around 5 years) have no real understanding of the exhibit, but the reaction of their parents is crucial to whether it’s a fun experience. Interestingly some parents see the opportunity to send a message into space on behalf of their child as a memorable event and one which their children can look back on in future years by returning to the associated website where they can track the progress of their message. Older children have a varied reaction, but almost all seem to enjoy the composing of the message (“what shall I write ….”?) and watching as the radiotelescope moves on the roof above them via a landcam. Teenagers are the most difficult to excite, but even they seem to be interested in the message tracking facility.

4.2 At the BT Young Scientist Exhibition The BT Young Scientist exhibition has about 30,000 visitors over the course of the four days when it is running. Such large numbers of visitors pose challenges for exhibitors. Any exhibit must not take too long to complete – there is simply too much to see and nobody wishes to queue. The same issue does not necessarily arise in other scenarios (such as classrooms), so any multi-site exhibit design must be flexible enough to be deployed in a wide range of scenarios. Nearly 1000 students, their teachers, parents and general public, sent a message to an exoplanet over the course of the exhibition. The following observations can be made: • The idea of sending a real message towards a real exoplanet was almost universally well received. • The ability to see a real radio telescope move at BCO in realtime to point at the selected exoplanet target was received with some lack of credulity. Comments such as “is that really happening now” were common. This is somewhat surprising, but nevertheless instructive


from the viewpoint of an exhibit designer. The flashes of light from the radio telescope which took place at the moment the radio signal was sent, and which were for illustrative purposes only, added enormous impact to the experience and were regularly met with comments such as “wow”, or “cool”, etc. We almost did not add in this feature to the exhibit (it was not incorporated when we were running the exhibit at BCO), but it will now most certainly stay as an integral element. The high-quality graphical interface using the touchscreen was very successful. Feedback from users suggests they did, for the most part, understand what they were doing because it was explained in carefully chosen graphical terms that guided them in an interactive way through the process. Crucial to the success of the interface was the ability to choose the planet you wanted, and to which you wished to send the signal. The idea that the signal sent by the radiotelescope would have passed the moon within 2 seconds of being transmitted, and would outpace the most distant spacecraft ever launched within eight hours, brought smiles to most faces and often initiated further conversations. The ability to track one's message by connecting to a website address was a big success. Subsequent evidence shows that about 30% of users have connected to the website on at least two occasions since the BT Young Scientist to check the status of their messages, some two months after they sent it. This hit rate could likely be improved by upgrading the web page to include additional information, as noted above. Presently the web page is rudimentary.

5 Future Plans Given the positive feedback from the deployment of the Send Message To Space project at BCO and at the 2011 BY Young Scientist Expo, we have been encouraged to explore a number of avenues for the exhibit and are developing plans for future projects based on the lessons learnt from our experiences. As mentioned above, the exhibit is currently installed in BCO where it is available to the public. It is also used in classroom settings with groups at BCO where the lessons delivered by sending a message to space are reinforced by a teacher working in “blended mode” (Graham 2005). Given the relative ease of deployment of the exhibit, we plan to visit schools in the region to educate pupils on the science of astronomy and to raise awareness of the centre. In time, we plan to add more educational aspects to the whole experience. Teachers will be provided with a resource pack so that they can discuss concepts such as (i) what are stars and planets, (ii) what is radiation, (iii) how far away are the stars, (iv) why did our ancestors arrange stars in constellation patterns, (v) is there life in space and what do you think alien life might look like, and (vi) what might a distant planet look like if you were to land on it? Sending a message to an exoplanet provides a unique context in which a teacher (or parent) can frame questions that cross almost all discipline areas, including social science, art, history, etc., along with what might be perceived as the 'usual' disciplines within science and engineering. The success of the hardware and software architecture used in the project has also encouraged us to consider new concepts that could be implemented using this technology. The linking of global communities of pupils with scientific instruments world wide is one possibility. We have strong international links with educational and scientific institutions, ranging from Chabot Observatory in Oakland California35 to the University of Pune, India and we are considering how to use the send-amessage-to-space concept to connect schools and universities in those regions together. Once connected, they can interact in multiple ways of their choosing. In this way the new technology enables our science centre to act as a catalyst for ideas, conversations, connections, without the need for users to physically visit the centre. We will, of course, encourage them to do the latter to broaden their experience where this is possible. We are currently discussing test deployments of the exhibit to other science centres and planetaria, two of which are in the US. These will be via the same web-based interface. In addition to the exhibit itself they can potentially act as a light-touch tourist portal. In a recent exciting development, we will be using a decommissioned 32m radio telescope in Ireland to send the message to space, but we do not wish to replace the concept of the small, inexpensive, reproducible radio telescope that is at the heart of the exhibit. We have considered making the exhibit accessible through a general web interface, such that anyone on the web can send a message to space. While this is possible technically, there are issues with scheduling and queuing if the volume of uptake is high. Would people be willing to put a message into 35


a queue that might only be sent some hours later? One possible solution is to deploy multiple small radiotelescopes that are connected together dynamically and when a radiotelescope becomes free it takes the next message in the queue and sends it. This approach introduces some uncertainty about the location from which the message will be sent, which could potentially be across different continents, and this could add some additional interest for the user (and educational opportunities for the teacher). There may be some internet security issues, but in any event it is a possible evolution that we intend to consider further. The system deployed by us relies on touch screen technology, which is now considered as a popular paradigm for human computer interaction. Although it is a technology that has been around for some time, the emergence of the smart phone and tablet market in recent years shows that systems based on touch screen technology have massive popular appeal and that users readily accept this mode of interaction (vom Lehn 2005). We can now deploy more interactive exhibits based on this technology, due to our experience in developing the Send a Message to Space exhibit. We can reach out to more schools and bring them “into” BCO via this exhibit and also via other initiatives such as our portable planetarium (Stardome) and our workshops. This is re-thinking technology – bringing the technology directly to the end-user and using it to promote awareness of BCO itself.

6 Conclusion Our experience in re-thinking technology in our own context has been a very positive one. We reexamined existing exhibits within our centre and devised an ambitious re-engineering of the technology to create a highly interactive touchscreen exhibit that links remotely via the internet to real scientific instrumentation. Users are given control over the technology and their inputs are relayed back to them using a live web cam feed. The exhibit is easily deployed to sites and can be transported easily, making it an ideal platform for schools outreach. We plan to build on this success and re-imagine a range of new exhibits that will engage pupils with science and technology and ultimately encourage them to broaden their horizons.

Acknowledgements We would like to acknowledge a President's grant from Cork Institute of Technology which made possible the deployment of the exhibit to the BT Young Scientist. Support from Cork Institute of Technology and Cork City Council is crucial for the ongoing operations of the science centre at BCO.

References Graham, C. R. (2005). "Blended learning systems: Definition, current trends, and future directions.” In Bonk, C. J.; Graham, C. R.. Handbook of blended learning: Global perspectives, local designs. San Francisco, CA: Pfeiffer. pp. 3–21. ISBN 0787977586. Dirk vom Lehn (2005), Accounting for New Technology in Museum Exhibitions International Journal of Arts Management, Vol 7(6): 11-21


Re-thinking museum assemblies Dagny Stuedahl & Ole Smørdal InterMedia University of Oslo [email protected] [email protected] Abstract Museums increasingly embrace social media and new forms of visitor participation. There are a number of reasons for this trend; museums want to become more relevant for their visitors, enhance visitors’ experience of exhibits, raise visitor numbers, and reach out to new audiences. There is an attention in research and in museum practices toward the implications of this transformation for design; i.e. how will it affect how we regard and design for new relations between museums and visitors, and what is the nature of the productive activities and knowledge building carried out by visitors in these new hybrid exhibition and collective spaces? We reflect upon a conceptual framework for thinking about design of hybrid museum exhibitions that integrate visitors’ onsite and online activities based on the concept of enacted assemblies. We advocate including visitors’ activities as constituting actants of an exhibition assembly. This implies integrating visitors’ activities such as co-creation, re-mixing and mediation of multiple interpretations in relation to artefacts are parts of the exhibition design. We discuss the solidity of enacted assemblages as framework in relation to our design cases; one focused on mobile phones and social media as means for involving younger visitor groups and the ways museums might embrace their digital literacy. The other focuses on how social media can be used to open museum processes – and in this way invite to visitors enactments in the form of making connections.

1 Introduction The current use of social media as means to develop the concept of the participatory museum (Simon 2010) involves multiple endeavour, such as building community relationships based on blogs, online exhibition curation on YouTube, collection management on Flickr and museum learning on online forums (Russo et al. 2008). Social media have been embraced due to their potential for visitors to cocreate and interact socially, and because they meet with the call for museums and heritage institutions to be responsive, democratic, and reflective and subsequently take `museum conversation` beyond the museum (Black 2010). Meanwhile, the integration of social media into museums’ curatorial and pedagogical practices preserves a situation where these media primarily are used to engage visitors in short term voting and rating, or to engage communities in collecting images (Russo et al. 2008), and there seems to be a potential for developing a conceptual understanding of the relations between social interactions as visitor enactments, hybrid exhibits with digital media, immaterial knowledge, and artefacts. Our intention is to conceptualize exhibition assemblies as constituted by visitors’ enactments of the exhibition; the physical and digital artefacts; the material and the immaterial are enacted by visitors’ interpretational activities. These enactments can be based on visitors` co-creation and coparticipation, where visitors share, combine, renew personalize and customize the museum content; all activities that position the visitor and his/her activities as a central actor in the exhibition. In this way we understand the role of visitors in exhibitions as actants that actively bring the exhibition to life by collective and individual interpretative acts that shape their emerging experiences of the exhibition. This perspective on integrating visitors enactment as actants in museum assemblies refer to ongoing discussion of how to integrate peoples expectations of abilities to engage in co-creative experiences (Sanders 2006 and 2008) into design, and the different ways that design in use can be realised in practical design (Ehn 2007, Björgvisson, Ehn, Hillgren 2010). We will in this exploratory paper report from two case studies where we use the concept of assemblies to capture visitors` enactments in relation to two separate exhibitions based on cultural historical reconstructions; first, an exhibition experiment on a reconstruction of a Viking boat, second, a late medieval boat reconstruction located in an open workshop at a maritime museum. Designing for visitors` participation in exhibitions becomes a matter of finding design concepts that constitute the social interaction and co-composition that are sustained by social media as part of the exhibition assembly.


Our research questions are related to the conceptual approach to integrating social media as part of participatory exhibitions, and we ask how we can re-think museum assemblies to integrate the participatory activities that social media can bring into the museum?

2 Assemblies as enactments The notion of assemblies is well known in museum literature designing the function of museums to collect objects from collections and organise them into displays and exhibitions. In CSCW and HCI literature assemblage has been used to name the transition from a focus on individual devices with single interfaces, towards a coherent assembling of multiple and hybrid devices which directs the design attention towards practical assembly of artefacts to support coherent experience (Fraser et al 2003) and the placement, arrangement, ordering and organisation of an array of technologies (Hindmarsh et al 2005). This also builds the background for the emergent use of the notion of assemblies to bring attention to the relationship between interconnected and interrelated artefacts and technology in museums (Hindmarsh et al 2005) and how to knit together multiple artefacts into a coherent visiting experience of co-participative and collaborative activities (Bowers et al. 2007). The museum context demands that ubiquitous computing, intelligent and ambient environments, augmented and mixed reality all fuse concerns of technology in museums into exhibition design (Hindmarsh et al 2005), and the introduction of social media into museums and their exhibitions do require a conceptualisation of visitors’ co-creative production of content in relation to the exhibition content. In this way, the discourse on participatory museums and the focus on visitors’ co-composition address the role of visitors activities as part of the exhibition assembly and in relation to museums as knowledge institutions. We have, as a framework for our design work used the notion of assembly from ActorNetwork Theory (Weibel and Latour 2007). Assembly in this perspective conceptualizes exhibitions as spaces of enactments, which open new alliances between authors, work and the observer. Assemblies are constituted in part by the activities of visitors and in part by the material objects that make up the exhibition (Yaneva 2003). This understanding differs in that the socio-material interactions are regarded as a constituent part of the assembly.

2.1 A conceptual framework for enacted assemblies The conceptual understanding of assemblies that we draw upon is based on actor network theory. What makes ANT especially attractive for design research is its focus on the diversity of ‘actants’, on interactions as negotiations, on the divergent understandings of the ‘what’ and the ‘how’ involved in collaborative processes. In the ten years of ANT related writings and discussions the uses and the focus of the framework have developed. From being used as a theoretical framework for analysing power structures in development of technology as successful stories, ANT is now more and more understood as a methodological framework for describing complex processes in heterogeneous networks (Law and Mol 2002). It is by taking multiple points of departure, with ‘actants’ not just relating directly but also as part of the cultural, historical, institutional, and political context of a project, that ANT addresses complexity. ANT provides in this way a semiotic framework for making available descriptions “which differ in important ways from many traditional social science approaches” by providing “an attitude and method emphasising sensitivity to the multiplicity of world-making activities” (Gad and BrunJensen 2009). ANT as a framework for studying design proposes several experimental ways of following relevant actors in their contextual networks and networks of translation, with an emphasis on heterogeneity and multiplicity. ANT based studies do not necessarily focus on the weak relations, the actants with less capability or potential for a network and the actants that do not pass the obligatory points or align to the network. What relevance do they have for the building of relation between the actors inside? The critique has been successful in that is has drawn attention to ways of thinking of difference. Marilyn Strathern points to that relations can be based on difference and discontinuity, and that in some cases the european-US understanding of relation as kinship, does not fit (Strathern 1996). This critique draws attention to our conception of what a relation is (Hetherington and Law 2000). ANT focuses on agencies instead of actors, taking capabilities and potential as the departure point. The more correct concept would therefore be to speak about actants as the role that makes the actors move in networks. The actant can be individuals or collectives, can be humans or machines; actants are the driving force of the network building activities (Callon 1986; Latour 1991). Networks inscribe actants with values, programmes or facts, or actants describe networks. But actants also negotiate the programmes of a network, actants persuade other actants to get allied to make their programmes


strong, they align or do not align with existing networks, and they accept or do not accept obligatory points of passage set up by powerful actants.

2.3 Design sensitivity for enacted assemblies Furthermore, ANT draws attention to the relational and non-singular aspects of objects and of materialization processes. Objects are performed and they are emerging. Storni explicates this view with respect to design, arguing that rather than talking about ontological multiplicity where the object becomes a completely different one in different places, it seems preferable to talk about metonymic plurality where the object does not become different, but is rendered such as it activates different elements, features and characters of the same (Storni 2007). ANT draws attention to the ‘politics’ in design, through its understanding of design as inscribing the object, the medium or the materials with competences, motives, and political ‘prejudices’ that will shape the relation of the user: ”Designers thus define actors with specific tastes, competences, motives, aspirations, political prejudices, and the rest, and they assume that morality, technology, science, and the economy will evolve in particulars ways. A large part of the work of innovators is that of inscribing this vision (or prediction about) the world in the technical content of the new object.” (Akrich 1992: 208). In the Norwegian and Islandic context the concept of ‘Thing’ is deeply related to democratic processes. Latour has pointed to the Norse Stringent as an example of how things are related to power and human values, as well as to physical objects: “Thus, long before designating an object thrown out of the political sphere and standing there objectively and independently, the Ding or Thing has for many centuries meant the issue that brings people together because it divides them” (Latour 2003). The process from thing to object is a negotiation process between not only the beasts of the material in the forefront. But also the socio-material history of this beast; the circulating references and the tacit knowledge that is bound to the material, that will be negotiated and transformed into new knowledge. This gives an understanding of ‘relations’ in actor networks as including a relation to the material (the physical negotiation with the material), as well as a negotiation with former practices and knowledge traditions that are culturally, as well as socially established and related to the same material (Stuedahl 2004). Thus, design for the participatory exhibition assembling use of social media is a matter of creating relations between the content matters of the thing and visitors’ media practices, and developing perceived affordances that trigger exploration and co-composition. Understanding exhibitions as assemblages of both interaction and communication, related to both the material and the immaterial content matters, guided us to rethink participation by young visitors. These understandings of exhibitions as assemblies that are not only appearances, but rather processes of actions and interactions, bring forward a perspective of the exhibition assembly as enacted by means of visitors’ co-composition, production and sharing. The visitors participate in interpretative actions and assemble the exhibition by producing images and representations with mobile phones, remediating and sharing the material on social media. Also the framework of assemblies provided us with concepts to rethink visitors’ participation in exhibitions in relation to their existing media practices and perceived affordances of the technologies introduced into the assembly (Stuedahl and Smørdal 2011 a forthcoming, Stuedahl and Smørdal 2011 b submitted).

3 Two cases: Experimental exhibition designs We will use two experimental exhibition design projects related to cultural historical reconstructions, that the authors have conducted as a backdrop for explaining how we integrated social media and participatory activities and to discuss exhibitions as enacted assemblies. The two cases are both related to reconstructions of boats; a Viking boat and a boat from late middle age. The two cases differ in than the first was a “lab” experiment, and the second an ongoing experiment at a museum workshop space. Both design cases are based on involving social media as an integrated part of the exhibition design, as well as of the interaction design. Also the two design cases explore participation in the form of opening the reconstruction process and inviting the visitor into enacting the assemblage in an exhibition – or in the workshop as a contextualisation of the live activities of a craftsman.


3.1 Visitors enacting assemblies by collecting, reflecting and sharing The first design case was related to the reconstruction of a Viking boat. The third Gokstad boat was found in an excavation together with the Gokstad longship, and the reconstruction was based on translating the excavated, wooden fragments stored in the museum magazine into cardboard representations and then in to building a full scale replica in wood, considering the many missing pieces when shaping the boat. The reconstruction process gives attention to doubts and uncertainties, and provides a practical contribution to present discussions on reflexivity in cultural history research (Planke 2005).

Fig. 1 A tangible cardboard model was set up for the visitors to experience the puzzling together of boat fragments. The poster marks the Bluetooth zone of model building, and push triggering questions about the act of reconstructing on the visitors phones. In this design case we used social media and mobile phones as prospect technologies to communicate the complex and contradictory aspects of reconstructions to young people. We set up a “lab” experiment to explore participatory activities such as visitors collecting clues, and remixing and co composing arguments with their camera phones as part of assembling activities that enacted the exhibition. We realised visitors` enactments by building an infrastructure that supported the young visitors’ use of mobile phones as interpretative and collecting tools to take photos and record video that materialised their interpretations of the exhibition according to the tasks. The enactments was defined as activities of collecting, reflecting and sharing that was realised in the exhibition in following ways: Collecting: Using their mobile phones, the visitors collected clues in the exhibition by the downloaded material, as well as producing their own video and photo to answer the questions that were posed. The photos and text-messages they collected and produced were shared with others on the collective wall, as well as on their blog page. The visitors were in this way invited into co-composing in terms of collecting pre-produced and self-produced pictures, videos and text, co-composing these collaboratively on the wall or in the blog. Reflecting: Visitors had to figure out how to solve the tasks that are presented to them. By doing this, they have to reflect upon the content of the different artefacts and media in the space, aligning the themes as well as the contributions from peer visitors into an understanding of the questionings and


doubts prevalent in the reconstruction. Sharing: Visitors were invited to share their views by sending text, pictures and short videos to the system that was projecting them on the collective screen. In this way, visitors could both contribute with reflections and pictures to the exhibition space and save their contributions on the blog. The concept of sharing was directed towards the collaborative aspects of experiencing an exhibition, as well as the collaborative aspects of contributing to an exhibition. The blog was intended to be a social media where they could remix and share images and reflections. In view of the character of the permanent exhibition in the Viking Ship Museum, which doesn’t invite to interactions, taking photos and video recordings seemed an appropriate way of offering an activity that supported collaboration and participation among young visitors, as well as interacting with the exhibition. Providing prompts and triggers for these enactments was a parameter of the design on the same level as planning the artefacts and functionalities of the digital media in the space. It was important for us to design for visitors’ enactments that resonated well with young visitors’ media practice, and which at the same time communicated the doubts and multiple interpretations of the reconstruction. These triggers were designed as part of the assembly as perceived affordances (Norman 2004) for enactments in the exhibition space that would link the collaborative, compositional and spatio-temporal aspects of visitors’ co-creations related to the objects and the narrative of the exhibition. Using assemblages as enactments as a conceptual framework for our exhibition experiment helped us to define visitor co-creative participation in the exhibition as acts that assemble clues and arguments in multiple media types. Their understanding of the reconstruction process was by this enacted by co-creating photos, statements and critical questions and sharing these with peer visitors on the wall-hang screen. The actors of this assembly are both human and non-human, the artefacts and media and as well as the interactions, and includes the material; i.e. the fragments, models and the boat copy, the mobile phones, as well as the immaterial; i.e. the knowledge of sailing boats, of tool and boat building skills, the semiotic and social forms of the visitors use of mobile phones and social media as tools to enact their understanding of the rather complex story if the reconstruction of the Viking boat. By focusing on these enactments of the exhibition assembly, we where forced to give the activities of taking photo and activities of sharing on the blog a place in the exhibition set-up, and conceptualise these as part of the exhibition assembly. Without visitors` enactments by way of these tools, the exhibition would in fact be brought to life.

3.2 Enacting assemblies by making connections between online and physical activities This exhibition experiment was set up to accompany a workshop that the museum opened to communicate their museum practices of reconstruction project. The boatbuilding workshop at The Norwegian Maritime Museum in Oslo was established in order to open the process of the reconstruction related to one of the boats of a current archaeological finding in the centre of Oslo, and by this to communicate the importance of Norwegian boat building traditions in relation to the museum reconstruction work. Also, the workshop space was set up to function as an experimental zone, where the museum could experiment with new forms of communication before integrating these into their more permanent exhibitions and programmes. They contacted us to collaborate on designing for new forms of visitor engagement and participation related to the boat workshop. The boat builder maintains a blog of photos, videos and daily descriptions of the reconstruction process. The puzzles, open questions and hypothesises are posted there, and visitors to the blog are invited to pose questions and comments. Also, documentation material from interpretative processes of the archaeologist as well as the conservators related to the original excavated fragments is made accessible on a touch-screen. In short, on the blog visitors enactments comes as access to the reflections of the boat builder and the discussions with museum practitioners that normally stays as part of museums internal knowledge building processes. Meanwhile, the blog provided insight in the reflections of the boat builder with his community, and are valuable for onsite visitors to understand the process and stages of development stays behind the reconstruction process represented in the workshop. The need for contextualising the reconstruction process, showing the relation the different phases from archeological documentation process of the pieces of the shipwreck, to the conservation of the wooden pieces to the building of the replica all are important part of the visitors enactments of the reconstruction assemblage.


Fig 2: The boat builder in the process of making a 1:5 model in cardboard based on the excavated fragments. This model becomes then a basis for building a full scale replica. For our design work, the focus on vistors enactment of the reconstruction assembly in the workshop, made us focus on how visitors create a relation between the digital material and the physical activities of the boatbuilder in the workshop. Being aware that the museum preferred visitors to communicate personally with the boatbuilder when he was in the workshop – it was an issue to support this dialogue with digital tools – and to find solutions for communicating the reconstruction when he was not in the workshop. Observations in the workshop made it clear that most visitors needed the contextualising information to be able to understand the process; what did they build, what happened to the original pieces from the wreck and how did the boatbuilding relate to these etc. The design project focused on supporting the visitors enactments of connecting the physical activities in the workshop with the digital information proveded by a touch table on the window space into the workshop. Here the documentation on the blog, the photo series taken by archaeologist and conservators during their work followed by audio based interviews that told about the process, as well as the Facebook pages written in english was collected and made accessable for both norwegian speaking and english speaking visitors. Visitors enactments connecting the different perspective and practice related to the reconstruction where for our design work important conceptualisation of the assembly that the visitors had to connect to get an understanding of the multiple practices and activities that are involved in such a reconstruction project. This perpsective also made us aware of the difference between enactments as co-creative activities and enactments as connecting activities.

!"#$%&'%%$()" Our two design experiments have made us reflect upon the notion of enacted assemblies as a conceptual framework, and how social media introduces visitors` enactments into the exhibition. Designing for assemblages as enacted and based on visitors activities requires a sensibility to visitors’ perceived affordances of technologies and their activities of interpretation – but it might also be directed towards mediating the assembling activities of the museum professionals. Understanding exhibitions as enacted assemblies of interaction and communication lead us to rethink design sensitivity related to participatory museums in general, and participative interactions in exhibition assemblies in particular. These interactions are here regarded as enacting the assembly, and challenge the current use of the notion of assemblies in several ways: Firstly, the cases clearly showed us that the relations between visitors’ activity and the objects in the exhibitions are vital to understand. The only means for visitors to make sense of the exhibit is to


do the exhibit, i.e. to perform, act, share and reflect. In the first case, the exhibit was deliberately designed with missing relations and gaps, in order to encourage visitor actions of collecting, sharing and co-composing. It is by creating relations that the exhibit is assembled. These relations exist as complex processes in a network of material objects and immaterial knowledge, shaped by visitors’ actions, interpretations, and reflections. In the second case, the exhibit works in the relation between the past and present in the documentation, conservation and boat building process, and in the relation between the physical and the digital media in the boat building workshop, by showing interpretation of involved museum professionals and visiting laymen and interested communities and visitors. However, these assembling relations have to be constituted by visitor enactment, and the relations are only meaningful when visitors actively couple what happens on the inside of museum processes with the interactive display on the outside, the current boat building activity with the narratives from the past and the physical artefacts in the workshop and the digital mediation of the processes related to these. Secondly, mindful of the risk of taking too much control of visitors’ experiences (Hein 2000), or to design for controlled experiences that reveal a preferably unquestioned truth (Henning 2006), we designed the participation into the exhibition cases as a loosely constructed assemblage that is depending on visitors’ creative and collaborative relations in composing meaningful narratives. In this way we explore the ways how exhibitions can trigger visitors’ curiosity (Henning 2006) by relating to practices of mediated popular culture outside the museum (Witcomb 2003). This relate to a larger discussion that this conceptual framework points to; If the exhibition is experienced as an enacted assembly, the museums need to address the expectations of visitors, in terms of their wish and ability to co-create, interpret, share, and participate. Also the museums practices need to address this issue, and discuss to what extent and in what ways they will be aligning with visitors’ expectations about their enactments. All these are institutional perspectives that relate to visitors` enactments, that might go beyond the specific design project – but that might influence the enactments of the visitor as well.

*"+()&,'-$)."/012/3%" We have argued that the concept of museum assemblies needs to be re-thought and expanded to include visitor enactments. This re-thinking of museum assemblies are necessary to understand activities and participation that comes with social media into the museum. These activities build new relations and strengthen existing relations, constituting the assembly as a complex process in a network of various actants. Participation is designed incorporating use of social media and mobile phones as a means for creating relations between the content matters and visitors’ media practices, and developing perceived affordances that trigger exploration and co-composition. Understanding exhibitions as assemblages that are enacted by visitors, in interaction and communication with artefacts and content, related to both the material and the immaterial content matters, guided us to rethink the concept of assemblages in museums.

Acknowledgements We would like to thank the participants in our two design studies, and the Viking Ship Museum and the Norwegian Maritime Museum in Oslo, for providing their exhibition space for these design case studies. The work at Vikingship Museum is based on collaboration between ENCODE01 2007-2008, InterMedia and RENAME, Department of Media and Communication 2004-2008, both University of Oslo. The design case at Norwegian Maritime Museum is part of the CONTACT project, InterMedia University of Oslo 2009 – 2013, financed by The Research Council of Norway.

References Akrich, M. (1992). "The De-Scription of technical objects." In: Steven Bijker and Jon Law Shaping technology, Building society: Studies in Sociotechnical change, Cambridge Mass: MIT Press, pp. 205224 Bjö rgvinsson, E. Ehn, P., Hillgren, P-A. (2010). Participatory design and "democratizing innovation". In Proceedings of the 11th Biennial Participatory Design Conference (PDC '10). ACM, New York, NY, USA. Black, G. (2010). “Embedding civil engagement in museums”. In: Museum Management and Curatorship, 25:2, pp.129-146


Bowers, J., Bannon, L., Fraser, M., Hindmarsch, J.,Benford, S., Heath, C., Taxén, G., Ciolfi, L. (2007). “From the disappearing computer to living exhibitions: Shaping Interactivity in Museum Settings”. In: N. Streiz, A. Kameas and I. Mavrommati (eds): The Disappearing Computer, LNCS 4500, pp.30-49, Springer Verlag Berlin Heidelberg Callon, M. (1986). "Some Elements of a Sociology of Translation: Domestication of the Scallops and the Fishermen of St. Brieuc Bay." In: Sociological Review, vol. Monograph 32, edited by J. Law. London: Routledge and Kegan Paul, pp. 196-233 Ehn, P. (2007) Design av alla? [Design by all?]. In S.Ilstedt Hjelm (ed.) Under Ytan [ Under the surface] Stockholm, Raster Förlag, pp. 76-95 Fraser, M., Stanton,D., Kher Hui Ng, Benford,S., O’Malley, C., Bowers, J., Taxén,G. Ferris, K., Hindmarsh,J. (2003): Assembling History: Achieving Coherent Experiences with Diverse Technologies. In K.Kuutti, E.H. Karsten, G.Fitzpatrick, P.Dourish and K. Smidt (eds). Proceedings of the Eight European Conference on Computer-Supported Cooperative Work, 14-18 September 2003, Helsinki Finland. Kluwer Academic Publishers. Gad, C., and Bruun Jensen, C. (2009). “On the Consequences of Post-ANT.” Science, Technology and Human Values Vol 34. No 6. 2009. Hein, H., S. (2000). Museums in transition. A philosophical perspective. Smithsonian. Institution Press, Washington and London Henning, M. (2006). Museums, media and cultural theory. Open University Press, Berkshire. Hindmarsh, J., Heath,c., Vom Lehn,D., Cleverly, J (2005). “Creating Assemblies in Public Environments: Social interaction, interactive exhibits and CSCW”. Computer Supported Cooperative Work (CSCW) vol.14 (1) pp. 1- 41 Norman, D. A. (2004). Affordances and design. Essay. Consulted January 18, 2011. Latour, B. (1991). "Technology is society made durable." In: A Sociology of Monsters. Essays on power, technology and domination, redigert av J. Law Routledge, pp. 103-131 Latour, B. ( 2005). “Reassembling the social: An introduction to actor-network theory”. Oxford: Oxford University Press Law J, Hetherington K, (2000).``Materialities, spatialities, globalities'', in J Bryson, P Daniels, N Henry, J Pollard (eds): Knowledge, Space, Economy Routledge, London, pp.34-49 Law, J. and Mol, A. (Eds 2001) Complexities in Science, Technology and Medicine Duke University Press, Durham, NC Planke, Terje (2005). 'Feltarbeid i fortiden'. [Fieldwork in the past]. In: Gustavsson, Anders. (Ed.). Kulturvitenskap i felt. Metodiske og pedagogiske erfaringer [Cultural Science in the Field Methodological and Pedagogical Experiences]. Høyskoleforlaget: Kristiansand. pp.203-217. Russo, A, Watkins, J, Kelly, L and Chan, S (2008). "Participatory Communication with Social Media". Curator. 51. pp.21-31. Sanders, E. B. -N. & Stappers, P. (2008). Co-creation and the new landscapes of design. CoDesign: International Journal of CoCreation in Design and the Arts, 4(1), pp. 5-18. Sanders, E. B.-N. (2006) Design Serving People. In: Salmi, E., & Anusionwu, L. (Eds.) Cumulus Working Papers, Copenhagen, University of Art and Design, Helsinki, Finland, pp. 28-33.


Simon, N (2010). The Participatory Museum. Museum 2.0, California Storni, C. (2007) Design in Practice: On the construction of objects, PhD dissertation University of Trento, Sociology and Social Research Department. Defended: February the 14th, 2007. Supervisor: Prof. Gianni Jacucci (UNITN), Prof. Ina Wagner (Vienna University of Technology) Strathern, M. (1996). Body thoughts. Ann Arbor, University of Michigan Press Stuedahl, D (2004). Forhandlinger og overtalelser. Kunnskapsbygging på tvers av kunnskapstradiskjoner i brukermedvirkende design av ny IKT ( Negotiations and persuasions. Knowledgebuilding across knowledge traditions in participatory design of new ICT) PhD- dissertation, Faculty of education, University of Oslo Stuedahl, D. and Smørdal, O. (2011 forthcoming). “Designing for young visitors’ co-composition of doubts in cultural historical exhibitions” Stuedahl, D and Smørdal, O (submitted 2011). “Why and how to sustain young visitors` participation in and through museums”. Weibel, P., and Latour, B. (2007). ‘Experimenting with Representation: Iconoclash and Making Things Public’. In Macdonald, S. & Basu, P. (eds.), Exhibition Experiments. Oxford: Blackwell. pp.94-108. Witcomb, Andrea (2003). Re-Imagining the Museum. Beyond the Mausoleum. Routledge London and New York. Yaneva, Albena (2003). When a Bus Met a Museum. To Follow Artists, Curators and Workers in Art Installation. Museum and Society 1 (3). pp. 116-131.



Museum as a playground? Designing digitally enhanced play spaces for children's engagement with museum objects. Katarzyna Warpas University of Wolverhampton, School of Art&Design, United Kingdom [email protected] Abstract This paper is devoted to the notion of designing digitally enhanced play spaces around conventional museum objects that nurture the engagement of young visitors with artefacts through intergenerational play. In the last few years, a growing number of researchers (Cummings & Lewandowska 2000; Hagebölling 2004; Zigler et al. 2004; Seitinger 2006) have been writing about the increasing need for play in educational and non-educational settings as an essential mode of social and cognitive learning. A potential modern playground emerges in the museum space, where play, as a mode of engagement with objects, has not yet been sufficiently explored. The social and interactive character of play, particularly in group activities, creates an opportunity to engage the audience in a more active way. As far as children in the context of family are concerned, recent research (Eagle et al. 2008) has revealed an absence of design solutions that encourage shared interaction (Gobeille & Watson 2009: The Funky Forest; Harris et al. 2009) between children and adults through play. The connection between play and new technologies is appealing as it enables participation in both virtual and physical worlds and gives seemingly unlimited opportunities to foster playful engagement with and around objects (Strommen 2004, p.1). It, therefore, clearly deserves further investigation. In this paper, I examine the more traditional play spaces and their main characteristics in order to define implications for design. An overview of possibilities of supplementing these spaces with digital technologies is presented. Current museum practice is analysed as regards the incorporation of digitally enhanced play spaces into exhibitions and areas for further investigation are identified.

1. Introduction 1.1

Context of the research

The theories of museum objects (Dudley 2010, Turkle 2007, Simon 2010) show that artefacts shape and influence a visitor’s experience of museum. The common feature is the role of objects as triggers for various real, emotional and imagined actions. They can release sensory and cognitive associations (Dudley), initiate trains of thoughts and feelings (Turkle) and even go so far as to encourage us to interact with other people (Simon). How can we reveal and underline these features? What methods can be used to enable the development of a more personal experience between the object and the child visitor? Sheldon Anis in his theory of the dream space (1986) mentioned imagination, which is present in our lives from early childhood, as a vital component of a museum visit. Imaginative play allows children to rehearse ‘life situations’ in a safe, stress-free environment. It is essential as a simulation and exercise of communication, social and language skills (Ariel 2002; Carisle 2009). Additionally, which is even more important nowadays, it encourages children to develop creativity. This allows children to learn how to deal with problems and gain skills that will accompany and help them in daily life from early childhood onwards (Robinson 2009). According to Ludmilla Jordanova (2006, p.22), we have become accustomed to think of museums in terms of discovery, and there is no category of people as closely associated with the novelty and satisfaction of discovery as children. In museums, everyone becomes a child because new, precious, and important objects are the source of revelation to visitors. That at least is the idealised image. The reality is rather more complex. In reality, museums very often operate with a number of assumptions, the most important of which is the notion that the acquisition of knowledge occurs through nothing more than viewing museum artefacts (Zimmer and Jeffries 2007, p.5). In this regard, the involvement of imagination in the museum landscape becomes a crucial device to engage visitors with the subjects presented. It allows visitors of all ages not only to gain knowledge but also to make this knowledge meaningful and useful by relating it to their own lives (Strommen 2004). 151

This research seeks design solutions which encourage visitor imagination, emotions, senses and memories. As play engages all of these components, we will look closer at what play is and how it can be used to engage visitors with artefacts.


Play in museum context

For many people the terms ‘play’ and ‘museum’ may not seem particularly compatible to one another. This can be explained by the contrasting associations in common language that they evoke. ‘Play’ is often associated with fun, activity and entertainment, and ‘museum’, on the other hand, with boredom, passivity and silence. This contrast is further illustrated in two points of view that exist regarding the museum as a learning space (Hein 2002). The first of these sees the museum as a teacher, who actively presents knowledge to the visitors. The other viewpoint is to see a museum as a place to learn, like a playground, where visitors are given various options and are themselves to decide what and when to learn. Historically, museums have endeavoured to follow the first scenario: museum as a teacher. In this setting, engagement with objects is based on instruction and explicit learning. James Bradburne (2004, p.88) has noted that many hands-on exhibits in science centres and technology exhibitions resembled textbooks that explained scientific principles but did not demand any active participation. These exhibits come with an inbuilt ‘right answer’ which, “once discovered, exhausts the potential for further visitor interaction”. According to the second point of view, the museum as a place to learn, the visitor is seen as an active participant. A museum situation which facilitates open, indefinite learning enables various ways of manipulation but it does not determine what the learner will take from it. In other words, the focus lies on storytelling based on the visitor’s own experience rather than on the intention of a curator (Hein 2002). In this context J. Bradburne (2004, p. 92-93) introduced a principle, where the museum allows visitors to make choices rather than merely being presented with information. As he states, “In abstract terms this means taking the visitor’s competence and abilities seriously, and creating opportunities for the visitor to actively shape their experience in the museum”. For instance, a showcase with a collection of Peruvian ceramics and a label above ‘Moche Ceramic – 100 AD’ involves the visitor much less than the same showcase with a label ‘One of these items is a fake’, which confronts visitors with a task and automatically drives them to examine the objects more carefully. In such an environment, it is not most important how many visitors will come, but how often the presented activities will be used and repeated (Bradburne 2004). The International Council of Museums suggests that a museum should offer the public the possibility to be active and to encourage the development of the interests of the people themselves (Vieregg 2006). In order to facilitate active learning among young visitors, play seems to be one of the most powerful and appealing methods. Positive examples of the whole-child approach to learning can be already observed in the museum-related landscape, mostly in the realm of science centres. For instance, The Pattern Pod of the Science Museum in London, opened in July 2000, is a space aimed mainly at 5-8 year-olds. By various means (sounds, foam blocks, digital technology, colours, textures) children learn about the term ‘pattern’. The lack of particular goals to achieve encourages children to explore the phenomena on their own in open-ended, spontaneous play (Science Museum 2009). For example, in Kaleido-patterns (Science Museum 2009a) even the youngest participants can design their own patterns by finger painting on the touch-screen. The shapes and colours are processed by a computer and displayed in kaleidoscopic view not only on the screen but also in the wall projection. The game can be expanded by additional options of play with reflection and rotational symmetry. This is an example of good practice that can be followed by more conventional institutions. By implementing such multisensory, self-directed interactive designs, a museum becomes a playground that engages visitors in more active way and encourages them to engage in open-ended exploration through play. But the question remains: is there a place for play in a conventional museum context? There is little written on promoting play in the conventional museum context. Most sources that are available come from the fields of educational play and are written by psychologists and education specialists. The majority of design literature derives from the field of architecture and focuses on the design of outdoor playgrounds. Nevertheless, in the last few years, a growing number of researchers (Cummings & Lewandowska 2000; Hagebölling 2004; Zigler et al. 2004; Seitinger 2006) have been


writing about the increasing need for play in educational and non-educational settings as an essential mode of social and cognitive learning. A potential modern playground emerges in the museum space where play, as a mode of engagement with objects, has not yet been sufficiently explored. The social and interactive character of play, particularly in group activities, gives the opportunity to engage the audience in a more active way. As far as children in the context of family are concerned, recent research (Eagle et al. 2008) has revealed an absence of design solutions that encourage shared interaction (Gobeille & Watson 2009: The Funky Forest; Harris et al. 2009) between children and adults through play. The connection between play with new technologies is appealing as it enables participation in both virtual and physical worlds and gives seemingly unlimited opportunities to foster playful engagement with and around objects (Strommen 2004, p.1). It, therefore, clearly deserves further investigation.


Traditional playgrounds

In order to enable the discussion of a museum as a play space, a more traditional meaning of ‘playground’ and designing for play will be discussed. According to S. Seitinger (2006, p.15) playgrounds are spaces where children can engage in open-ended play patterns of their choice. Play patterns are described as ‘the sequence of choices and decisions a child makes in creating a game, challenge, or scenario’ (Seitinger 2006, p.15). For example, one pattern is to climb up a slide; another pattern is to slide down. There are three key terms that characterise playgrounds: open-endedness, physicality and sociability. ‘Open-ended play’ describes an activity that encourages children to experiment with different types of play patterns (Seitinger 2006, p.16). ‘Physicality’ means engaging children into a physically active play. This is not only beneficial to the physical health of the children but also greatly improves the experience of self-directed play. ‘Sociability’ is one of the main features of play. Play often requires a high level of interaction between players and promotes collaborative learning between them (Strommen 2004; Garvey & Rubin 1977 cited in Seitinger 2006; Rieber 1996). It is, for this reason, advisable to design playgrounds that support multiple users with all their differences and needs (Sturm 2008). Such an approach not only allows the enhancement of play pattern possibilities among children, but also aids in the development of interpersonal skills such as collaboration and competition. Marta Rojals del Alamo (2004) also advises the division of a playground space into different play areas, separating physical, group or noisy games from individual and quiet ones. Such a division uses space in a very functional way and allows children to learn about various purposes of the surrounding environment. The connection between playground design and value of play achieved has been examined by Peter Heseltine and John Holborn, who have identified measures of the quality of play as follows (Heseltine & Holborn 1987 cited in Seitinger 2006, p.35): - time: “the more time a non-repetitive play activity lasts and holds the child’s attention, the greater the play value” - change: the play value is greater when the children are free to shape their environment in their own way. The environment should be modular and adaptable to various needs. - relevance: the more relevant play is to an individual child, the better it is. - challenges: high quality play has the potential to increase the level of challenge according to the players skills and needs - suitability: high quality play can be adapted to players of different ages - co-operation: play should encourage cooperation and interaction between players. These parameters can be used in the context of both outdoor and indoor playgrounds. The focus of this paper is the issue of introducing play as a method of engagement with artefacts in the conventional museum setting. As digital media have an enormous potential to engage young visitors in a more active way (Dansk 2007: The Interactive Storytelling Exhibition Project; the V&A 2009: Web Quests), we will have a closer look at digitally enhanced play-based environments, their usage in current museum practice as well as possibilities for further research and development.


Interactive playgrounds

The concept of digitally enhanced play spaces is related to the fields of cognitive psychology, education, architecture, user experience design and human-computer interaction. There is no set definition of a digitally enhanced play space in a museum context, but there are examples, mostly from outside the conventional museum practice, which represent similar categories of digital and interactive 153

environments. The first category, which most resembles traditional playgrounds, is that of ‘interactive surfaces’ such as floors and walls. These surfaces are play areas where children can use digital features to enhance games that are already known. One example here is a design by Clara Gaggero and Sabine Feketen called dot°, which was awarded the Innovate to Educate Award by FutureLab in 2006. Its description states that dot° is an interactive playground that can be unrolled like a carpet. It uses interactive pressure sensors and lighting to illuminate games onto the surface of the playground (...). Different games can be uploaded and changed at any time (FutureLab 2006). The possibility of changing the rules of the game as well as designing new playground patterns fosters children’s creativity and playfulness. The notion of play props, typical for traditional playgrounds, is picked up in the second category of digital playground props, which are brought by children to the playground in order to enhance their play experience.

Illustration 1: Morel in use. Iguchi & Inakage 2006

One example, Morel (Illustration 1), is a soft cylindrical object, that can be used like a ball, but which can detect other Morels and connect with them (Iguchi & Inakage 2006). When another Morel is detected, the player is warned by a sound signal. The Morel behaviour does not contain rules of any particular game. This encourages players to enhance already known games or make up their own. The third category of products that is relevant to the idea of digitally enhanced play spaces are interactive playground installations. There are installations that are designed for outdoor play. For example, the SmartUs installation (Illustration 2) is outdoor play equipment that allows children to take part in a ‘live’ computer game. It consists of a three basic elements:

a central unit that controls all actions (iStation), poles with built-in sensors (iPosts), an interactive grid (iGrid), and identification labels (iTags) that interact with the iStation through sensors on the iPosts. In addition, the installation provides a software package that enables the users to create their own games (Sturm at al. 2008, p.259)


Illustration 2: iPost from the SmartUs interactive playground. Lynch 2007

As in the case of digital floors, interactive playground installations give players the possibility to enhance or adapt well-known games and make up their own ones. The forth category is that of pervasive (or location-aware or augmented reality) games. As a consequence of the swift growth of mobile communications and wireless technologies, pervasive games join the experience of a computer game with the physical environment of the player. In contrast to other categories, they are not restricted to any specific location and can be played in unconventional game venues, such as streets, conferences and museums (Strum 2008; Montola, 2010). For instance, ‘Can You See Me Now?’ is an urban chase game that happens simultaneously online and on the streets. Online players move their avatars through a virtual map of the city of Sheffield, while runners on the actual streets of the city track the avatars in a real environment, equipped with a handheld computer connected to GPS (Blast Theory Website, n.d.) (Illustration 3). This enables the players to comprehend and establish a connection between virtual and real world. All of these categories can be included or relate to the notion of an intelligent playground. This is an environment with interactive objects that, using technological elements such as sensors and actuators, reacts to the actions of children and actively encourages children to play (Sturm et al. 2008, p.258). Janienke Sturm and partners (2008) have identified five key issues regarding the design of intelligent playgrounds: social interaction, simplicity, challenge, goals and feedback. Social interaction is an essential feature of the activity of play. Playgrounds should, therefore, be designed to facilitate cooperation, competition and exchange of experiences between users. In order to enable self-directed play without adult supervision, the rule of simplicity should be applied. Children should, therefore, be able to begin the game without preparation and surroundings should be modifiable so that tasks can be changed according to children’s development. In its social, cognitive and physical aspect simplicity is closely connected to challenge. For play to be challenging, not only do different levels of development among players have to be respected, but it also has to be adjustable to the progress players undergo while playing. ‘Goals have been shown to be very important for the design of (computer) games. (...) Providing goals contributes to the appeal of games, and to the intrinsic motivation of users’ (Sturm et al. 2008, p.262). There might be predefined goals or the design of the game might allow users to set up the goals by themselves. As far as intelligent playgrounds are concerned, the second option is generally more appealing as it leaves children more space for exploration and the development of their own targets. Finally, designing for feedback maintains the balance between goal-directed motivation and fun.


Illustration 3: The probe tool in action. The Ambient Wood Project. Rogers et al. 2005

Illustration 4: A virtual map of the city used by online players to control runners in a real city tracked by satellite. Blast Theory 2001 An example of an intelligent playground is the Ambient Wood Project, conducted by the Equator Project, which took children on a digitally augmented field trip into the English woodlands. This was designed to help children connect knowledge gathered in nature and through observation, with classroom activities. Providing participants with several portable devices, the Ambient Wood Project enabled them to explore and gather information on biological processes observed in a forest (Illustration 4). The children’s positions were tracked by GPS, and the locations of their discoveries were transmitted to their handheld computers. The children could then reflect upon the outcomes of their research on the screen after coming back to the classroom by analysing and comparing data gathered, discussing and sharing them with other participants (Rogers et al. 2005). 156

The intelligent playground, although derived outside the museum context, provides insights into the designing of digitally enhanced play spaces that can also be useful in the conventional museum. This research seeks to create intelligent playground solutions that are realisable in a museum context and that enhance the visitor’s overall experience with museum objects. For this reason, we will examine museum practice and design in order to identify and analyse possible currently existing examples of such spaces.

4. Digitally-enhanced Play Spaces In the following overview of digitally enhanced play spaces in museum practice, we will reassess how play features influence and reflect in the design of play spaces. This will allow us to identify criteria of good practice that as far as the design of digitally enhanced play spaces in a conventional museum context is concerned. It will also set a sign post for a further research in the field. Play is sociable, participant-driven and process-focussed. A playground should mirror these features and in its design should encourage: ! sociable, cooperative behaviours ! self-directed, participant-driven actions ! open-ended exploration The designers of play spaces add ‘physicality’ to this list, which implies encouragement into a physically active play. Analogically, an interactive playground embodies four features of a traditional playground and enhances it through added value of ! mixed-reality that allow players to connect real and virtual worlds ! enhanced interaction and communication between visitors ! stimulation and encouragement as well as feedback A digitally enhanced play space in a conventional museum context is a further development of an interactive playground that is object-centred. According to Ludmila Jordanova (2006, p.25), artefacts can trigger people’s emotions, thoughts and interactions. In other words, they can be a starting point of sociable, process-focused and participant-driven engagements in a museum context. They can trigger play. A digitally enhanced play space in a conventional museum context combines all of the features of interactive playground with a focus on a museum object as a trigger for action. It increases the entertainment value of a museum visit, encouraging visitors to more physical and social experience. Through combining tangible sensations, virtual content and child’s imagination, a digitally enhanced play space becomes a compelling and immersive experience (Hinske &Lampe 2007). In the following section, a non-exhaustive overview of digitally enhanced play spaces that enhance visitor’s experience of museum objects will be presented and discussed. They will be analysed according to the features of a digitally enhanced play space in a conventional museum context which are: ! sociable, cooperative behaviours ! self-directed, participant-driven actions ! a museum object as a trigger for these action ! open-ended exploration ! mixed-reality that allow players to connect real and virtual worlds ! enhanced interaction and communication between visitors ! stimulation and encouragement as well as feedback The products and projects listed were selected from current museum practice in Europe and the United States on the basis of their relevance to the topic of the research.


The Media Kindergarten – indirect engagement

One of the most frequently employed uses digital media in the museum context is as interactive play installations, which aim to engage visitors with activities loosely connected with the artefacts. They often do not demand the visitor to observe museum objects. For instance, The Media Kindergarten (Mastej et al. 2008) in the modern art gallery WroArt Centre in Wroclaw in Poland is aimed at small children who want to learn about the “classical rules and terms of arts (perspective, colour, texture) as well as various aspects of media technologies used in contemporary art (image and sound processing, interaction, real time processing)” (WroArt Centre, 2008). In four different stations, children are 157

encouraged to have not only full-body engagement, but also social interaction with other users of different ages. For example, the installation ‘Textures’ consists of five boxes each containing different textures (stones, water, steam, bark, grass) and a large screen wall projection (Illustration 5). This installation explores the aspect of connection between real and virtual worlds and between child and object. It incorporates basic features of play as sociable, participant-driven behaviours and open-endedexploration. However, it does not encourage children to explore museum artefacts and enhances social interaction only in a limited way.

Illustration 5: The installation 'Textures' in use. Media Kindergarten, Wroclaw. Mastej et al. 2008


Jill Scott’s installations – engagement through an interface

The works of Jill Scott not only engage with the history of “ordinary people and their levels of collective desires and struggles” (Scott 2000) but also with fostering collaboration between visitors. The installation ‘Digital Body Automata’ (Scott 1997) was designed to “encourage intimate and contemplative participation”, which meets with the notion of imagination of a play experience. This three part digital installation invited visitors to explore how technology might change their body and what effects it might have for the future. The part called ‘A Figurative History’ comprises five interactive terminals with smart objects and corresponding screens. By touching a metal point on the objects, visitors can trigger animated figures on the screens that tell them different stories about the development of the idea of artificial intelligence. The viewer is not only connecting with the objects by touching the trigger point, but also with other viewers (Illustration 6). By meshing physicality of objects with virtual information, this object-driven installation creates an added value to a real experience in a social context.


Illustration 6: Jill Scott 1997: Figurative History. A interactive terminal with smart object in use. The Database of Virtual Art 2007

The notion of collaboration was developed in another of Jill Scott’s installations called ‘Beyond Hierarchy’, which presents insights into the personal lives of Ruhr region workers. It uses ‘secret handshake interface’, where a meeting of two virtual characters of Ruhr region is triggered by a simulation of a handshake of two visitors in real life. By touching hands of other participants, visitors can trigger another animation. These installations use tactility as a vital component of the digitally enhanced environment. Through corporal contact with a real object, it is easier for visitors to establish a more personal connection with a virtual world. This idea has been explored even further in the SHAPE project in the Hunt Museum in Limerick

3.3 Re-Tracing the Past at the Hunt Museum

Illustration 7: The Virtual Touch machine at the Hunt Museum, Limerick, Ireland. Ferris et al. 2004

‘Re-tracing the Past: exploring objects, stories, mysteries’ is a learning space created to supplement the exhibition held in the Hunt Museum, in Limerick, Ireland in 2003 (Ferris et al. 2004). The designers of 159

the exhibition aimed to meld the technology into the setting in order to ensure that the technology itself is not a distraction to the users. For that reason, they used objects such as a trunk, a picture frame and a radio as interfaces for their games. The visitors were given key cards with RFID tags which represented one mysterious object. The key card triggered interactive stations, giving the visitor various information on the object chosen. For example, a ‘Virtual Touch Machine’ (Illustration 7) allowed visitors to examine details of an artefact by zooming in and out of its virtual model. A key card placed on the ‘Interactive Desk’ gave the visitors the access to information on geographical origin of objects. This project is an example of using intuitive interfaces in order to engage visitors into an activity. The technology was purposely hidden in the commonplace object to avoid visitor’s distraction from the given task. These were animated in order to investigate and interpret information to find the scientific truth about the mysterious artefact. This environment incorporates some elements of play such as freedom of interpretation and movement around the setting as well as open-ended outcome. Even though replicas of the objects were to be touched in the Room of Opinion, all interactive activities were designed in detachment to the real artefacts. The following example shows a way of engaging a visitor with the object in a more direct way.

3.4 Electro Bacchanalia – a peep box The peep box ‘Electro Bacchanalia’ is a digital installation that aims to encourage young visitors to examine paintings in the gallery more thoroughly (Andersson et al. 2007). It consists of a 17th century style wooden box with a replica of a painting on one of its sides. There is a peep hole in the middle of the painting. When looking through it, the visitor will notice a stage-like set-up of the painting with a faun and a woman in its centre (Illustration 8). The figures start dancing to music while the visitor is looking through the hole. When they go away, the performance stops.

Illustration 8: Electro Bacchanalia. Andersson et al. 2007

When visitors look again, the figures dance to different piece of music. This exhibit explores two feature of a digitally enhanced play space: object as a trigger for action and mixed reality. It engages visitors with the artefact, through drawing their attention for a longer period of time and making them to look closely to the details of the painting as well as nurturing their imagination about the story hidden behind a depicted scene.


3.5 Ghosts of a Chance An example of engaging visitors into exhibition through play was the project ‘Ghost of a Chance’, launched in The Smithsonian American Art Museum (SAAM) in 2008. It was the first Alternate Reality Game (ARG) to be hosted by a museum (Bath 2008, p.1). An Alternate Reality Game is an immersive experience game that encourages players to interact with a fictional world using the tools of a real world (websites, email, telephone conversations, etc.) (ibid.). The plot of ‘Ghost of a Chance’ involved putting spirits that haunted museum objects to rest by solving several clues embedded into various media such as emails, websites (Facebook, Flickr and YouTube) and in-gallery meetings of participants and museum workers. As a final event, held in the SAAM, visitors could join six quests that were linked to six spirits and, by completing them, put those spirits to rest. Each quest started with a marker by an artwork and led visitors through whole museum to fulfil a specific task (Illustration 9). This forced many visitors to visit parts of the museum to which they had never been or look at objects from new perspectives (Bath 2008, p.12). Some of the participants admitted in a post-event interview (Bath 2008, p.13) that the game “gave them a sense of community with the museum workers and other participants”, made them interact with museum and the objects and experience museum in a new, exciting way. This project shows, that play can be incorporated into a museum setting, turning it into an “exciting place of wonder” (Bath 2008, p.13). This project is the closest to the typology of the digitally-enhanced play space found so far. It uses artefacts as starting points of actions taken up in reality and in virtual world. By using social media, it encourages development of cooperative behaviours between players giving them the opportunity to decide what will happen in the game as well as to communicate with one another through various means.

Illustration 9: Searching for final clues in front of Georgia Stele by Jesús Moroles. Smithsonian American Art Museum 2010

3.6 Summary: The criteria of good practice To sum up, this overview of examples of digitally enhanced play spaces in museum practice reveals that digitally enhanced spaces often lack physical contact (real or imagined) with objects as well as the 161

social aspect of a play experience, especially in the intergenerational context of a family. They concentrate mostly on content transfer or the possibilities that technology gives. In this they resemble traditional computer games, where children spend hours in front of the screen isolated and immersed in a virtual world. Due to their interactive nature, though, digitally enhanced play spaces ‘can offer immersive designs and mental challenges that constantly adapt to the players’ skills’ (Hinske &Lampe 2007), which is not achievable with traditional games. For this reason, my further research aims to design digitally enhanced play spaces around conventional museum objects that enhance engagement of young visitors with artefacts through the experience of intergenerational play.



The overview of design practice from the field of digitally enhanced play spaces shows that there is still need for research, in particular in the design of those spaces that enhance children’s personal engagement with museum objects in the context of cross-generational play. This paper delivers examples of good practice that create a starting point for further investigation. It is clear from the examples taken from museum practice that some of the features evident in the examples from outside the museum area, such as physical contact (real or imagined) with objects as well as the social aspect of a play experience, were lacking. Incorporating these two characteristics into the design of a digitally enhanced play space in a museum context will be one of the main tasks for the further research. This paper also opens questions that need to be approached: ! How shall designers employ digital technology to enhance children’s engagement with conventional museum objects? ! What are design recommendations for designing such spaces? Applying this framework brings various challenges, especially to the attitude of museums towards play-based learning and taking one step beyond “'interactive' push-button technologies” (Cummings & Lewandowska 2000, p.116) as a way of engaging visitors in a seemingly more active way. Recognising the value of play as a method of engaging audiences with museum objects pushes museum workers and exhibition designers to abandon the authoritative and directed mode of knowledge transfer. This fits to the re-opened discussion on public access to museums of the 1990s, when the need for solutions that enabled a wider audience to engage with the interpretation of the exhibited collection was expressed (McPherson 2006, p.46). In this context, the challenge for the Museum remains for it to try to 'reanimate' its objects, to encourage its public to engage with the active reconstruction of the past in the present. Perhaps this might mean dropping the reliance on 'evidence' - the information emanating from the academic heart of the Museum - and, instead, making a commitment to discussion with a broad range of interest groups as means of bringing museum artefacts into the realm of playful possibility enjoyed by things in contemporary exchange (Cummings & Lewandowska 2000, pp.181-182). In this context, ‘to reanimate objects’ does not mean to attract visitors’ attention by implementing the latest interactive technologies into the exhibition, but to use all means available to create space for visitors’ intimate interaction with museum objects. By connecting museum objects, intergenerational play and digital media in the museum context, this research will contribute to the knowledge base of exhibition design. It will provide a better understanding of children’s personal experience of museum objects and how this can be supported by the design of digitally enhanced play spaces. It will also offer design recommendations for technologically-supported play-related engagement with artefacts and produce demonstrations of practice.

References Alamo, MRd 2004, Design for fun: Playgrounds, Links International, Barcelona. Andersson, GH, Brecevic, R, Gottlieb, H & Nilsson, D 2007, Electro Bacchanalia. A peep box and interpretive tool for Old Masters’ paintings, The Interactive Institute AB, Stockholm, Sweden. Blast Theory n.d., Can You See Me from: [04 March 2010].




Bradburne, JM 2004, 'Setting the Stage for Interaction. Digital Craft and the Museums of the 21st Century.' in Interactive dramaturgies. New approaches in multimedia content and design, ed H Hagebölling, Springer, Berlin, pp. 75–97. Cummings, N & Lewandowska, M 2000, The value of things, Birkhäuser, Basel. Carlisle, RP 2009, Encyclopaedia of Play in Today's Society, Volume 1 /// Encyclopaedia of play in today's society, Sage, Los Angeles u. a. Danks, M 2007, Interactive Storytelling Exhibition. How to produce a virtual interactive story to engage visitors with the real world?, The Interactive Institute AB, Stockholm, Sweden. Available from: [11 March 2010]. Dudley, SH (ed.) 2010, Museum materialities. Objects, engagements, interpretations, Routledge, New York. Eagle, S, Manches, A, O’Malley, C, Plowman, L & Sutherland, R 2008, From research to design: Perspectives on early years and digital technologies, FutureLab, London. Ferris, K, Bannon, L, Luigina Ciolfi, PG & Tony Hall, ML 2004, 'Shaping experiences in the hunt museum: a design case study' in Across the spectrum. Designing interactive systems; DIS 2004, Cambridge, Massachusetts, 1-4 August 2004 ; publication based on the Designing Interactive Systems Conference, ed. D Benyon, Association for Computing Machinery, New York, NY, pp.2005–2210. Available from: [11 October 2010]. FutureLab 2006, Children’s interactive playground – dot° scoops Innovate to Educate award, FutureLab. Available from: [07 March 2010]. Hagebölling, H (ed.) 2004, Interactive dramaturgies. New approaches in multimedia content and design, Springer, Berlin. Available from: Harris, A, Rick J., Bonnett V., Yuill N., Fleck R., Marshall P. & Rogers Y., 'Around the Table: Are multiple-touch surfaces better than single-touch for children's collaborative interactions?' in Proceedings of CSCL 2009, Rhodes, Greece, June 8-13, 2009, pp. 335–344. Available from: [09 December 2009]. Hein, GE 2002, 'Evaluating teaching and learning in museums' in Museum, media, message, ed E Hooper-Greenhill, Routledge, London. Hinske, S & Lampe, M 2007, 'Interactive Educational Play with Augmented Toy Environments', ERCIM News 2007, pp. 38–39. Available from: [29 October 2010]. Iguchi, K & Inakage, M 2006, Morel: Remotely Launchable Outdoor Playthings. Available from: [10 March 2010]. Jordanova, L 2006, 'Objects of Knowledge: A historical perspective on museums' in The new museology, ed P Vergo, Reaktion Books, London, pp. 22–40. Lynch, G 2007, Smartus, the interactive playground - you know for kids! Available from: [02 November 2010]. Mastej, P, Sobolewska, D & Janicki, P 2008, Media Kindergarten. Interactive installation in WRO Art Center, Wroclaw, Poland.


McPherson, G 2006, 'Public memories and private tastes: The shifting definitions of museums and their visitors in the UK', Museum Management and Curatorship 2006, pp. 44–57. Available from: 0UK.pdf [22 August 2010]. Montola, M, Stenros, J & Waern, A 2010, Blog: Pervasive Games: Theory and Design. Experiences on the Boundary between Life and Play. Available from: [06 March 2010]. Rieber, L 1996, 'Seriously Considering Play. Designing interactive learning environments based on the blending of microworlds, simulations, and games.', Educational Technology Research & Development 1996, pp. 43–58. Available from: [28 June 2010]. Robinson, K 2009, The Element: How Finding Your Passion Changes Everything, Penguin, London Rogers, Y, Price, S, Stanton Fraser, D, Randell, C, Weal, M & Fitzpatrick, G 2005, 'Ubi-learning Integrates indoor and outdoor experiences.', Communications of the ACM 2005, pp. 55–59. Available from: [23 November 2009]. Science Museum 2009, Kaleido-patterns. An interactive installation. The Pattern Pod. Science Museum, London. Science Museum 2009, Pattern Pod. Available from: [12 November 2009]. Scott J 1997, Figurative History. A interactive terminal with smart object in use. The Database of Virtual Art 2007, Digital Body-Automata by Jill Scott. Available from: [02 November 2010]. Scott J 2000, Digital Body-Automata, ZKM, Karlsruhe Seitinger, S 2006, Animated Props for Responsive Playspaces, Massachusetts. Available from: [03 February 2010]. Simon, N 2010, The Participatory Museum, Museum 2.0., Santa Cruz, Calif. Available from: [25 October 2010]. Smithsonian American Art Museum 2010, Ghosts of a Chance / American Art. Available from: [02 November 2010]. Strommen, E 2004, Play?Learning?Both…or neither? presented at The Annual Meeting of American Education Research Assosiaction, April 12-16, 2004. Available from: [29 November 2009]. Sturm, J, Bekker, T, Groenendaal, B, Wesselink, R & Eggen, B 2008, 'Key issues for the successful design of an intelligent, interactive playground' in Proceedings of the 7th international conference on Interaction design and children. July 11-13, 2008 - Chicago, ed Interaction Design and Children., ACM, New York, USA, pp. 258–265. Available from: [23 November 2009]. This happened – Utrecht 2009, Emily Gobeille & Theo Watson talk about Funky Forest. Video documentation of a presentation held on 26 October 2009 at the Utrecht School of the Arts. Available from: [10 March 2010]. Turkle, S (ed.) 2007, Things We Think With, The MIT Press, Massachusetts Victoria & Albert Museum 2009, WebQuest. Available from: [12 164

March 2010]. Vieregg, HK 2006, Museumswissenschaften. Eine Einfuhrung, Wilhelm Fink Verlag, Paderborn. WRO Art Center 2008, Media Kindergarten. Available from: [20 November 2009]. Zimmer, R & Jefferies, J 2007, 'Accessing material art through technologies of mediation and immediation', Journal of Futures 05 June. Available from: ( doi:10.1016/j.futures.2007.05.007 [25 October 2010].



Digitizing the valuable – value the digitized Anne Britt Ylvisåker Art Museums of Bergen, dep. West Norway Museum of Decorative Art Email: [email protected]

Abstract A new art scene requires new responses to the challenge of museum collecting. By using digital tools, and together with the game development company Turbo Tape Games, our museum has explored a method for collecting objects that do not fit the usual physical concept of acquisition. We are testing digital representation, which we hope will be perceived as equivalent to 'the real thing' in augmented reality - this obscure field between the real and the virtual. This essay is an attempt to approach this field theoretically using ordinary museum experience as the point of departure. The intention is to explore potential uses of augmented reality rather than discussing concrete practical implementation or specific technical challenges.

Figure 1: Clare Twomey 'Monument', Zuiderzee Museum 2009



In autumn 2009, I visited the ceramic installation ‘Monument’ by the British artist Clare Twomey, in which she had stacked a huge pile of broken ceramics in the corner of the atrium at Zuiderzee Museum. The work consisted of a mixture of heritage and waste: Delft tiles from the museum collection, together with assorted breakage and defective products from 1today's industrial production. In addition, the work was site-specific, physically extensive and temporal.


‘Monument’ was supposed to be exhibited for eight months only, but the exhibition period was later prolonged.



Ephemeral art

The work really confronted me with my two more or less incompatible tasks: manager of our museum collection, and acquisition manager for contemporary craft at the same museum. On the one hand, I am expected to preserve the museum’s objects for posterity, while, on the other, I am responsible for collecting works from an art field in flux. At present, it is not obvious that an artist’s work is objectoriented. He or she could just as well make site-specific or temporal projects, be engaged in processes or performances, or go far beyond the relevant format in terms of what museums of decorative art have strategies to meet, as in the case of Twomey’s work. Many artists have also rejected solid materials such as silver, glass and ceramic materials in favour of materials of a more ephemeral nature, or they make solid materials instable, as Clare Twomey did in an earlier installation, in which she required the audience to step on ceramic tiles to evoke breakage. Such works of art simply cannot be collected in the usual manner.

Figure 2: Clare Twomey 'Consciousness/Conscience", Tate Liverpool 2004 On the contrary, the volatile contemporary art field in flux stands in contrast to the stability and permanence usually associated with museums. Core qualities of ordinary museum objects and these new kinds of art objects are incompatible. At the same time, however, these instable works can be particularly relevant contributions to a museum’s collections. How can a museum like ours approach this intriguing dilemma? Although this art field in flux is a relatively recent phenomenon for decorative art museums, it is nonetheless a well-known challenge that other art museums were forced to deal with and tried to handle in different ways throughout most of the 20th century. Now that the decorative art museums are meeting this challenge a hundred years later, this 2 coincides with the introduction of digitization as a proper strategy for a forward-looking museum. In our project, we therefore wish to unify the challenges of collecting contemporary craft with the demand from the museum community to digitalize. Based on current technological capabilities, we consider digital re-creation to be an adequate substitute for artworks that cannot be preserved in their original condition, and this forms the basis for our research. Let us make the suppositional experiment that our museum would like to incorporate the abovementioned installation ‘Monument’ in our ceramic collection as a regular acquisition, but that we want 3 to do so virtually. What qualities are essential if the digital representation is to be perceived as equally meaningful and relevant as usual acquisitions. 2 Digitizing is clear requirement for Norwegian museums. It is mentioned in countless reports and white papers, such as (2008-2009): Nasjonal strategi for digital bevaring og formidling av kulturarv, (2008-2009): Framtidas museum. Forvaltning, forskning, formidling, fornying. NDU-utvalget (2009): Nasjonalt digitalt universitetsmuseum (NDU), available on line at:, last accessed 15/04/2011 3 Since the acquisition of ‘Monument’ is only suppositional, we have not made any effort to discuss the idea with Zuiderzee Museum. The artist, Clare Twomey, has been informed, however.



The characteristics of museum objects

When a work of 4 art is incorporated in a collection, it also becomes a museum object, with its distinctive characteristics. The object is protected to ensure that it remains stable for as long as possible, it cannot be discarded without further ado, and strict requirements apply to its management, loans and other kinds of use. The museums have a long-term horizon for their collections. Seen from the outside, this can be seen as strict and limiting, but it is only one aspect of collection management. At the same time, those of us who work closely with objects on a daily basis observe that their existence can be rather fluctuating and complex. A museum object can be moved from storage to be displayed at an exhibition, it can be on loan, be a subject for conservation, or be an object of study for a researcher. In my view, a museum piece first and foremost has a potential to be used in several different contexts. In that respect, the traditional museum object will also undergo a kind of instability and volatility, but in a completely different manner from contemporary artworks in flux. Where the state of the traditional museum object is close to stable and static, the state of the new artwork can often be ephemeral and in flux. But where the use of a museum object is dynamic and constantly changing, the role of a contemporary artwork will be more fixed. An ephemeral work of art tends to have a here-and-now character. It cannot simply be moved in and out of roles, and in and out of different contexts.

Object Traditional museum objects Contemporary works of art

Time eternal here and now

Substance solid ephemeral

Use / roles dynamic fixed

Table 1: Differences between traditional museum objects and many contemporary works of art When the museum wants 'difficult' work such as ‘Monument’ to be digitally represented in its collection, these works must be converted into museum objects. Our solutions must therefore be able to: • • •

make their temporariness eternal make their ephemeral substance solid give ephemeral works the same possibilities and flexibility as regular museum objects

This indicates that it must also be possible to adapt the digital substitute to changing contexts and move it from exhibition to exhibition, to have it on loan or in storage and available to researchers. At the same time, the solution must represent the work of art in a relevant way, by: • •

providing an art experience equivalent to ‘the real thing’ being faithful to the artwork

Thus, the ambition behind this project will be twofold: to digitally represent key aspects of the physical work adapted to the needs of collection management: in other words, taking its form and substance into account, while also highlighting the more abstract impact of the work – its intangible values. In this paper, I will first indicate how this can be accomplished technically, and then try to examine the value of the outcome.


What and where is augmented reality?

What, then, can a digital substitute be? Instead of searching in the narrow field of digitalization that has 5 dominated in museums so far , I seek the answer in augmented reality, 6 together with the game development company Turbo Tape Games, which has expertise in this field. Augmented reality is an unexplored field in the museums context, but it could open up new ways of approaching acquisitions, for example by creating digital representations that are as related to time and space as physical objects, at the ambiguous border between the virtual and the real. Many people associate the term digitalization with what they experience through a computer screen, 4

These properties are partly defined in ICOM’s Code of Ethics, revised 2004; available on line at:, last accessed 15/04/2011 5 Museums have mainly created digital catalogues of their collections, accessible by web, such as:,, 6 Managing Director Fredrik Sundt Breien has been the company’s main representative in this project, bringing with him a wide portfolio of game and simulation projects intended for the education, culture and industry fields: See also the reference: Breien and Rødseth (2006).


where the surface of the screen forms a clear boundary between the real 'here' and the virtual 'there'. Typical representations are scanned images and/or digital photos. In augmented reality, this distinction is not so clear. As van Krevelen and Poelman put it in their essay ‘A Survey of Augmented Reality Technologies, Applications and Limitations’ from 2010, augmented reality supplements the real world with virtual (computer-generated) objects that appear to coexist in the same space as the real world. The virtual object will be integrated in the real world in a way felt like a unity, explains Azuma in his essay ‘A survey of Augmented Reality’ from 1997. According to him, an augmented reality system: • combines real and virtual objects in a real environment, • registers (aligns) real and virtual objects with each other, and • runs interactively, in three dimensions, and in real time Let us give an example. A physical table is located in a room, and two vases sit on the table top – one physical and one virtual. Similarly, a physical and a virtual chair are placed next to the table. By means of appropriate equipment, all these objects will be experienced on equal terms, and, under ideal circumstances, the virtual objects are apparently impossible to separate from the real ones.7 In other words, while a virtual reality invites us into a completely artificial world, augmented reality allows the user to discover both the real and the virtual reality simultaneously. The virtual objects are superimposed on or merged into the real object. We will not be aware of the difference until we try to use the virtual objects by sitting on the virtual chair or putting flowers into the virtual vase. In other words, augmented reality is not either a real or an artificial world; on the contrary, it is a combination of both. This quality makes augmented reality particularly relevant to digital collecting. Works that are collected for augmented reality can be displayed in the museum's actual surroundings, and be experienced in real time side by side with physical museum items. On the other hand, collecting for augmented reality requires that the art objects are translated from one medium into another, and from one reality into another – and into a totally different visual language. But how do we actually do this? Let us go back and again use the installation ‘Monument’ as an example.


Implementation of Augmented Reality – a possible/theoretical approach

‘Monument’ cannot be taken in at a single glance. It requires you to move around in order to get an impression of the whole installation. From a distance, the work looks like a lot of landfill, randomly tipped from above. When you get closer, the work looms high above your head in a vast pyramid, made up of numerous ceramic fragments that are so randomly placed that it looks as if the mound could tip towards you at any moment. Although the work is as stable as traditional museum objects, it looks 8 sinister and instable. The artist confirms that everything really is unfixed; nothing is glued. It is like a frozen moment that can change completely the next second – a kind of latent dynamism. It makes me realize that everything is broken, the museum objects included. Both its huge bulk and the feeling of instability are important physical qualities that we must be able to transfer to our virtual substitute, as well as the fact that everything is broken. But how do we proceed? The virtual substitute does not necessarily have to be a true copy of the physical shape of the original work. On the contrary, a physical work and its virtual substitute belong in two totally different visual languages, with their own set of characteristics and conventions. Nor is every quality of a work immediately transferrable from one reality to the other. For example, the materiality of a work is an entirely physical attribute that can only be emulated in virtual reality. On the other hand, what is lost in this type of translation has to be compensated by something the digital tool is particularly suitable for. In brief, the objects need to be carefully translated.


Digitizing the valuable

The fragments in a virtual ‘Monument’ can never emerge from a real pile of ceramic plates, but at best imitate them. At the same time, it is unrealistic in terms of resources to virtually recreate each of the many thousand ceramic shards of which the installation consists. In our experiment, we focused on what we perceived as the main characteristics of the art work: crushing, instability and quantity, and 7

At present, special equipment is required to experience augmented reality, such as different kinds of head-mounted displays (HMD) to visualise, and gloves to feel the weight and/or texture of an object. For further details, see van Krevelen and Poelman (2010). This may change soon, however. Recently, an experiment was carried out by a commercial actor on experiencing augmented reality without head-mounted display, but it has not yet been published. 8 This information was provided by e-mail by the artist Clare Twomey 07/09/2010.


selected three individual elements as our starting point: an anonymous cup 9 from the present, an old hand-painted tea caddy from the museum’s store, and a simple ceramic tile. First, we made virtual 3D models of each of them that could be turned and/or manipulated in space using the cursor. Then we looked at different ways of disintegrating the objects, and, finally, at how to create a big virtual pile of the shards.

Figure 3: The virtual cup, appearing and exploding, obeying the spectator’s instructions as they are given by using a cursor What we observed was that relevant physical characteristics of the work must be accompanied by technical solutions that respond to the characteristics in a satisfactory manner. These solutions must be able to handle physical qualities such as size, shape and colour, texture and tactility, and have functionalities that can reproduce movement and speed, among other things. Each physical attribute will require a series of technical solutions to be translated digitally. Breaking an object, for example, is first about modelling a 3D object, then animating it to showing how the parts break apart, and, finally, programming the weight and directions to enable the virtually broken shards to behave in accordance with real physical laws. Taking ‘Monument’ as an example, we simultaneously experimented with different ways of breaking apart a single object, stacking virtual objects in a heap, and how we should address the essential feeling of instability. In other words, we tried to both emulate the physical work and explore specific possibilities in augmented reality; for example, experiencing a museum piece collapsing as you pass by. We also unsuccessfully investigated whether it was possible to project the digital substitute onto the floor in order to recreate the feeling of balancing on top of a shaky pile of broken ceramics. At the same time, it is important to develop standards that are easily transferable to other art works. Based on a well-known method from constructing augmented realities, we decided that all the technical solutions should serve as modules, or 'building blocks', that can be used in every context where the corresponding attribute is present. Eventually, an entire 'toolbox' will be created, which, in turn, must be adapted to the specific work of art that is to be recreated virtually. The process is deductive, from 9

This selection was made prior to our contact with Clare Twomey, and, in retrospect, we would not have picked the tea caddy, which is far too exclusive in relation to the artist’s intentions. Nevertheless, it fulfilled our needs in the experimentation phase, when we were primarily searching for a method of virtual collecting.


the general to the specific.

Figure 4: Kathrine Køster Holst: 'Throw 1’, 2009 Based on the module sketches we generated for ‘Monument’, we also carried out a comparative test on another work of art with similar characteristics: 'Throw 1' by Kathrine Køster 10 Holst, which, in addition to individual elements and movement, also contains elements of process. The comparison revealed that processes consisting of a great number of small variations can quickly become too complex to transform into augmented reality. Similarly, it is difficult to reproduce a work of art based on interaction with the audience if the interaction is not distinctively defined. A performance with few limits and an open outcome would, for example, be extremely challenging. Stacking up too many broken fragments in the virtual version of ‘Monument’ also proved to be too much to manage for a computer of the current standard. Despite the fact that augmented reality has been an attractive topic for research and development for more than 20 years, the technology is not yet sufficiently mature for projects like this. Neither photographic techniques nor hardware offer satisfactory solutions so far, and the field is still principally a subject for experimentation, with little use of standards. It is nevertheless too early to dismiss our idea as useless. In many respects, today’s approach must be of a strictly theoretical nature. If we now return to the simple form we presented in the introduction, the experiment so far shows us that: •

Based on today's technology, we cannot produce a substitute for augmented reality that is able to capture complex works, but this will probably change over time. The field is developing rapidly, with frequent presentations of new and improved solutions. The lack of technological standards means that the preservation issues are not addressed, which was the point of departure for our project. But this challenge applies to the entire digital field. It is well known in, for example, photography and video, and is subject to a massive worldwide attention.


Based on strict rules, the artist sits in front of the turning wheel at a given distance from the wall, and turns hand-sized clay objects which she throws at the wall with her left hand with great force. The way they hit the wall will differ slightly, producing diverse forms on impact with the wall. !


With more sustainable technology, augmented reality is ideal for rendering spatial work. A representation based on augmented reality is not site-specific, but can be re-created in different locations based on simple frames. In this way, a spatial artwork transformed for augmented reality can operate in different contexts and achieve some of the dynamism and flexibility that characterizes the museum object.

Some works, such as spatial installations like ‘Monument’, may be more suited to re-creation in augmented reality than others, in which case augmented reality can be used to supplement other methods. It should primarily be regarded as an alternative to photo and video documentation, based on what is most appropriate to the work in question.


Value the digitized

Suppose that the technology was fully developed, and that our experiment had been entirely convincing: With 3D glasses, we could, for example, enter an exhibition hall, where the virtual ‘Monument’ rose above our heads as we passed a tag on the floor. Reaching another tag, a virtual object silently 'explodes’ on the plinth and its fragments fall to the floor, and, just as we were standing opposite ‘Monument’, some ceramic shards on the top of the installation began to tilt dangerously. Suppose we felt a similar uneasiness as I experienced face to face with the original art work at Zuiderzee Museum. Are museums and their visitors really prepared to replace original works of art with virtual substitutes?

Figure 5: Clare Twomey 'Monument', Zuiderzee Museum 2009


‘The real thing’

As a museum, we often argue that museum objects separate us from corresponding knowledge institutions and distinguish us from archives, libraries and pure research institutions. We have 'the real thing', a distinction we like to present as an advantage. Using virtual substitutes rather than original works can, on the one hand, be perceived as a threat to such a notion. But, equally importantly, do we want a situation in which museums become translators of artworks for augmented reality, rather than pursuing their ordinary tasks as managers of physical museum objects? And a supplementary question: Is this actually a new situation for museums, or is it simply a visualization of an already wellestablished activity, an activity we are either not aware of or do not talk about? ‘Do Museums Still Need Objects?’, the historian Steven Conn asked in the title of his book from 2009, demonstrating how museum objects in the recent past have lost ground in museums in favour of museum cafés, museum shops, mingling areas and exhibitions almost without artefacts. From this point 173

of view, a transformation from physical to virtual museum objects should not be particularly threatening; we are not really concerned. Walter Benjamin, on the other hand, presented quite a different point of view in his famous essay ‘The Work of Art in the Age of Reproduction’ from 1936 (1972/91). Aura, he said, can be perceived as a ‘here-and-now experience’ with a ‘unique appearance of distance’. Benjamin saw aura as an inherent quality of the authentic work of art, and claimed that the new reproductive mediums, such as film and photography, would involve reducing the aura of art. From this perspective, our ‘Monument’ re-created for augmented reality would not be an adequate alternative to the original displayed in Zuiderzee Museum. But where does this lead us?


How real is the real?

I neither believe in empty museums, as outlined by Conn, nor in museums as an objective site for auratic experiences evoked by unique works of art. But I do believe in museums as constructed frameworks for understanding on several levels, as does the museologist Michael Fehr in his essay: ‘Art-Museums-Utopia: Five themes on an Epistemological Construction Site’ from 2005. He understands the museum reality as different from the reality beyond the museums, and distinguishes between what exists without man-made influence, Wirklichkeit/Actuality, and different man-made realities. These realities can be fictional, futuristic and idealistic utopias, or museological reflections that are quite similar to utopias, but more focused on the past than the future. Even a conventional museum object can be a construction: an ancient work of art or an old museum object will, regardless of intention and treatment, undergo alteration. At best, this can be postponed by museums; at worst museums can speed up the embedded decomposition. During its lifetime, an object goes through three main stages, the museologist Peter van Mensch claims in his comprehensive essay: ‘Methodological museology; or towards a theory of museum practice’ from 1990. Firstly, the object exists as an idea, he says; then it will be realized and achieves what he calls its 'factual identity'. Finally, the test of time will leave traces on the object that affect how it appears to us today. What precisely it is that constitutes the authentic artwork seems to be the main issue in the whole conservation field, followed by the question of which part of the object's biography the conservator should address when the object is to be treated. The idea, the factual state as it was when brand new, or its actual state, as it appears after its lifespan? I do not want to join this discourse here, but merely wish to stress my point: to do his job, the conservator needs to consider what he perceives as the most important characteristics of an artwork or museum object before he makes any intervention. That means that he interprets, selects and omits, and, when something is emphasized, something else has to be downgraded. The conservator may have to balance between different dilemmas and possibilities, and make compromises that can prevent him from restoring the objects optimally in every sense. Do evaluations like these differ in principle from the way we attacked the challenge of creating the digital version of ‘Monument’?! In my view, these processes are closely related, but, as ordinary visitors, we do not generally engage in deep reflection on questions of this kind. What do we really encounter at a museum: the unique work of an artist, the work tampered with by conservators, or an artwork captured in a curator-made environment, more or less designed to guide the public's experience in certain directions? The work of art as it appears in a museum is already loaded with interpretations and interventions that have taken place in many steps. In his essay ‘Translating knowledge, translating cultures’ from 2009, Peter Burke confronts the idea of cultural heritage as transmitted more or less unchanged from one generation to another, from one cultural field to another. He also believes that there has been a widespread consensus in recent years that cultural reception is not passive, but active. Cultural influences are not simply adopted, but, on the contrary, they are adapted to their new cultural environment. That means that they are decontextualized and then recontextualized, or, as he puts it: domesticated or ‘localized’. In his view, they are translated, and, as in almost every translation, this is no easy task. A text can be translated word by word, but many expressions in one language lack exact equivalents in another, and the translator’s dilemma is whether to choose between and keep strictly to the original version or to translate the work in a way that is easily accessible to a local audience – and this is what Burke means by domestication: using words and phrases that feel natural in the specific language and culture where they are to be used.! In such contexts, the translator often works freely and becomes a kind of co-author. As I read Burke, he also makes me a translator simply by virtue of my position as a curator and collector of contemporary art. In my view, I do not consider this position to have essentially changed in the effort to transform ‘Monument’ from a physical to a virtual work intended for augmented reality. On the other hand, I do not feel that our experience is a kind of domestication. On the contrary, the attempt is more like a kind of ‘foreignization’, but not in Burke’s sense of the concept as keeping the translation strictly to the original from a context that is hard to access. To illustrate what I mean by ‘foreignization’, let us recall the different stages ‘Monument’ undergoes, from Clare Twomey’s idea, to 174

its factual state in Zuiderzee, and finally to our experimental virtual ‘Monument’. When Clare Twomey realized her idea, she chose ceramic shards from the domestic field; broken tableware everybody can recognize and is familiar with. Stacking them in a pile, on the other hand, decreased their familiarity for everybody except those who work with ceramics or live in a ceramics producing area. But a waste dump is a well-known phenomenon to most of us, which means the installation corresponds to common domestic experiences. Nevertheless, the installation does not appear to be an artistic statement until the pile is decontextualized and recontextualized in a museum where we do not expect to find it. The shards, domestic or not, have become ‘foreignizised’ and the museum context invokes ‘the difference beyond difference’, as the philosopher Boris Groys puts it in his essay ‘On the New’ from 2002. History, he says, is a dynamic process editing the boundaries between life and death, between what is new and what is already filed, and the museums hold the positions as editors. As I understand it, this position is unaltered even when transferred to our virtual ‘Monument’. What seems to constitute the main difference is our translation of the installation not to a familiar but to a foreign language in the museum context. That means that we have ‘foreignizised’ the ‘foreignizised’. In a way, we have acted in the same way as the artist. To do this, we did not translate the factual ‘Monument’ at Zuiderzee Museum shard by shard but instead tried to translate the artist’s idea and the installation’s effect into our selected visual language: augmented reality. We became co-authors. But how does this correspond to Benjamin's ideas about aura? In Gérard Genette’s opinion, an art experience can occur without being directly in touch with the original. An artwork consists of two identities, he says: the material work itself, and the awareness of the work’s existence, indirectly experienced through ‘everything that can provide more or less precise knowledge of a work, whenever the work itself is definitively or temporarily absent’ (Genette 1997, quotation from Daatland 2001). Every representation will emulate some but not all characteristics of an artwork, and the degree of congruence with the original will separate the copy from reproduction and documentation. A virtual object will also borrow aura from the physical objects in a mixed or augmented reality, Bolter, MacIntyre, Gandy and Schweitzer wrote in their essay ‘New Media and the Permanent Crisis of Aura’ from 2006. The research group used Benjamin's concept of aura to examine experiences related to digital media. It believes that new media do not remove aura from art, but expose aura to an ongoing crisis, in which it alternates between being challenged and emphasized: ‘The tension between far and near – between the unapproachable and the approachable both at the psychological level and the cultural and economic level – in fact ensures that aura as our collective or individual reaction to art can never simply disappear’(2006:26). Augmented reality, which consists of both virtual and real elements, will be able to create a unique here-and-now experience, the team continues. This virtual closeness can either take us unconsciously beyond our physical location or make us forget about the technology and believe in the virtual representation. But technology will also, by its existence, be able to create the distance required in order to achieve aura. That is to say that the experience of the virtual representation alternates between ignoring and emphasizing the medium; between evoking aura and questioning it.



So far, we have argued in favour of the virtual ‘Monument’ being perceived as meaningful. There may be multiple explanations for why this does not feel entirely convincing to us, and why we also sense a certain scepticism among both artists and museum colleagues. Burke explains a successful translation as a sense of perceiving the translated text as it was originally made in the target language. In that case, the translator becomes invisible, overshadowed by the original work. Since the virtual ‘Monument’ is the first time we have conducted such an experiment, without having any experience in this field, we have probably not created a credible translation, and we therefore became too visible to be ignored. But this is not necessarily the only way to put it. As a museum approaching a new area without any established conventions for digital translations made for augmented reality, we expose the audience to an unexpected experience, and they do not know how to respond. This can be changed, however. When we enter an exhibition by Christo; the artist celebrated for his ‘wrappings’ of famous buildings and sites, we expect a series of photographs visualizing these events, and not the buildings and sites themselves. Correspondingly, an educated audience will easily comprehend a video showing a performance as a piece of art. They are already familiar with these ways of reading artworks that no longer exist. But much remains to be done before the audience is equally familiar with augmented reality in museums.


References Azuma, R.T. (1997): “A Survey of Augmented Reality”. Presence 6 (4): 335-385, available on line at:, last accessed 15/04/2011 Benjamin, W. (1975/1991): Kunstverket i reproduksjonsalderen. Oslo, Gyldendal Bolter, J.D.; MacIntyre, B.; Gandy, M. and Schweitzer, P. (2006): “New Media and the Permanent Crisis of Aura”. Convergence 2006 Volume12 (1): 21-39. Breien, F. and Rødseth, I. (2006):”Usability Factors of 3D Criminal Archive in an Augmented Reality Environment”. NordiCHI '06, ACM, New York: 385-388. Burke, P. (2009): “Translating knowledge, translating cultures”. M. North (ed.): Kultureller Austausch: Bilanz und Perspektiven der Frühneuzeitforschung, Köln-Weimar-Wien: 69-77. Conn, S. (2009): Do Museums Still Need Objects? University of Pennsylvania Press Depocas, A.; Ippolito and Jones (2003): The Variable Media Approach. Permanence through change. Montreal-New York Daatland, L. (2001): Det virtuelle kunstmuseet. En studie av formidlingsteoretiske og estetiske problemstillinger. Thesis, University of Bergen, Norway Fehr, M. (2005): “Art-Museum-Utopia: Five themes on an Epistemological Construction Site”. Jörn Rüsen, Michael Fehr and Thomas W. Rieger (ed.): Thinking Utopia. New York-Oxford: 169-173. Groys, Boris (2002): “On the New”, available on line, last accessed 15/04/2011


Hall, G. (ed.) (2009): “Monument Clare Twomey”, exhibition catalogue, Zuiderzee Museum/MIMA Krevelen, D.W.F van and Poelman, R. (2010): “A Survey of Augmented Reality Technologies, Applications and Limitations”. The International Journal of Virtual Reality, Volume 9(2):1-20. Mensch, P. van (1990): “Methodological museology; or towards a theory of museum practice”. Susan M. P. (ed.): Objects of Knowledge. London: 141-157. Twomey, C. (ed.) (2009): Possibilities & Losses: transitions in clay, exhibition catalogue, Crafts Council/MIMA


Paper Title: Mobile Phones in U.S. Science Museums: A Report of an Exploratory Survey Hyunsook Youn School of Communication & Information Rutgers University 4 Huntington Street New Brunswick, NJ 08901 USA Email: [email protected]

Marian Stewart Titus School of Communication & Information Rutgers University 4 Huntington Street New Brunswick, NJ 08901 USA Email: [email protected] Abstract A survey with 94 science museums in the United States was conducted to examine how science museums in the United States have utilized advanced technologies (e.g., mobile phone interactive programs). This study explores the use of mobile phones to enrich the visitor experience in 94 science museums in the United States. Most museums were not using mobile phones, and for the minority that were, their decision to adopt the technology was primarily visitor-driven: the most popular exhibits, extra exhibit content, and visitor recommendations. These museums were not significantly influenced by peer institutions with similar programs, indicating that they were responding less to industry best practices, and more to visitors’ needs. Museums made a clear distinction between hands-on exhibits and mobile phone interactivity. Museums not using mobile technology were emphatic about maintaining the hands-on nature of their exhibits, and saw mobile phones as a distraction from these exhibits, and as a passive medium that reduced the visitor’s ability to interact with exhibits. This study revealed that mobile phone technology in U.S. science museums is still an emerging trend, but there was great interest in using this technology in the near future.

1. Introduction Mobile phone usage continues to increase in the United States and worldwide at a staggering rate since they were commercially introduced in 1983. Mobile phone penetration continues to increase in the United States, where in 2009; there were over 285 million mobile phone subscribers, which is equivalent to 91% of the total population (Foresman, 2010).. According to International Telecommunication Union (2010), there were approximately 4.6 billion mobile phone subscribers globally at the end of 2009. ITC also estimated that there would be 5.3 billon mobile phone subscriptions by the end of 2010, which is equivalent to 77% of the world population. As Kalba states, “mobile phones have out-diffused virtually every prior technology, including bicycles, radios, television (TV) sets, wallets, wireline phones, and wristwatches, and have done so in twenty-five years” (p. 1). Academia and other educational fields are now incorporating mobile devices into their education strategies to enhance students’ ability to learn in formal educational settings or for informal learning. Museums of various types have been using mobile phones to enhance the visitor experience since the late 1990s. Interactivity is a common feature of science museums in particular, and increasingly, science museums in the United States are now using mobile phones to increase visitor interactivity with exhibits. Science museums in particular, even before adopting mobile phone technology, have often encouraged visitors to interact with their exhibits, unlike art museums for instance. Science museums have tended to reach out to children much more than other types of museums, in order to encourage younger visitors to touch and interact with the exhibits.


2. Literature Review Museums, along with heritage sites and tourist trails have been at the forefront of mobile technology use (O’Hara, Kindberg, Glancy, Baptista, Sukumaran, Kahana, et al, 2007). The research has focused on the exploration of context and location-sensitive computing, whereby content on a device, such as personal digital assistants (PDAs) is triggered by the visitor’s presence in a particular place or proximity to a particular object (O’Hara et al., 2007). Museums offer vast amounts of information, but a visitor’s receptivity and time while in a museum are typically limited (Bohnert, Zuckernam, Berkovsky, Baldwin, & Sonenberg, 2008, p. 195). Mobile technology offers an avenue for museum visitors to personalize their visits by zeroing in on exhibits or features that are of most interest to them. Lonsdale, Byrne, Beale, Sharples and Baber (2004) describe an example of context aware computing where visitors’ movements in a British art museum can be monitored with an ultrasoundbased tracking system connected to an iPaq PDA. Based on how much time visitors view an artifact, they receive more detailed information about the particular artifact via the PDA. The PDA also enables visitors to communicate with each other and share opinions about the exhibits. The use of mobile devices in museums of any kind has been employed for many years, going back at least as far as the late 1950s (Grinter, Aoki, Hurst, Szymanski, Thornton, & Woodruff, 2002). With more advanced mobile devices applicable, the Tate Modern in London in 2002 pioneered an interactive, audio-visual tour of its galleries using iPaq PDAs and wireless network equipment (Proctor & Burton, 2004, p. 127). At Tate Modern, the mobile technology was used for location- specific content delivery, a creative play facility, an email facility, and a polling facility to gauge response (Proctor & Burton, 2004). Visitors responded enthusiastically to the new mobile technology by spending a longer time in the museum, and stated that their visit had been improved with the new technology. At the Fitzwilliam Museum, also in the U.K., Scruton (2005) projected that mobile phones offered the possibility of introducing layers of interpretation and access to a wide range of information that otherwise would not be acceptable or possible in the space, as large panels of text among the paintings would disturb the aesthetic integrity of these spaces (p. 174). The San Francisco Exploratorium has successfully used handheld devices since the early 2000s to capture and record the trips of its visitors (Fleck et al., 2002, as cited in Scanlon, Jones and Waycott, 2005, p. 5). Scanlon et al. (2005) concluded that mobile learning in informal settings such as museums had particular benefits to offer (p. 13), as using a PDA in an art museum appeared to enhance learning activity and expanded the type of information available to visitors (Waycott, 2004, p. 215). However, it is observed that some technical difficulties with the actual PDA, including difficulty with text messaging, and temporary breakdowns, which forced visitors to shift their focus from the activity to the device itself (p. 215). Mobile technology used in science museums demonstrated similar positive results as in the case of art museums. An evaluation of MyArtSpace in a historical museum in Britain by Sharples, Lonsdale, Meek, Rudman and Vavoula (2000) showed that the project had a positive impact on school museum visits, including greater student engagement with museum exhibits, enhanced museum appeal, longer museum visits, enhanced student motivation, and support for students of differing abilities (p. 5). The difficulties centered on equipment problems and the time-intensive nature of the mobile technology model used (p. 6), but these were outweighed by the advantages observed. Between 2005 and 2007, the Liberty Science Center in Jersey City, New Jersey introduced a pilot program using mobile phone technology aimed at middle and high school students from underserved communities. An evaluation of the program (Foutz, S., Koepfler, J., & Haley Goldman, K., 2007) revealed mixed results. Museum visitors were unaware of the program, or if they were, were unsure of what it was, or how to use it. Visitors were concerned about the cost of using their cell phones, or thought there was a fee involved. However, visitors were much more receptive to receiving text updates about the exhibits. Many visitors had also taken pictures with their mobile phones in the museum. Mobile phones are being used to support social and collaborative aspects of the visitor experiences in museums and outdoor spaces such as zoos. O’Hara, et al. (2007) described schoolchildren and parents using mobile phones while visiting the London Zoo to collect video, audio and print content of exhibits they visited. The children showed adults having difficulty how to use the mobile phones to read bar codes and capture data at animal enclosures (p. 21). The children compared and viewed content on shared phones, and engaged in playful competition as they raced to complete their content collections of their visit (O’Hara, et al., 2007, p. 40). Despite the positive impact of mobile technology on the visitor experience in science and art museums, the impact of this technology was still not fully understood (O’Hara, et al., 2007, p. 12). O’Hara et al. (2007) asserted that the existing behavioral work lacked detail about the visitor activity that happened at these places; both alone and with others, and how these activities were shaped by the 178

particular characteristics of the technologies and artifacts involved (p. 12). O’Hara et al. (2007) added that, despite the ubiquity of mobile phones, they have received little analytical attention in terms of their use in location-based visitor experiences, although these authors predicted that mobile phones were likely to become one of the dominant devices for experiencing location-based experiences in the near future (p. 13).

3. Research Objectives This study proposed three main research questions to explore the extent to which science museums in the United States were using mobile phones as interactive vehicles in their exhibits. Secondly, for science museums using mobile phone interactive programs, the study explored the motivation of these museums to use this technology, to examine how these programs were working, and to examine the impact of these programs on these institutions. For science museums not using mobile phone programs, the third research objective was to explore these museums’ perceptions of the utility of this technology, their reasons for non-usage, and their willingness to use mobile phone technology in the future

4. Research Method The complete list of 386 science museums in the United States was extracted from the 2009 official museum directory as our population. Of 386, we removed those which did not have any contact information (e.g. working telephone numbers, email or website information), and attendance or staff figures listed, classified incorrectly, or closed. Thus, for this study, 303 science museums in the United States were contacted via email or telephone if the email address was not valid or available. All respondents were first sent an introductory email or a phone call, followed by an invitation email or a phone call explaining the intent of the survey and pointing them to the survey Web site. Those who did not respond in a week or two were sent a reminder email. Data collection took place during a 4 week period in November and December 2009. An online survey program ( was used to collect the data for this study. The initial questionnaire was pre-tested with two museum experts at the Liberty Science Center and Wellesley College respectively. The survey consisted of either 18 or seven questions, depending on the institutions’ mobile phone use or non-use. Our emails to 303 museums yielded 94 responses, which indicated a 31 % response rate. The sample questions we asked were reasons of employing mobile phone interactive programs, criteria that influenced their decision making of adopting the programs, and impacts the programs brought to the organization. For the museums not using the programs, the questions like their consideration of adopting the programs, reasons of not adopting the programs and their willingness to utilize the programs were asked.

5. The Findings 5.1 Science museums and their use of mobile phones in their exhibits RQ1: To what extent do science museums in the United States were using mobile phones as interactive vehicles in their exhibits? A majority of science museums, 88 out of 94, indicated that they allow museum/center visitors to use their mobile phones in the museum/center. In terms of mobile phone use policies, 70% of total respondents report that they do not have any policies in their museum/science center. Five respondents explicitly state that they just politely ask visitors to refrain from using them sometimes, whereas two of the respondents said they do not have policies because mobile phones do not work inside their building. A minority of 22 science museums (23%) out of the 94 who responded had employed mobile interactive programs. Only two museums reported that this program was introduced before the year 2000. Seven museums introduced this program between 2000 and 2005, whereas the other 10 introduced it between 2007 and 2009. The museums did not specify any particular audience for mobile interaction programs as most identified the general public as their main target. 5.2 Museums using a mobile phone interactive program RQ2: For those using mobile phone interactive programs, what are the motives and criteria that influenced your institution to make the program available to visitors? In order to answer RQ2, we asked them to indicate how important the following five


motivations were when they developed the program. These institutions’ motives for introducing this technology were varied: to enhance the visitor experience (83%), visitors already had mobile phones so they may as well use them in the museum (67%), visitors asked for this feature (67%) and some museums were following the trend of others that had this technology (45%). However, a substantial number did not indicate any particular reasons for adopting the mobile interactive program (See table 1). The museums listed other reasons for adopting mobile interaction programs such as their cost efficiency, their potential to offer more to visitors, their relevance to visitors, and the good impression of their museums that the technology conveyed. For example: “Wanted to give impression of being on the cutting edge of technology. Wanted to offer more to visitors. Wanted the extra content to also be available as podcast from our website” (Florida) “It's much easier and cost efficient to change the information on our cell phone based audio tour as opposed to changing exhibit signage” (Texas) Also, we asked them to indicate how importantly the following criteria influenced when they include a mobile interactive program in their exhibits. For museums using mobile interactive technology, several criteria influenced their decision to introduce this innovation, including: the most visited exhibits (58%), museums having extra content to share (42%), and visitors’ recommendations (42%). Most museums (69%) were not influenced by others using this technology, although one museum did see this external peer influence as the most important criterion for their institution (See table 2). Similarly, many museums (53%) did not regard recommendations from affiliated or professional organizations as a very important influence, although one museum identified professional network recommendations as the most important criterion for them. One museum mentioned that the mobile technology came with a traveling exhibition, but most museums (68%) did not share this experience, and saw this criterion as least important. Museums cited other criteria such as mobile interaction technology being an inexpensive way to share additional information, and, due to limited space, this technology allowed museums to use these devices for audio tours to share information with visitors. For example: “We don't have a lot of space to include text about our exhibits, so we are using the cell phone based audio tour to share more information. We are also developing a tour in Spanish so that we may reach a broader audience base, as our signage is NOT currently in Spanish” (Texas) 5.3 Current status of using a mobile phone interactive program RQ3: What is the current status of using a mobile interactive program? In order to answer RQ3 and examine the impact of the mobile interactive programs on these museums/centers, we asked how the mobile interactive programs were working in museums/centers, and asked them to share their opinion on better suited exhibits for the programs. Museums using mobile interaction technology reported mixed results in actual practice (See Table 3). The most noticeable benefit was an enriched visitor experience (56%), although 39 percent of museums neither agreed nor disagreed with this impact on their museums. Twenty eight percent of museums stated that mobile interaction technology enabled them to reach their target audience, which was previously classified as the general public. Thirty nine percent of the museums disagreed that this technology had significantly increased their membership. Some museums (33%) felt that mobile technology had made a difference to their institutions, whereas a smaller number (22%) reported no real difference. Nevertheless, most museums were neutral (44%) in this regard. In addition, most museums were noncommittal about mobile technology increasing repeat visits (67%), increasing visitor feedback (67%) or bringing in more visitors (56%). There was no consensus on less interactive exhibits as being optimal. It was interesting to note that three museums reported that implementing the mobile interactive programs was not satisfactory. Of those, one museum indicated that they canceled the program and two of the museums reported that the program was not successful, while the other museum indicated that their visitors really like the feature. For example: “We have been somewhat disappointed by the low numbers of visitors who have taken advantage of the offering.” (Florida) “We canceled our mobile phone interaction program after a number of years. We thought that it was not being used enough and the program was not working well.” (Vermont) "It had not been successful for us. The costs far outweigh the benefits.” (Michigan) 180

Most museums using the technology (67%) believed that some types of exhibits were better suited to this technology than others. However, their perceptions on the types of exhibits which might be better suited to the program differed from each other. Three museums noted that less interactive exhibits with less hands-on features were better suited, whereas one other museum indicated that exhibits including interactivity, rather than passive observation, are better suited. Further, three museums cited larger galleries with more content and open-ended exhibits, and two others identified places in museums where people stood longer and that were less noisy, as well as exhibits with better lighting. Given more detailed explanation in terms of ways to enrich visitor experiences, museums also reported that the mobile interactive programs could do so by providing visitors with a deeper level of interaction with the exhibits and made it easier for visitors to get information (7 respondents). For example, “It allows guests to get more information than they could from a sign that they would have to stand and read” (Ohio) “It allows our visitors to dive deeper and listens to information as opposed to reading it.” (California) It is also illustrated that these programs provided something good for families (3 respondents), better opportunities for visitors without expanding their exhibits and incurring staffing expenses (2 respondents), and real time feedback from visitors (2 respondents). For example: “It offers great feedback from visitors and their interest patterns; it meets the needs of the visually impaired.” (Florida) 5.4 Museums not using a mobile interactive program RQ4: What are the main reasons for not adopting a mobile interactive program? In order to answer RQ4, we first examined whether they have considered using a mobile phone for visitors to interact with their exhibits. Although most museums in this study (77%) were not using mobile interaction technologies, most (63%) were receptive to varying degrees to introducing this technology at some point: within a year (13 %); within two years (9%); and had considered but not for the near future (41%). One museum stated that they had mobile interaction programs but that they had discontinued them. For example: “We offered an audio tour via cell phone for one of our exhibitions in early 2008.” (Minnesota) Furthermore, we specifically asked the reasons of not adopting a mobile interactive program in their museums/centers (See Table 4). These museums cited several systemic reasons for nonadoption: financial limitations (81%), staff constraints (65%), and technical difficulties (57%). Some museums (39%) cited insufficient knowledge of the mobile technology as a drawback. Others (32%) saw this technology as inappropriate for their institutions because they had insufficient space, or their exhibits were fragile, although a larger number (48%) did not see this latter problem as relevant to their institutions. The museums also mentioned other reasons for not using mobile interaction technology. Eight museums saw the technology as being detrimental to the hands-on interaction nature of their exhibits, and representing a distraction from the exhibits. For example: “We are a hands-on museum, not a passive or listening museum.” (Idaho) Another respondent gave a more detailed explanation: “Mobile interaction programs would not be appropriate at our center, because the idea conflicts with our pedagogy of engaging visitors in the direct manipulation of physical materials with which to investigate the concepts at exhibits. For example, there are many loose parts and variables to manipulate. The exhibits are based on materials, not informational content.” (New Mexico) Four respondents said that their museums were too small for a mobile interaction program as visitors could see everything in 30 minutes. Two others stated that this technology was not suitable for their exhibits or target audience. One respondent noted that: “I believe there are solutions for the following issues but they would need to be addressed. Since we are hands-on, holding on to a device could limit visitors’ ability to interact. They may limit face to face interaction. Many visitors in our target audience (2-14) do not have mobile devices. Could discourage experimentation/self discovery.” (Alabama) 181

The other respondent explained that mobile phones were unsuitable because: “This Science Center is very much an interactive, hands-on establishment. This does not lend itself to use of mobile phones - phones would need to be put down in order for visitors to ‘play’ with each exhibit.” (Maryland) There were some more mundane but urgent considerations as well, according to another respondent: “Mobile phones have the potential of falling into the exhibits.” (Washington) Three museums mentioned that they had no wireless reception or signals in their buildings, while two other museums were looking for a better method, and one respondent elaborated on this: “We allow guests to photograph exhibits using whatever technology they have - cameras, phones, video cameras. We have exhibits that allow the guests to interact directly, rather than through the medium of technology. Hands on vs. texting, I suppose. The idea of audio tours by podcast or cell phone has been discussed, but we have not researched or implemented it.” (North Carolina) Two museums were developing their programs and needed time to assess their progress, and one museum did not want their visitors to pay extra for mobile phone calls.

6. Discussion Most of the museums in the study (95%) do not prevent visitors from bringing their mobile phones into their museums, and most (70%) have no formal polices for cell phone use. The museums are pragmatic in their acceptance of the ubiquitous mobile phone. They expect visitors to exercise good judgment and personal restraint in using them. However, it’s also convenient for the museums to use them in their programs as visitors already have them. This study does show that mobile phone technology in U.S. science museums is still an emerging trend, as only a minority (21%) of the sample was using it. However, there is a great interest in the technology by non-users, and even plans by several museums (23%) to introduce the technology within two years. Museums using mobile technology appear to have responded to visitor preferences, rather than taking the initiative to introduce the technology. The main criteria influencing their decision to adopt the technology are all visitor-driven: the most popular exhibits, extra exhibit content, and visitor recommendations. Moreover, these museums are not significantly influenced by peer institutions with similar programs, which further indicate that they are responding less to industry best practices, and more to visitors’ needs. These museums also cite an enhanced visitor experience as the major benefit of this technology (55%), whereas it has had little impact on increasing their museum membership. This could be due to the short time span since the technology was introduced, primarily between 2007 and 2008 for most of the museums sampled. Therefore, they have not had enough time to accurately determine any impact on membership. Two unexpected findings have emerged from this research. Firstly, museums—especially those not using mobile technology—are making a clear distinction between hands-on exhibits and mobile phone interactivity. Museums not using mobile technology are emphatic about maintaining the hands-on nature of their exhibits, and believe that mobile phones would be a distraction from these exhibits. These respondents do not regard mobile phones as an interactive device per se, but rather as a passive medium that reduces the visitor’s ability to interact directly with exhibits. The second finding unearthed an unexpected ritual: all museums in the sample said that visitors use their mobile phones to take pictures of the exhibits, or family members in front of the exhibits. The museums do not discourage this practice. One respondent disclosed that “Visitors have a way to capture photos of exhibits they want to remember [using their mobile phones].” Another respondent explained that mobile phones “give families a memento of their visit, and reinforce the most important visitor motivation…doing something good for the family.”

7. Recommendations for Future Research It would be valuable to gain a better understanding of the respondents’ roles in these science museums, as they were from many different departments. Therefore, some of them may have had only cursory contact with exhibit management. A follow–up study would be useful by conducting in-depth interviews with the 12 representatives who were willing to be contacted again. As the focus of this 182

research was on science museums nationwide, conducting an on-line survey of other types of museums and mobile phone technology use would present a useful comparison with this study. Similarly, as this study focused on the use of mobile technologies in US museums, it would be valuable to expand the research by incorporating how museums in other countries have employed such technologies, and how it influences both visitor’s experiences and the institutions. For example, art museums in Japan or history museums in France may have the same or different criteria and priorities regarding visitor interaction programs. Also, its impacts on visitor’s experiences and institutions will be different from one another. Surveying these museums could indicate whether the type of museum content, the degree of interactivity desired and cultural differences affect mobile phone technology use. Variable Name To enrich the visitor experience

Category 1.0 Very Unimportant 2.0 3.0 4.0 5.0 Most important

Frequency 1 0 2 4 11

Percent (%) 5.6 0 11.1 22.2 61.1

To increase our attendance

1.0 Very Unimportant 2.0 3.0 4.0 5.0 Most important

0 0 10 8 0

0 0 55.6 44.4 0

To follow trend in science museums/centers to offer mobile phone interaction

1.0 Very Unimportant 2.0 3.0 4.0 5.0 Most important

0 1 9 7 1

0 5.6 50 38.9 5.6

Visitors already have a mobile phone so why not have them use it

1.0 Very Unimportant 2.0 3.0 4.0 5.0 Most important

1 1 4 8 4

5.6 5.6 22.2 44.4 22.2

Visitors have asked for this feature

1.0 Very Unimportant 2.0 3.0 4.0 5.0 Most important

3 3 6 6 0

16.7 16.7 33.3 33.3 0



Total Number of Respondents

Table 1. Frequency Distribution on Motives of museums to develop a mobile phone interactive program


Variable Name



Percent (%)

1.0 Very Unimportant



2.0 3.0 4.0 5.0 Most important

0 5 5 3

0 26.3 26.3 15.8

1.0 Very Unimportant



2.0 3.0 4.0 5.0 Most important

1 5 6 5

5.3 26.3 31.6 26.3

1.0 Very Unimportant



2.0 3.0 4.0 5.0 Most important

4 3 2 1

21.1 15.8 10.5 5.3

Recommendations from Affiliations/professional associations.

1.0 Very Unimportant



2.0 3.0 4.0 5.0 Most important

1 6 2 1

5.3 31.6 10.5 5.3

We have extra content to share.

1.0 Very Unimportant



2.0 3.0 4.0 5.0 Most important

0 8 4 4

0 42.1 21.1 21.1

1.0 Very Unimportant



2.0 3.0 4.0 5.0 Most important

0 4 1 1 19

0 21.1 5.3 5.3 100%

Visitors’ recommendations.

Most visited exhibits.

Other museums have similar exhibits.

The mobile capability came with the traveling exhibition.

Total Number of Respondents

Table 2.Frequency Distribution on Criteria That Influenced Museums to Include a Mobile Interactive Program in Their Exhibits


Variable Name Brought in more visitors.

Category 1.0 Strongly Disagree

Enriched visitors’ experience.

Increased membership.


3 10 2 1

16.7 55.6 11.1 5.6

1.0 Strongly Disagree



2.0 3.0 4.0 5.0 Strongly Agree

4 8 3 1

22.2 44.4 16.7 5.6

1.0 Strongly Disagree



2.0 3.0 4.0 5.0 Strongly Agree

0 7 6 4

0 38.9 33.3 22.2

1.0 Strongly Disagree



4 10 1 0

22.2 55.6 5.6 0



3 11 1 2

16.7 61.1 5.6 11.1



1 11 5 0

5.6 61.1 27.8 0



2.0 3.0 4.0 5.0 Strongly Agree Getting more feedback from visitors.

1.0 Strongly Disagree 2.0 3.0 4.0 5.0 Strongly Agree

Reached target audience.

1.0 Strongly Disagree 2.0 3.0 4.0 5.0 Strongly Agree

Increased repeat visitors

Percent (%)


2.0 3.0 4.0 5.0 Strongly Agree Made no real difference.


1.0 Strongly Disagree 2.0 3.0 4.0 5.0 Strongly Agree

1 5.6 12 66.7 2 11.1 1 5.6 Total Number of Respondents 18 100% Table 3. Frequency Distribution on Impact of Using a Mobile Phone in Museums/Centers


Variable Name Financial limitation (N=59)

Frequency 4

Percent (%) 6.8

2.0 3.0 4.0 5.0 Very Relevant

1 6 20 28

1.7 10.2 33.9 47.5

1.0 Very Irrelevant



2.0 3.0 4.0 5.0 Very Relevant

5 16 20 13

8.6 27.6 34.5 22.4

1.0 Very Irrelevant



2.0 3.0 4.0 5.0 Very Relevant

2 16 19 18

3.5 28.1 33.3 31.6

Not appropriate in our museum/center (e.g. fragile exhibits, not enough space) (N=62)

1.0 Very Irrelevant



2.0 3.0 4.0 5.0 Very Relevant

16 13 11 9

25.8 21.0 17.7 14.5

Don’t know enough about a mobile interactive program (N=57)

1.0 Very Irrelevant



2.0 3.0 4.0 5.0 Very Relevant

13 16 16 6

Technical difficulties(N=58)

Staff constraints (N=57)

Category 1.0 Very Irrelevant

22.8 28.1 28.1 10.5 Total Number of Respondents 100% Table 4. Frequency Distribution on the Reasons of Not Adopting a Mobile Interactive Program

References Boehner, K., Gay, G. and Larkin, C. (2005), “Drawing Evaluation into Design for Mobile Computing: A Case Study of the Renwick Gallery’s Hand Held Education Project”, International Journal of Digital Libraries, 5:219-230. Bohnert, F., Zuckernam, I., Berkovsky, S., Baldwin, T. and Sonenberg, L. (2008), “Using Interest and Transition Models to Predict Visitor Location in Museums”, AI Communications, 21: 195-202. Foresman, C. (2010), “Wireless survey: 91% of Americans use cell phones. Telecom and Internet Telephony”, available online at, last accessed 04/14/2011. Foutz, S., Koepfler, J. and Haley Goldman, K. (2007), “Science Now, Science Everywhere (SNSE) Formative Signage Testing”, Edgewater, MD: Institute for Learning Innovation. Unpublished manuscript. Grinter, R. E., Aoki, P. M., Hurst, A., Szymanski, M. J., Thornton, J. D. and Woodruff, A. (2002), “Revisiting the Visit: Understanding How Technology Can Shape the Museum Visit”, in Churchill, E. 186

F., McCarthy, J., Neuwirth, C. and Rodden, T. (eds), Proceedings of the 2002 ACM conference, New Orleans, November 2002. International Telecommunication Union (2010), “ITU estimates two billion people online by end 2010”, available online at, last accessed 04/14/2011. Kalba, K. (2008). The global adoption and diffusion of mobile phones. Center for I Information Policy Research, Program on Information Resources Policy, Harvard University. New Haven, CT: Kas Kalba International, Available online at, last accessed 04/02/2011. Lonsdale, P., Byrne, Beale, W., Sharples, M. and Baber (2004), “Spatial and Context Awareness for Mobile Learning in a Museum”, in KAL CSCL Workshop on Spatial Awareness and Collaboration, available online at wareness+for+mobile+learning+in+a+museum&hl=en&as_s dt=2000, last accessed 04/03/2011. O’Hara, K., Kindberg T., Glancy, M., Baptista, L., Sukumaran, B., Kahana, G. and Rowbotham, J. (2007), “Collecting and Sharing Location-Based Content on Mobile Phones in a Zoo Visitor Experience”, Computer Supported Cooperative Work, 16:11-44. Proctor, N. and Burton, J. (2004). “Tate Modern Multimedia Tour Pilots 2002-2003”, in Attewell, J. and Savill-Smith, C. (eds), Learning with mobile devices: research and development. A book of papers. London: Learning Skills Development Agency, available online at EB, last accessed 04/11/2011. Scanlon, E., Jones, A. and Waycott, J. (2005), “Mobile Technologies: Prospects for Their Use in Learning in Informal Science Settings”, Journal of Interactive Media in Education, 25:1-17. Scruton, D. (2005), “The Networked Museum”, New Review of Information Networking, 11: 171-178. Sharples, M., Lonsdale, P., Meek, J., Rudman, P. and Vavoula, G. (2000), “An Evaluation of MyArtSpace: A Mobile Learning Service for School Museum Trips”, in Norman, A. and Pearce, J. (eds), Proceedings of 6th Annual Conference on Mobile Learning, Melbourne, October 2007. Waycott, J. (2004), “The Appropriation of PDAs as Learning and Workplace Tools: An Activity Theory Perspective”, PhD thesis, Open University, Milton Keynes, UK



LodeStar: a mobile device to enhance visually impaired people experience of cultural and naturalistic places Francesco Bellotti, Riccardo Berta, Alessandro De Gloria DIBE – Department of Electronics and Biophysical Engineering University of Genoa {franz, berta, adg}

Massimiliano Margarone Smartland Project SRL [email protected] Abstract In this paper we describe a location-aware multimedia mobile guide, called LodeStar, designed to enhance visually impaired people’s fruition and enjoyment of cultural and natural heritage. A lot of research work has been done about the development of efficient Orientation and Mobility (O&M) skills. We present a novel and complementary approach aimed at supporting visually impaired persons in the construction of location awareness by providing them with location-related added value information about object-to-self and object-to-object spatial relations. We present the underlying psychological and theoretical basis, the description of the CHI design of the mobile guide and user test results coming from trials conducted with real users in two contexts of authentic use: the Galata Sea Museum and the Villa Serra naturalistic park in Genoa. Results mainly reveal that the guide can contribute to people’s ability to construct overall awareness of an unfamiliar area including geographical and cultural aspects. The users appreciate the guide descriptions of point of interests in terms of content and local and global spatial indications. Interviews have highlighted that the guide can improve the visit experience of visually impaired people.

1 Introduction Visually impaired people generally have difficulties in knowing their own position, the path for navigating unknown spaces and the relative position of objects within the environment. As a consequence they typically use other sensorial channels and exploration methods [1, 2] to build their Orientation and Mobility Skills (O&M) [3]. The awareness of the environment is essential to fully appreciate environments, such as museums, natural parks and works of art. Although usually most of the information required for this awareness is collected through the visual channel [4], visually impaired people can be able to build mental representations of objects that have never been experienced perceptually that have been described verbally. In such cases, the generation of visual images does not result from the reactivation of previously stored memories but does result from on-line construction of internal representations on the basis of the processing of verbal instructions and their encoding in a visual-spatial format. Despite the fact that such images may lack details or vividness, they have been shown to reflect properties similar to those of images based on perceptual experience [5]. In this context several studies have established that the visually impaired are able to form and manipulate mental images, using either perceptual information (verbal or haptic) or long-term knowledge. Several authors [6, 7, 8] have shown that the performance of sighted and visually impaired people in tasks requiring mental imagery was very similar. This paper presents aims, developments and user test results of a location-aware multimedia mobile guide (LodeStar, see Figure 1) aimed at supplying visually impaired people with location-related added value information on objects and spaces. The project’s assumption is that LodeStar may provide verbal instructions in a suitable way to allow people to construct mental imaginaries and location awareness. Although location awareness is closely related with vision, it processes different information: while vision collects spatial information to identify invariant features to allow object recognition, location awareness is more focused on spatial relations [9]. For this reason, the LodeStar guide supports users by identifying important object-to-self and object-to-object spatial relations.


Fig. 1 The LodeStar guide used by a visually impaired person. To investigate the research’s assumption about provision of location related added value information, we have conducted a pilot formative evaluation of LodeStar in a case study with two visually impaired persons using a fully operational LodeStar guide. The main focus of such investigation was to study if and how visually impaired could build mental representations of a space, by going through it using a mobile guide providing ad-hoc designed, location-related information. We have tested LodeStar in two ecological contexts of use [10] – the Galata Sea Museum [11] and Villa Serra Natural Park [12] in Genoa.

2 Related Works Literature suggests that the inability to travel independently and interact with wider world is one of the most significant handicaps that can be caused by visual impairment, second only to the inability to communicate by reading and writing [13]. For this reason, research on mobility of visual impaired people is focused on the development of efficient support for the acquisition of Orientation and Mobility (O&M) skills at perceptual and conceptual level. For example, other senses are used to compensate the visual deficiencies: echo feedback to estimate distances, smell to identify particular situation or people and haptic to perceive object shapes [14]. Mobile technology is used, in conjunction with GPS positioning system, to provide user with travel instructions and on-course confirmation when pointed in the proper direction [15]. In particular [16] provides a survey on visually impaired assistive technology in way-finding tasks. Our research topic is not focused on developing assistive technology to allow visually impaired people independent navigation. Instead, we concentrate our support system on a particular context of use, where visually impaired people are tourists involved in the visit of an exhibition in a museum or in a walk in a natural park and they wish to better appreciate the cultural and natural contents. More specifically, we are interested in understanding how a mobile tool can be a valid aid to the visit by giving information on Point-Of-Interests (POI) – items related to objects available in the heritage space – by contextualizing artifacts or natural objects in the environment and by providing clues and feedback to enhance the understandings of the space.

3 Design methodology The necessity for combining the multimedia potential of a mobile device with the extreme simplicity of interaction required by the use by visually impaired people involves facing several CHI issues. The tourist has little time and willingness to learn how to use the new tool, since she/ he is in the museum or in the natural park to visit the place and not to learn a tool. Most of the persons use such a tool for the first time and just for the time of the visit (typically, from 60 to 90 minutes). Thus, the complexity of the technical architecture has to be hidden to users, making the guide immediately usable, with no effort by users. This implies that the interface is to be as simple and intuitive as possible. Usability by visually impaired people strengthens the problem. The visit is a critical task for the visually impaired: the site is crowded, unfamiliar and noisy. It is very difficult to orient in such a highly


dynamic place. In this context the guide have to be not intrusive, with few and very recognizable input interface elements, possibly with tactile and audible feedback. We have tackled such issues resorting to the methodologies of the user-centric design [17], in an iterative development of the LodeStar involving participatory design [18]. Participatory design consisted in the participation at the design decisions of authors skilled in writing for visually impaired people and visually impaired end-users together with technical developers. The most significant contribution of these categories consisted in the definition of the targets and in the concrete perspective they brought into the project. Usability specifications provide explicit and measurable targets to verify the suitability of the work done. Examples of such goals are: “90% of the users have to be able to operate the guide without asking questions”, “90% of the users have to be able to use the interface with no errors”, “90% of the users have to be able to understand the meaning of all the controls within 120 seconds”. All these objectives were verified in early lab tests in order to take the appropriate corrective actions. In particular, as we describe later, we have made several interesting adjustments in the interface layout design to adhere to end-user requirements.

4 The LodeStar mobile guide We have based the LodeStar design on four main requirements: • • • •

the system is to be able to link physical-world objects with cultural contents in a just-in-time way, when the user gets close to the interesting objects [19]; the system has to be used in outdoor and indoor spaces, like museums and natural parks; the system has to rely on low-cost infrastructure, in order to have chances for a wide system diffusion; the system has to promise pervasive availability of the system in the near future.

4.1 Technology We have extended the concept of multimedia tour guides on mobile devices [20, 21, 22] to widen the usability audience also to visually impaired people. LodeStar is based on a Microsoft Windows Mobile device enhanced with a Compact Flash Radio Frequency Identification (RFID) [23], Reader to provide the device with location-aware capabilities (see Figure 2). The RF signal is in the UHF radio band (915 MHz or 868 MHz), providing long-range communication and high-speed transmission rates for reliable data exchange.

Fig. 1 The long-range RFID tag placed on a fruit tree. We placed 80 RFID tags on an area of 10.000 mqs of the exhibition at “Galata” Sea Museum and 60 RFID tags on an area of 90.000 mq at “Villa Serra” Natural Park. We have covered 140 POIs in total. See Figure 3 to see the tags distribution on the natural park map.


1. Fig. 2 Spatial distribution on Villa Serra natural park. Red spot represent the Point of Interest (POI). As software platform, we have used the MADE development environment [24] – which features a complete support for CHI design for mobile devices and RFID management [25]. The MADE RFID module implements a localization algorithm based on scanning of tags in the area. At present, it supports the iCARD Identec reader in a Compact Flash Card format [26]. This reader can be integrated in handheld, portable or laptop computers to communicate with the iQ and iD active RFID tags at a distance of up to 100 meters. The experimental tests performed have shown an overall resolution of about 5 meters, which is the minimum choose distance among tags. RFID tags used were IP65 compliant in order to resist to water and dust, and self-powered.

4.2 User Interface design The interface of LodeStar has been designed to support immediate use by visually impaired people, but also for sighted people. To achieve this goal we have adopted the design principles of overall simplicity, low intrusiveness and support for natural interaction and knowledge acquisition. Moreover, to fulfill user needs we have designed the interface with the following features: •

simple interface interaction modality;

tutorial stage;

event-driven operational mode.

The basic element of the interface is the POI multimedia card. A card corresponds to each contents or orientation information (e.g. a boat narrative in the museum, a flower species in the park), as shown in Fig. 3. Each card provides an audio contribution specifically written for visually impaired people (i.e. highlighting olfactory and tactile sensorial information and providing detailed ambient descriptions) and a descriptive image to widen the usability also for visually impaired people companions.


Fig. 3 A multimedia card example.

The interface has to enable the users to control the functions of the guide: play, pausing and stopping a multimedia description and setting the audio volume. The developed interface is the result of an interactive design phase that we have conducted in a real-context of use during the EuroFlora 2006 international exhibition [27]. We have explored several possibilities to provide such controls to the user by using embed controls in the graphic interface through the touch screen and hardware buttons with tactile feedback During the field tests, visually impaired people pointed out some important shortcomings. They reports that the touch screen was too large and their fingers were lost in a space without roughness: they had problems in locate the various controls in the interface. A second problem was to understand the topdown orientation and then remember the 2D layout of interface. Since most of users are just well experienced with the cellular phone keyboard, then we have switched to a solution using hardware buttons. Such choice of hardware navigation control allows visually impaired people to have a tactile feedback. The users felt that hardware buttons are too tiny and closer one each other and the operations to locate the correct button are very time consuming. From the lessons coming from user needs analysis, we have decided to design a hybrid interface based on the touch screen and hardware buttons: •

we have joined play/pause – which are the more frequently used commands – in a single control and linked it with the whole touch screen. With this solution the touch screen becomes a single big “hardware button”, that the user can locate easily in the device. In this way the issue of lose oneself is solved because the screen have now a unique meaning;

we have leaved on hardware buttons the others less frequently used controls. The lateral hardware button controls the volume level; the central hardware button is used to escape from the current multimedia description. With this solution hardware buttons are spatially separated (one lateral and one central) and the user can locate them in a simple way.

The initial LodeStar section is a two minute tutorial stage in which users could experiment the tool interface by pressing freely the controls. During the tutorial, a speech description briefly explains the meaning of the controls when the user presses them. Visually impaired people become familiar with touch screen and hardware buttons in order to use the guide in an independent way. This tutorial stage prevents the necessity for providing verbal explanations when users take the guide.


Fig. 4 Snapshot of the tutorial stage from Villa Serra Guide.

The event-driven operational mode provides information and orientation about POIs when users are in the proximity of them. For example (Fig. 5) in the Galata Museum, a user near to the portrait of Cristoforo Colombo receives from the guide a warning about the presence of this masterpiece. The LodeStar warning is composed of a pop-up window – to widen the guide audience also to visually impaired companions – and a corresponding recorded voice. If the user wishes listen to the POI description, she/he can press the touch screen; otherwise the guide skips the content. The event-driven operational mode allows: •

low intrusiveness: the user is asked about her/his willingness to accept the multimedia contents, but this request requires a user action only to accept the contents (by simply touching the device screen), otherwise the system automatically skips the content;

just-in-time: alerts are offered to the visitor in the exact moment of the requirement, when she/he is near to the related POI.

At the end of the description, another touch in the screen leads the user to an audio contribution to deepen the knowledge of some themes related to the current object. Information is not only about contents, as in the example of Cristoforo Colombo’s portrait, but it also concerns orientation and mobility issues as we describe in the contents design section.

Fig. 5 Snapshot of the event-driven interface. When the user is close to a POI (in the example a Cristoforo Colombo’s portrait) the LodeStar gives a warning.


4.3 Content design Preparation of contents has been particularly critical given the special requirements for the user community. We tackled the issue with a design team involving CHI designers and personnel specialized in assisting visually impaired people. To meet user requirements we have developed a multi-level structure of the contents for each card, which consists of a title, a descriptive multimedia introduction, a more extended and detailed description intended for curious users and advices to invite the user to directly experience the POIs in the surroundings. Each card description emphasizes the shape, color, smell and tactile aspects of objects, beyond more usual information. For example, LodeStar asks the user to touch the cold metal of brigantino cannons in the museum, the rough cortex of a pine in the park, and smell the gardenia flowers’ scent in the natural park. Moreover, we have involved specialist in communication with visually impaired people in order to structure the contents of the guide in a way that can be easily understood, memorized and assimilated. The main idea of LodeStar’s design is that, beyond the punctuality of the location of people in the environment, which is the main focus of current research, we argue that the location information provided to users are a key factor to allow users to construct mental imaginaries. For this reason, in addition to cards about masterpieces of the museum and plants and flowers in the park, the guide provides also cards about the relative position of POIs and about the position of service areas in the site (e.g. entrance to the exhibition, entrance to park sections, bars, cafeterias, toilettes, etc.). We have identified and tagged a set of key landmarks (e.g. crossroads in the park, elevators in the museum, and descriptions of thematic zones) within the test places and inserted them in the descriptions, sometimes in a redundant way. This kind of spatial support allows the users to be able to build and go through their own paths by constructing awareness of the place. Indeed, general users typically have two main types of experience: •

direct experience: by sensing the environment, mainly using the visual channel, people gradually builds up a knowledge of the place and routes in that environment [28];

indirect experience: a person can gain geographical information from verbal descriptions (e.g. spoken direction about the route between two places) and from written or depicted descriptions (e.g. guide books or graphical maps [29]).

It has been shown in a number of studies that visually impaired people have difficulty in constructing an accurate and flexible representation of the environment from direct experience only [30, 31]. And several authors found out that the visually impaired use less efficient strategies than people with sight in learning a map. Multimedia mobile guides have great potential in this respect. The guide improves the visually impaired’s fruition/perception of the environment by providing them with information in the exact moment in which they need it, and giving them a new way to sense the environment. In designing card contents, we have identified three types of added value information to be provided by LodeStar. •

Object information. This is a description about the meaning of the POI (e.g. how the various specimens live in the natural park) and has to be synergic with the direct experience of the user. Provision of information has to be structured in order to enhance the direct perception of the visitor, leading to a better and more pleasant comprehension of her/his surrounding environment. For example, LodeStar makes use of environmental sound (e.g. waterfall) and scent (e.g. flower smell) to connect content information and the objects in the space. An example of such content associate with a plane tree is: “The trunk of the plane tree is speckled. Irregular brown spots alternate with other grey and greenish giving an overall lighting impression to whole trunk, which stands law and cylindrical. The foliage is formed by thick branches. The jagged leafs are similar to an open hand.”

Route information. This is spatial information on the local path between the current POI and subsequent (and previous, if relevant) POIs along the planned user path. Providing this information allows the user to reach the desired destination (e.g. the cafeteria, the room with the brigantino model) through a set of local indications. An example of route information in Galata museum is: “You are at the end of the floor one. Two stairs and a ten meters long main hallway start from here. The staircase on the left leads you to the upper floors, where you find the reconstruction of the brig, the area of temporary exhibitions and the Mirador. The main corridor leads you to the barcafeteria of the museum, the Galata Cafe. The long staircase leads right to the ground floor below and to the exit. If you want to use lift, you have to follow the cafe direction.”


Configuration information. This is another type of spatial information that extends previous route information in order to allow user to progressively build a mental imaginary of the environment. At each POI, configurational information provides a description of the spatial relationship between the current user position and the overall park or museum space. A meaningful example of configuration information extracted from Villa Serra Guide is: “Here we enjoy a panorama of the park. At the end of the sinking green, two pairs of trees are placed at the sides of the lake. In the center of the lake, we see the circular gazebo built at the end of a thin peninsula, a sort of magic island placed out of sight by the soft foliage of willow and cercis siliquastrum.”

To minimize the mental overloading and support an efficient assimilation of the mental map, the configuration information is spread over all cards. In other words, knowledge is built incrementally. We have defined the concept of “influence region”: each POI is characterized by a space area in which it is meaningful to have information on that POI. For example, because it is visible from this area or because it is part of a collection whose components are also there. Different influence regions may be partially overlapped. When the user is close to a POI, LodeStar provides object and route information about that POI and configuration information related to all POIs that have an influence region that includes the current user position. In this way, the configuration information concerns not only the narrow surroundings, but also the relationships between objects and the whole site’s structure.

Fig. 6 Influence regions for POIs in a section of the Villa Serra park

5 User test We have developed a case study aiming at collecting information on the user response to LodeStar and the guide features that have higher effect on her/his location awareness performance. To this end we are interested in observing the following aspects: •

the users’ understanding of unfamiliar places;

the support that LodeStar gives to the user in her/his conception of the environment beyond the mere direct experience.

Real evaluation of mobile systems and of the impact on their intended population is difficult and costly. Evaluation requires analysis of real users, in a real context of use. In order to evaluate continuous interaction with computation resources, test-people have to use a fully operational, reliable and robust tool. Nevertheless, it is important to perform early tests in the context of authentic use in order to verify end-user acceptance and overall usefulness of the system, and to receive feedback to inform future design [10]. In this study our target is to analyze how visually impaired people form mental representations of space from a direct experience enhanced with information provided by the mobile guide. To this end, we have built an experimental framework involving:


a LodeStar guide consisting of a Windows Mobile device, equipped with a long-range RFID reader;

two real tourist settings: the Galata Sea Museum of Genoa (80 POIs), and the Villa Serra naturalistic park (60 POIs).

This experimental framework allowed us to analyze, in a user-centric perspective, several aspects of continuous interaction with the system. The methodology involves three stages: •

familiarization with LodeStar;

tourist navigation tasks in the environment;

semi-structured interviews and tests.

At the beginning of the first stage, the subjects received a short explanation about the LodeStar features and how to operate with it. The tasks at the second stage include free navigation and navigation with the guide. Finally in the third stage we have collected data on the subject’s location awareness by a semi-structured interview, graphic and sentence tests and a log mechanism built-in in the mobile guide which stores the user’s movements within the environment. Two user’ data are reported. C. is 65 years old; he became visually impaired at the age of 18. He is a beginner computer user. He lives with his wife, she is the usual companion of C in outdoor movements and she has attended the experimental trials. E. is 59 years old congenital visually impaired. He is skilled with computers with voice recognition technology. He lives alone and he is independent for outdoor mobility. He loves cultural tourism and usually he is accompanied by his friend L, which has attended the field trials. None of the participants had any residual vision which could be effectively used during the experiment. Every test user attended two sessions, one in the Galata Sea Museum and one in the Villa Serra natural park. During the sessions, users are equipped with LodeStar only in a part of their tour. A variety of techniques are used in literature by researchers to collect data on how visually impaired people construct spatial knowledge [33]. However, a multi-task and multi-analysis strategy needs to be adopted to understand the different aspects of how the visually impaired people cognitive skill is supported by the mobile guide. These techniques can be divided in two main categories. •

Route based techniques measure user’s knowledge of the relationship between two locations and how to travel from one to the other (i.e. retracing, distance, and direction). In our tests, we have asked users to estimate the distance and the direction between the start and end POIs of a route.

Configuration techniques measure aspects of configuration knowledge (i.e. how places are located in the whole environment). We have adopted basic and normal sketch mapping technique [34, 35] in order to obtain from the user a sketch map. The user is given a blank paper and she/he is asked to freely draw the museum or natural park space that she/he has just visited. Then, the researcher gives to the user some indications about the objects to draw: “draw the main route that go through the park”, “draw the most appropriate shapes of a chunk of path (i.e. straight or curved)”; “draw the main objects that are present on the path (squares, avenues, stairs, etc.)”. Beside this, we also asked users to complete a set of frames with the following typical format: “Cristoforo Colombo portrait is close to “____” and “_______ is next to gardenia flowers”. This sentence frames technique complements previous sketch technique without involving the need for skills that are unfamiliar for visually impaired people, such as graphic abilities.

User tests, as reported in Table 1, reveal that the mobile guide can contribute to improve visually impaired people’s ability not only to learn a route through an unfamiliar area, but also recall important features in the environment, like POIs. People from Villa Serra and Museo del Mare have evaluated the users answers from a semi-structured interview. We have adopted a 0-5 Likert scale on 4 dimensions: description of path shape, relative position of POIs, distance estimation and number of POIs encountered.


With multimedia guide

Without multimedia guide

Description of the path shape



Relative position of POIs



Accuracy in distance estimation



Number of POIs encountered 5 2 Table 1 Evaluation of users performance in the semi-structured interviews through a Likert scale on four dimensions. The results highlight that LodeStar has an impact on the number of POIs that users are able to recall after the visit. Users are able to describe at the same level the path shape and the distance estimation with and without the guide. Users with the mobile guide are able to recall almost all the POIs encountered during the visit (we attribute this to the provided object information), describe the position of POIs one related to others (due to route information) and give a good overall description of the POIs in the environment (due to configuration information). To explore more in detail this result, we have conducted a graphic test Fig. 7. As we would have expected, users have drawn quite accurately the general shapes of the museum and park maps, but in the part of the path with the guide, they have placed in the path also several landmarks, as shown in Figure 9. All the salient features of the natural park (i.e. the distances and relative positions of POIs and the shape of the path) are reported. To be conclusive our study need to be improved with more subject and with concrete metrics (e.g. number and distribution of landmarks on the sketches), but this first informal experiment is encouraging about the potential of LodeStar. "

Fig. 7 A section of the natural park map compared with the sketch of a subject with the mobile guide. The user have correctly identified the entrance (I), bar ( R), the lake center (X) and his initial path (V).


Reasoning on the previous preliminary results, we can argue that the physical features of the visit (distance and shape of paths) are adequately perceived by the visually impaired people’s direct experience. For example, they estimate distance with echo feedback, perceive smells that identify particular places and use the haptic sense to discern object shapes. Instead, more high level features of the visit - such as the meaning of POIs, their relation at cultural level (e.g. the portrait of Cristoforo Colombo is “related” to the caravel model not only because they are spatially closed) and the overall exhibition meaning – are not perceived if additional information is not provided. We have discovered that – although the performances in recalling the path are similar in both users groups – users equipped with LodeStar perceive better directional indications. We argue that this may be explained by the fact that users are preventively informed by the guide through route information. Users without the guide perceive a continuous, sequential path progress and cannot be aware of POIs that are not close to the visit path (thus do not trigger local - Object - information), but that would be seen by not-visually impaired people, and that are important in order to get a good idea of the overall environment. For example, the landscape with the lake that one can appreciate from the villa inside the park is impossible to figure out without the guide's configurational information. Two major content design indications have emerged during the interviews: •

some information (e.g. geographic indications in some POIs) is more valuable than other, in particular for an understanding of the overall area. It seems thus important that a priority mechanisms (also pre-emptive) should be implemented in order to allow users not to miss such important descriptions;

geographic indications are more effective at the beginning, since they provide the overall context framework (a map) into which specific information can be provided (inserted in the map).

Given the very limited number of interviewees, we are now further investigating these points in more detail in more targeted tests with a greater number of users.

6 Conclusions and future work We have introduced a novel approach to assistive technology for the visually impaired that extends current paradigm on orientation and mobility skills by supporting users in the construction of global area awareness. The approach is based on a mobile guide enhanced with location-aware capability through RFID technology. The guide provides a simple interface interaction modality, a tutorial stage to introduce users to the new tools and an event-driven operational mode. A key feature of the guide is the content development, which involves the concept of multimedia card as a description of POIs. The card contents concern three information types: object description, route information and configuration information. In order to test this novel approach we have developed two real guides – a museum guide and a natural park guide – and we have defined a experiment methodology involving semi-structured interviews and graphics test. Finally we have performed a field tests with two visually impaired persons. Results reveal an encouraging impact on the performances of users in recalling the POIs that they have encountered during the visit. The guides developed and the evaluation results reported in this paper are part of an ongoing research project. Following this pilot evaluation, an empirical study involving thirty subjects who are asked to navigate through the two target environments is currently being conducted. Our long term goal is to consolidate the body of knowledge required for the development and implementation of this mobile guide, which assist visually impaired people in constructing overall awareness of all the aspects of an area that are generally considered as necessary to fully appreciate a cultural and naturalistic experience in the site.


References Abowd G.D., Mynatt E.D. (2000), “Charting Past, Present, and Future Research in Ubiquitous Computing”, ACM Transaction in Computer-Human Interaction. Vol.7, No. 1, pp. 29-58 Beck A. (1993), “User participation in system design: Results of a field study”, Human-Computer Interaction: Applications and Case Studies, M. J. Smith and G. Salvendy, pp. 534-539, Eds. Elsevier, Amsterdam Bellotti F., Berta R., De Gloria A., Margarone M. (2002), “User Testing a Hypermedia Tour Guide”, IEEE Pervasive Computing. Vol.1, No. 2, pp.33-41 Bellotti F., Berta R., De Gloria A., Margarone M. (2003), “MADE: developing edutainment applications on mobile computers”, Computers and Graphics, Vol. 27, No. 4. pp. 617-634 Bellotti F., Berta R., De Gloria A., Margarone M. (2006), “Guiding blind people in the exhibition”, Virtuality Conference, Mobile Guide Workshop. Bellotti F., Berta R., De Gloria A., Margarone M. (2008), “Widely usable user interfaces on Mobile Devices with RFID”, Handbook of Research on User Interface Design and Evaluation for Mobile Technology, Idea Group Inc. Blades M. (1993), “WayWnding theory and research: The need for a new approach”. Cognitive and linguistic aspects of geographic space, pp. 137–165, Dordrecht, The Netherlands: Kluwer Burroughs W., Sadalla E. (1979), “Asymmetries in distance cognition”, Geographical Analysis, Vol. 11, No. 4, pp. 14-42 Carroll J.M. (1997), “Human-computer interaction: psychology as a science of design”, International Journal of Human-Computer Studies. Vol. 46, No. 4, pp. 501-522 Galata Sea Museum web site, Golledge R. (1993), “Geography and the disabled: a survey with special reference to the vision impaired and blind populations”, Transactions of the Institute of British Geographers, Vol. 18, pp.6385 IDENTEC Solution Ltd web site, Jacobson R.D. (1992), “Spatial cognition through tactile mapping”, Swansea Geographer, Vol. 29, pp. 79-88 Jacobson W. H. (1993), “The art and science of teaching orientation and mobility to persons with visual impairments”, NY: American Foundation for the Blind, New York Jacobson R. D., Kitchin R. M. (1995), “Assessing the configurational knowledge of people with visual impairments or blindness”, Swansea Geographer. Vol. 32, pp. 14-24 Kerr N.H. (1983), “The role of vision in visual imagery experiments: Evidence from the congenitally blind”, Journal of Experimental Psychology: General, Vol. 112, pp. 265-277 Kuflik T., Albertini A., Busetta P., Rocchi C., Stock O., Zancanaro M. (2006), “An Agent-Based Architecture for Museum Visitors' Guide Systems”, Proceedings of ENTER 2006 13th International Conference on Information Technology and Travel and Tourism. Lahav O., Mioduser D. (2002), “Multi-sensory virtual environment for supporting blind persons' acquisition of spatial cognitive mapping, orientation, and mobility skills”, Proceedings of the Fourth ICDVRAT. Reading, UK: University of Reading, pp. 213-222, 2002


Lancioni G.E., O'Reilly M. F., Singh N.N., Sigafoos J., Oliva D., Bracalente S., Montiron G. (2007), “Orientation systems to support indoor travel by persons with multiple disabilities: Technical aspects and applicability issues”, Technology and Disability. Vol. 19, No. 1 Loomis J. M., Golledge R. G., Klatzky R. L., Marston, J. R. (2007), “Assisting wayfinding in blind travelers”, Applied Spatial Cognition: From Research to Cognitive Technology, G. Allen, Ed. Lawrence Erlbaum Assoc. Mahwah NJ Mann W. C. (2004), “The aging population and its needs”, IEEE Pervasive Computing, April-June, Vol.3 No.2. p. 12-14. Marmor G.S., Zaback L.A. (1976), “Mental rotation by the blind: Does mental rotation depends on visual imagery?”, Journal of Experimental Psychology: Human Perception and Performance. Vol. 2, pp. 515-521 Marr D. (1982), Vision. San Francisco, CA: Freeman Marston J. R., Loomis J. M., Klatzky R. L., Golledge, R. G., and Smith, E. L. (2006), “Evaluation of spatial displays for navigation without sight”, ACM Trans. Appl. Percept. Vol.3, No.2, pp. 110-124 Mellet E., Tzourio N., Crivello F., Joliot M., Denis M., Mazoyer B. (1996), “Functional Anatomy of Spatial Mental Imagery Generated from Verbal Instructions”, The journal of neuroscience, Vol. 16, No 20, pp. 6504-6512, October 15 Mioduser D., Lahav O. (2004), “Anticipatory cognitive mapping of unknown spaces by people who are blind using a virtual learning environment”, Proceedings of the 6th international Conference on Learning Sciences. pp. 334-341 Quaadgras A. (2005), “Who joins the Platform? The Case of the RFID Business Ecosystem”, 38th Hawaii International Conference on Systems Science. IEEE Computing Society Press, pp. 855-864 Rosa A. (1993), “Ocha’ta, E. Psicolog’a de la Ceguera”, In Madrid: Alianza Psicolog’a Saarinen T.F., MacCabe C.L., Morehouse, B. (1988), “Sketch Maps of the World as Surrogates for World Geographic Knowledge”, Discussion Paper 83-3, Department of Geography and Regional Development, University of Arizona, Tucson. Sherry H., Fait H. (2005), “RFID enhances visitors’ museum experience at the Exploratorium”, Communication of the ACM. Siegel A. W., White S. H. (1975), “The development of spatial representations of large-scale environments”, Advances in child development and behavior, Vol. 10, pp. 9–55. New York: Academic Press Spencer C., Blades M., Morsley, K. (1989), The Child in the Physical Environment: the development of spatial knowledge and cognition. Chichester: Wiley. Villa Serra natural park web site, Want R. (2004), “Enabling Ubiquitous sensing with RFID”, IEEE Computer, pp. 84-86 Zimler J., Keenan J.M. (1983), “Imagery in the congenitally blind: How visual are visual are visual images?”, Journal of Experimental Psychology: Learning, Memory and Cognition. Vol. 9, pp. 269-282



Prototype places: curating research in a museum context


Deborah Turnbull New Media Curation, Sydney, Australia

[email protected] Matthew Connell Powerhouse Museum, Sydney, Australia

[email protected] Ernest Edmonds Creativity and Cognition Studios, School of Software, FEIT, University of Technology, Sydney, Australia

[email protected] Abstract This paper examines the role of prototyping in a living laboratory via the medium of public exhibition. The 'living laboratory' exhibition space reported on is a small theatrette located in the Cyberworlds exhibition of the Powerhouse Museum, Sydney, and is called Beta_Space. It is the public research facility of the Creativity and Cognition Studios at the University of Technology, Sydney. The two key mandates of this space are that the artworks displayed must be interactive, thus necessitating a sophisticated digital component, and that the works must be evaluated in a public context, allowing for audience feedback on the iterative lifecycle of the prototype on display. A series of case studies performed in Beta_Space are presented which illustrate the challenges that university-museum collaborations face when embarking on such an endeavor, and show how their informal processes were able to subvert traditional museum practice, thereby creating a climate for change and growth in a normally static environment.

1 Introduction "To maintain their relevance to a new generation of potential customers, corporations and institutions are forced to seek out the virtual habitat in which these consumers reside. In turn, exhibitions have become more interactive and increasingly have a dialogue with this virtual world." (Hughes, 2010:18) There is a growing expectation amongst museum visitors that they become active participants in the museum experience rather than passive recipients of curated presentations expressed primarily through artefacts in glass showcases. The Powerhouse Museum in Sydney answered this expectation by consulting experts across all relevant fields, with the focus on what the challenges of such a shift might entail, operating in such an environment. In 2008, at the behest of new director Dawn Casey, the Powerhouse Museum held a series of Future Forums. A range of key stakeholders from industry, academia, media, government, other museums, and the Powerhouse Trust and staff, were invited to these forums to discuss possible new directions for the Museum. One of the motivations for undertaking this process was the recognition that digital technologies have changed the capacities and expectations of the museum audience. Instant information access, new understandings of knowledge, authenticity and interaction are calling into question the role of the museum and the meaning of curatorship. As the issues and problems were discussed along with proposals for new ways of being, it became apparent that there was a project that had been underway in the museum for the previous 4 years that was providing some insight into these concerns and likely strategies for the way forward. The Beta_space project had been established in 2004 with the Creativity and Cognition Studios at the University of Technology, Sydney. This project operated on the premise that interactive art research became richer, more varied and more useful when the audience participated within a real public context, sometimes more so when they didn't know much about the field they were evaluating. Therefore, this small space, established to prototype new media and interactive art works, was also an 203

exercise in establishing a new type of program; one that explored information technologies and became a model for partnership within the museum (Muller and Edmonds, 2006).

2 The Living Laboratory The notion that a museum could act as a site for artists, curators, and the audience to collaborate was a revolutionary one, made by Alfred Barr of the Museum of Modern Art. He stated his case quite simply: “The museum of modern art is a laboratory; in its experiments the public is invited to participate.” (Barr, 1939) It is this idea that established the core mandate of the Beta_Space project, that an exhibition space could be active, that both exhibition and evaluation could take place here, and that its visitors could be active participants, rather than passive observers (Muller, Edmonds and Connell, 2006). The process that goes into curating such a space is a dynamic one. It involves many elements similar to the traditional curatorial practice of showcasing an object: collaboration, selection, presentation, explanation, discussion, marketing, and evaluation. Where it becomes dynamic, is when audiences are not only allowed to touch the objects in these showcases, but are encouraged to, and are afterward expected to provide feedback to assist the research happening in the same space. This action, this participation becomes the median by which the work Figure 1. Beta_Space, 2007. Image ! Julien Phalip, is measured. Each action that leads up to showcasing the art system comes back to how the audience might experience and evaluate the work. As such, the curator of such a space becomes a producer of experience, rather than a carer of objects (Ibid). The role of this producer of experience is dependent on relationships; with other curators interested in the space, with the artists, with the institution, with the work, and with the research. Each exhibition in Beta_Space is expected to yield results in terms of practice-based research; research that informs the art making process and possibly contributes to learned articles or further engages the museum, new media, and interactive arts community. The evaluation that takes place will inform a more robust prototype, suitable for final exhibition. When programming the space, we follow a similar series of processes, informal enough to modify should the situation require it. These processes are reliant on a core relationship of an institutional curator and an independent curator, who worked together to realise the exhibitions within the space in keeping with the mandate of research and evaluation.11

2.1 The Institutional Perspective This section is written from the Powerhouse curator's perspective and in his voice. In the late 80s, when I was appointed Curator of Computation and Mathematics, I came across an article by Neil Postman. Though I disagree with some of his statements, his definition of an ideal museum struck me; he defined them as: ""!

#$%!&!'())!*+,-$%.!$'!/0-&1,2&30!2)0&,0!,00!,03-+$4,!565!&47!568!$'!9())0%:!0-!&)6!;<+=+4>!<&?$%&-$%+0,!'$%[email protected]%&3-+=0! A%-;6!!"#$%&'()*+(,$-(.,&"(./*0"1-(./*"2*!"!-$.,&"(*&(*#$%&'(*.(3*,4$*5-,%6!5BCD:!226!"EFG5HI6!


…thermostats of culture [that] maintain a dynamic involvement with [their] symbolic environment. The most vital function of a museum is to balance, to regulate what we might call the symbolic ecology of cultures, by putting forth alternate views and thus keeping choice and critical dialogue alive. (Postman 1994:68) It is this kind of philosophy that, having inherited the objects I was now to care for, I began ruminating on. Because my collection largely consisted of calculators and computing devices, I became aware quite early that there was more to computers than the beige or grey boxes housing the electronics. Where I saw the historical significance evident in the design and materials of these casings and their insides, I was mindful that the essence or the stuff of computers was something else. Software: programs, data, operating systems, perhaps better reflect what computers are; and are only fully attained once a human interacts with the machine. I began to consider different ways that I could redevelop the galleries that housed these objects, whilst better tapping into the symbolic environment of the world of computers. The way I approached this, was through collaboration with university and other research groups, with became my earliest attempts into employing practice-based research methodologies for curatorial purposes. In working with these groups, I became aware of several challenges repeatedly faced in trying to represent not only the materials of digital culture, but also the current, very connected digital world that these objects existed within. In the late ‘80s and early ‘90s, the museum was renowned for its interactive exhibitions comprised mostly of button-pressing. Interactions were part of the interpretation of the objects in all of the galleries of the new museum, not just the science and technology galleries. The advent of the Internet and subsequent rapid evolution of digital media, social media, and the devices they inspired has meant that visitors come to the museum tooled up and seeking a different kind of interaction and educational engagement other than that for which the museum was famous. I found, primarily, that visitors sought a less distant and more connected experience with the collection. What was also made clear to me was that the museum was becoming a forum for people’s ideas and opinions to be heard. Therefore, over the next decade, some in the museum started questioning what interaction meant in light of the knowledge that the average visitor’s expectation and literacy had shifted. When Ernest Edmonds of Creativity and Cognition Studios (CCS) approached me with an idea for an active laboratory inside the museum, I recognised immediately that there was something to his approach of incorporating the audience into not only the exhibition and evaluation process, but also considering them in terms of curatorial process, in the selection of certain exhibitions that would support the overarching message of the museum's programs. Though I had my reservations, which are outlined in this paper as challenges the museum faced, I was intrigued, and began making preparations to modify the space to mirror the private lab at the CCS Studios. From this mirrored state of private and public laboratory came a series of informal processes, which when followed, formalised the curatorial process that went into realising an exhibition in Beta_Space. Some took place in the private lab, and some in the public lab, the divisions of which will be made clear in the next sections. It is important to note, that once the work that took place in the private laboratory had concluded, or sometimes simultaneously, or sometimes in advance of, the work within the museum to support the exhibition would then take place. Liaising with departments such as Education, proved successful, as did informing Marketing and Operations of our plans, so that they might better, more easily, and more fluidly support the exhibitions. In the event that a work needed a little polish or presentation, we were able to engage the Workshop staff, provided they weren't busy with a major exhibition, or that we hadn't hit them with anything “big” for the previous exhibition. In an inclusive environment such as a museum, managing an external collaboration is usually about timing.

2.2 The Independent Curator Perspective This section is written from the independent curator's perspective and in her voice. One of the first things that Connell said to me when I started curating Beta_Space was that we have been very luck with staff. By this he meant that the curators and technicians that supported Beta_Space 205

were unique in their dedication to the space, particularly in terms of longevity. I knew this from watching the first independent curator of the space, Lizzie Muller, and her technical collaborators, Greg Turner and Alastair Weakley. Connell also said that this converted theatrette was a challenging space, almost inflexible in terms of usability, and that the physical support from within the museum was based on relationships comprised mostly of handshakes and favours. As an emerging curator, even these modest arrangements were appealing to me, as they came with a small budget, an available and educated technician, an evaluation specialist and a research methodology to test. From what I could deduce in speaking with Muller and Connell, and watching them work, there were a series of critical tasks. It is important to note that many of these tasks overlapped, as the medium of exhibition is rarely linear: 1.

I would review the proposed works at CCS and discuss with Connell which ones might be robust enough for public exhibition in the next 6-12 months. He would then consult the internal calendar at the museum for possible crossovers in subject matter or technical inquiry. Student works were given priority due to their need for evaluative data to either publish or augment their academic study. 2. We also involved other academic practitioners and independent spaces such as DorkBot, Firstdraft, or Serial Space. This allowed independent practitioners to evaluate their work in a public context, adding a dimension to their work that they may not have previously considered valuable. 3. Once the program was set, each potential artist would meet Connell at the CCS studio, the private laboratory model for Beta_Space, and present their prototype. 4. Because the artist could test and re-test their prototype in the studio to near completion, installing the work was usually simple. This was the time that press and promotion were also planned. 5. An ethics application to the CCS group allowed the artist and curator to map out their exhibition plan and evaluation, including focused questions and methodologies. Once approved the artist and curator would set about planning the public face of the exhibition, namely the installation, the launch, and the public programme (Bilda, Turnbull 2006:Appendix 1).

The key to the public events was that they were informal; they were low-key and inward focused. It wasn't until some of the disparate departments in the museum noticed we were getting quite a bit accomplished with very little funding that they started to pay attention to our process. Staff from the departments of Design and Education started attending our little do's, as did staff from the Director's office. They were intrigued and interested in what we were doing with technology in our small designated space, which luckily Matthew let us experiment with quite a bit. They were also interested with what we were representing in terms of our evaluation process. We started being consulted by other curators working with Matthew who were now being asked to formalise their evaluation. Rather than go outside the museum, they set up small conversations with Matthew and I. It was then that we started to realise the slow burn effect we were having on the overall culture of the museum. The next steps to us became normalising these informal processes so that Operational and Marketing staff came to trust and even support our decisions with very few questions or complaints. Comments started coming to us in the form of positive feedback.

3 The Bigger Picture: how prototyping became possible within the Powerhouse Museum This adoption of the living laboratory model by museum staff happened gradually, and can be identified within a series of case studies framed around Connell's initial reservations regarding launching the space. Where specific examples of these reservations will be discussed later, it is important to understand that Connell and Edmonds dealt with these reservations not as problems in themselves, but as challenges that could be worked through via the medium of exhibition and evaluation, or in scientific terms, queries with hypotheses and data that could be proved or disproved. The point being in the end, that the relationship, the collaboration that began was one that became mutually beneficial; the external researchers were able to make something robust enough for public evaluation, and the museum came away with a model for partnership that was proven over 30 case studies, 6 post-graduate degrees, and a good collection of academic publications. The empathy for the 206

project within the institution came from the belief that, in Connell's words, "We are a museum whose collection is based around the idea of making things. We are a museum that says people are at their most human when they’re making things." (Turnbull and Connell, 2010) The above quotation comes from an interview between Turnbull and Connell, where Connell muses again on the writings of Postman, who wrote compellingly on how all museums strive to answer the fundamental question: what does it mean to be a human being? The Powerhouse Museum, known by its old moniker of the Museum of Applied Arts and Sciences, strives to answer this question in terms of making. Being a museum charged with the task of collecting, caring for and exhibiting objects related to science, design and technology, Connell subscribes to the aspect of Postman’s philosophy where “human beings are essentially tool-makers, and at their best when solving practical problems”. He and Edmonds addressed the fact that museums should function in terms of supporting the question one would like to answer, as the answer will change depending on the time frame in which someone visits a certain centre that tries to define a particular aspect of humanity through their objects exhibition and explanations (Postman, 1995:65). Within the Powerhouse Museum, there was a climate of change that Beta_space could exist and operate within, one becoming more interested in creative activity concerning technology and, as with most museums, audience engagement.

4 Case Studies: Beta space as a model for a new museum When Edmonds came to the University of Technology, Sydney in 2002, he approached the Powerhouse Museum seeking a public space for user-testing experimental artwork in which data could be collected for his artist/researcher students. He was referred to Connell, who had some reservations based on previous museum-university research relationships, which succeeded or faltered for a number of reasons. Edmonds persisted, however, seeking to identify the problems and consider how they might be resolved, re-awakening in Connell the desire to establish a model for sustainable partnership between a museum and a university. Through his previous work with university and other research groups on the topic of digital creativity, Connell had identified 4 main challenges: 1) a certain level of robustness was required of the systems exhibited; 2) simplicity and recognisability in the systems were key; if a system was too complex, the level of engagement too subtle, the key demographic (families and school children) would give up and move on; 3) a certain level of presentation in regards to museum standards needed to be adhered to by the researchers; and 4) the museum needed to compromise rigorous operational procedures to accommodate the CCS research group.

4.1. Robustness – private vs. public problem solving The research works that came in had to be robust, to be far enough along in their development that it had already answered some of the questions posed by the researchers. The Creativity and Cognition Studios, accommodated this by devising a way forward early on, with the first prototype exhibition, Sidney Fels’ work Iamascope. From this study, these researchers, in collaboration with the artist, devised a stratagem for identifying which emotions to measure, and then how to measure them. A series of mandates were then developed, ones that all artworks being considered for exhibition would, from then on, aspire to meet, particularly in terms of evaluation (Costello et al., 2005:52). Some would meet these mandates, and some would not, depending on what iterative stage they were at. But usually, the more closely these mandates were met, the more successful the exhibition. A few key mandates were that each Beta_Space exhibition be tabled for discussion at the weekly CCS meetings, if possible with an evaluation outline; that each work then would receive an initial screening by Connell, usually at the studios where most of the works were constructed; and on a more technical note, that each system had a component of both self-starting and automated shut-down, or be otherwise started and shut-down each day by the artist, the technician, or one of the curators.


Figure 3. Andy Polaine, Time Sketches, 2006. Image Figure 2. Sidney Fells, Iamascope. Image

! Mike Leggett.

! Lizzie Muller, 2004.

Another work came out of collaboration with artist and academic Andy Polaine. Titled Time Sketches, this work was successful because it contained the nuances of immediacy and playfulness, and required only a simple level of engagement that could be understood by a range of audiences. Comprised of a duo of image-recognition based works called Time Smear and Time Splicer, Polaine and Muller (then the curator), explored and exhibited some of the very basic components of interactivity, time, movement, and visual feedback to stimuli, in a quite pleasurable and consistent way (Leggett [2], 2006).

4.2. Simple and recognizable is best Through the three examples below, the CCS researchers discovered certain modes of interaction that could be employed to engage the audience; ready conventions built into the system design and processes, that people could rely on their tacit knowledge to operate. These are described in CCS literature as attractors, sustainers, and relaters, and are the evaluative attributes assigned to the behaviour of audiences whilst observed interacting with various art systems in Beta_Space (Edmonds et al, 2006:307). George Khut’s work Cardiomorphologies, exhibited in September 2005, is an example of a museum attractor. As surmised by Edmonds, et al., an attractor is what “causes the audience to take note of the system in the first place” (Ibid, 2006:307). Khut’s work was installed for a research intense time frame of 2 weeks. Each day he showed up, ensured the system was operational, and performed user testing on the system with the general museum audience as his core subjects. The Cardiomorphologies system was also Figure 6. George Khut, Cardiomorphologies, 2005. Image intelligently designed to feedback the ! Greg Turner. subject’s heart and breath rate in a beautiful and compelling series of pulsating circles of colour; in effect dealing with everyone’s favourite topic: themselves. Though this exhibition/evaluation model was not sustainable over long periods of time, it was a successful snapshot of what could happen over an intense and focused


evaluation.12 Brigid Costello’s work Elysian Fields, 2003 exhibited at the launch of Beta_Space at a joint conference with the Powerhouse Museum in 2004 titled SPARK (Leggett [1], 2006). In terms of retaining the participant after their initial engagement, the sustainer attribute, Costello was able to adjust the system in situ, thus allowing for a more immediate feedback loop from audience engagement to system upgrade. The adjustment made to accommodate the audience was designed to be quite quick, a reaction time in the artist’s mind to the requirement of the audience recognizing Figure 7. Brigid Costello, Elysian Fields, 2004. Image ! the work was interactive. If the reaction time wasn’t adjustable, then physical Mike Leggett. objects could be placed within the space to indicate that a certain level of time was required to understand/enjoy the work.13 The third attribute, the relaters attribute, tries to encourage a relationship between the user and the system via a series of recognizable devices. We had the opportunity to test this idea over 2009, when Connell requested that a number of works inspired by or utilising video games aesthetics be showcased as a way to gauge singular and collaborative interactivity through the notion of play. A number of such works were exhibited and evaluated: Chris Nelson’s work, Seven Valleys, 200814 and Aram Dulyan’s work, AUXie, 2008-9.15

Figure 8. Aram Dulyan, AUXie, 2008-9. Image ! Aram Dulyan.

Figure 9. Chris Nelson, The Seven Valleys, 2008. Image ! Chris Nelson.

"5! #$%!&!3$J2)0-0!&33$(4-!$'!K*(-L,!0=&)(&-+$4!$'!!.-3&"7"-84"/"'&$%!+43)(7+4>!&(7+0430!%0,2$4,0:!2)0&,0!%0'0%!-$!-*0! 2&20%M! K*(-:! N0$%>0! O$$4P*+4! &47! 9())0%:! <+QQ+0! B5HHFD6! RS=$)=+4>! T%0&-+=0! O%&3-+30M! A! U0')03-+$4! $4! V$%P+4>! W+-*! A(7+0430! SX20%+0430! +4! T&%7+$J$%2*$)$>+0,Y6! 9&,./* :&'(%)* !-$.,&;$* <-.=,&=$* >* [email protected]* A$3&.* ?"@! A4&,-&,+$(:! O6:! Z%+,-:! K6! [! ! \J+-*+0,:! U:! BS7,[email protected]!O(?)+,*+4>:!90)?$(%40!:!5HHF6!226!5"6! "8! #$%! &! J$%0! 70-&+)07! &33$(4-! $'! -*0! 3%0&-+$4! $'! B/C%&.(* D&$/3%! &47! T$,-0))$L,! %0,0&%3*! +4-$! ,(,-&+4+4>! +4-0%&3-+=+-.! -*%$(>*!2)&.:!2)0&,0!=+,+-!-*0!$4)+40!=+70$!?.!9+P0!<0>>0--!B5HH]D!6!R/%+>+7!T$,-0))$M!\-+J()&-+4>!T%0&-+=0!O)&.Y6!A!TT\!2%$7(307! %0,0&%3*! '+)J6! ^Z\M! \.740.:! A(,-%&)+&6! 5! J+4(-0,:! CI! ,03$47,! +4! )04>-*6! A330,,07! 5H! #0?%(&%.! 5H""! &-M! *--2M__WWW63%0&-+=+-.&473$>4+-+$463$J_=+70$_/%+>+76J$=!! "C! A! 70-&+)07! $(-)+40! $'! -*0! \0=04! `&))0.,! 3&4! ?0! '$(47! +4! a0),$4L,! &%-+3)0! RZ*0! \0=04! `&))0.,Y! +4! ! [email protected]! LHIM! O%$3007+4>,!$'!-*0!547!+4-0%4&-+$4&)!3$4'0%0430!$4!b+>+-&)!+4-0%&3-+=0!J07+&!+4!04-0%-&+4J04-!&47!&%-,6!AT9:!a0W!c$%PM^\A6! \02-0J?0%!5HHI6!226!"[email protected]\/aM!EIde"eFEFE8eIHdeI6!! "F! b().&4L,! %0')03-+$4,! $4! -*0! 0X*+?+-+$4! &47! 0=&)(&-+$4! $'! A^f+0! 3&4! ?0! '$(47! +4! *+,! &%-+3)0! W+-*! S%40,-! S7J$47,M! RA^f+0M! +4+-+&)! 0=&)(&-+$4! $'! &! ?)+47e&330,,+?)0! =+%-(&)! J(,0(J! -$(%Y6! [email protected]! L"HM! <-"=$$3&('%* "2* ,4$* EE(3* !"(2$-$(=$* "2* ,4$* !"781,$-FG17.(*+(,$-.=,&"(*:8$=&./*+(,$-$%,*H-"18*"2*51%,-./&.*"(*!"781,$-FG17.(*+(,$-.=,&"([email protected]\@N!&47!AT9:!a0W!c$%P:! ^\AM!5H"[email protected]\/aM!EIde"eCFH8eHFH5eH6!


What was learned from these examples was that there were different ways to approach and facilitate experimenting in the prototype vein. As long as the system was both recognisable and robust enough, it could withstand both museum audiences as a stand-alone installation, and user testing when being evaluated.

4.3. Research vs. Museum Cultures – researchers needed to accommodate for museum standards Creating a living laboratory within a museum was no small task. There was a need for researchers to understand the rules we needed to abide by, safety operational standards for example. As this research took place in public space and real time, researchers needed to respect that it was not as free an environment as they were perhaps used to. Where operational rules did need to shift, researchers needed to encourage this shift slowly, step outside the mould carefully, in small doses and in contained ways. One Beta_Space exhibition comes to mind when considering this ability for exhibition design to shift museum culture, and that is Julia Burns work, The Musicians, 2007 (Musicians, 2007). Because she was a contemporary practising media artist involved with initiatives such as Dorkbot (Dorkbot, n.d.) and Firstdraft (Firstdraft, n.d.), Burns approached the museum as an exhibition space for new media art, rather than as a public laboratory. She joined the CCS research group from UNSW and went on to complete an MSc with the group. In doing this, her predominant interest was in exploring the creation and exhibition of new media work. Where The Musicians was an early prototype work, we learned much from it in terms of audience enjoyment. Many of the people who spent a lot of time near the Figure 10. Julia Burns, The Gaze, 2007. Shown here installed work, museum workers such as gallery attendants and security at Firstdraft, Sydney. Image ! Julia Burns. personnel, quickly tired of the hold music, which acted as Burns’ attractor. Once inside, the film work was engaging and entertaining (two musicians who became animated in certain ways depending on where you stepped on the hidden floorpads), albeit for too brief a time. The input device to reaction ratio was often too subtle, even when adjusted in situ. Overall, the key demographic needed more immediate feedback for less investment, which can be an immediate, yet difficult lesson to learn.

Figure 11. Julia Burns, The Musicians, 2007. Image ! Julia Burns.

Figure 12. Julia Burns, projected MaxMSP patch on mattress for Drifting, 2009. Image ! Julia Burns.

However, when it came time to host a public programme for the work, it was one of our most well attended seminars. Here Burns presented her preliminary research and called for group feedback based 210

on experience of the work. Along with quite a lively debate, she invited the actual musicians from the film in to play for the forum guests adding a layer of live experience to the virtual one being hosted in the gallery. Though Burns found it difficult to exhibit this work, it was ultimately a rewarding experience for both artist and curators. In fact, her second exhibition in Beta_Space, the previously mentioned Drifting in 2009 (Burns, 2009), garnered funding and toured both Australia with D/Lux Media Arts (D/Lux Media Arts, n.d.) and The Gaze (Burns, 2007), the first prototype Turnbull saw of hers at Firstdraft, has gone through an iterative cycle and is now touring Asia with Experimenta (Experimenta, n.d.). Burns’ initial discomfort with the museum environment was overcome, and based on her previous experience with The Musicians, Burns was better able to navigate the perceived restrictions of the environment.

4.4. A shifting museum culture – when the museum had to adjust to researcher’s needs Connell knew, and Edmonds agreed, that the museum might need to bend; the imperatives of university research are not always recognised within the museum. The understanding that we were dealing with experimental works and ideologies, and that museum rigour in regards to things like security and protocol would need to be relaxed for academic visitors, came quickly within the first two years, creating an environment for the below, quite unconventional exhibitions. Ian Gwilt's Mundane Traces tested not only the technological capability of augmented reality, but it prototyped a number of aesthetics that would support the exhibition of an artist working with multiple platforms to convey meaning. The components that comprised this portable augmented reality work, a handheld device containing a computer algorithm activated when recognising a mounted perspex folder (Save_as, 2007), fed into a larger culture of image ecologies, each of which were themselves smaller explorations into digital arts experimentation. Rapid prototype sculptures of desktop iconography sat on top of underlit plinths (Folder field, File field, One addition of Many, all 2006), an animation of one of which was produced, scored and played back via the large screen and a 5-channel surround sound system (Scrolling_heaven, 2007). Finally, a series of laser-cut cardboard sculpture were mounted on the wall (Folder_wall, 2007).

Figure 15. Ian Gwilt, Mundane Traces, 2007. Image ! Deborah Turnbull.

Figure 16. Ian Gwilt, Folderwall, Mundane Traces, 2007. Image ! Ian Gwilt.

This exhibition showcased six works, only one of which was interactive. There were reservations, but upon persevering, this exhibition marked our ability to work within and around both the museum and Beta_Space mandates to produce a sort of miniature exhibition, activating the space as a showcase for not only the prototype art system, but for the ability to create discourse regarding the ideological framework of the objects on display. It was a discourse that was appealing to both the Design department and the university research group, and it was permitted the length of its proposed evaluation cycle (Gwilt & Turnbull, 2008).


Figure 13. Ian Gwilt, Save_as, Mundane Traces, 2007. Diana Lorentz, Head of Design, augmenting reality. Image ! Ian Gwilt

Figure 14. Ian Gwilt, Save_as, Mundane Traces, 2007. Detail. Image ! Ian Gwilt

Similarly, Seevinck’s work, +/-now, became an 'ice-breaker' in terms of what Beta_Space could accomplish. Her design called for the construction of a wooden plinth that would house a computer tower and image recognition camera facing upward. Atop the plinth, a rectangular Perspex container would house sand. A projector situated above the plinth would project gestural images onto the sand in rainbow colorations. A final instalment of visual feedback came to the audience via the 3mx3m backlit screen which contained gray scale images of their gestures on the sand. There was an overarching concern with the use of sand, considered a live material by the Occupational Health & Safety committee in the Design Department at the museum. Connell worked with the Design Department to locate suitably treated (bleached) sand, devoid of any living entities, available at the local hardware store. The outcome of works that push these operational boundaries, which we are largely meant to uphold, is that people within the museum begin to see what is possible. With Burns’ works, we saw that there was a massive interest in the more popular media arts community with any space engaging in interactive arts creation, whether or not the prototype was successful. This was something we tapped into in terms of advertising our events over social media and slightly less academic mailing lists in order to generate interest and discussion regarding our practice. With Seevinck’s work, we learned that asking why is very important. Finding the right kind of sand to suit the museum, has in fact allowed a much larger university-museum collaboration, which mimics a Mars environment. The resulting exhibition is largely comprised of the same sand that Seevinck used as her tactile input mechanism.

Figure 17. Jen Seevinck, +/- now, 2008. Detail of input mechanism comprised of sand. Image ! Aram Dulyan.

Figure 18. Figure 17. Jen Seevinck, +/- now, 2008. Detail of screen output. Image ! Aram Dulyan.


5 Conclusion: the test becomes the control Beta-Space is a new organisational model that enables practitioners to prototype art systems and evaluate experiences with active audiences. Beta_Space, blurs the distinction between production and presentation through an iterative approach to creating and displaying art works. The audience is closely involved in this process, changing the relationship of the artist and curator to the audience, and of the audience to the artwork. In these ways, the museum can begin to play a vital role as a laboratory for the creation of new work and new knowledge. Beta_Space became a hybrid space of change, one that touches on and moves within the museum context in several ways. Within the museum it has brought about a cultural shift. Beta_Space broke all the rules a museum sets in terms of securing, programming, exhibiting and launching design objects; but it broke them gently so that the museum was able to see in small doses what was possible to change with successful research partnerships. The museum is now prepared to take on this model of partnership both with external entities, but also, and most importantly, with an active and integrated audience as well.

Acknowledgements The authors would like to specially thank those who made the Beta_Space project possible, namely Lizzie Muller, Greg Turner, Alastair Weakley, Zafer Bilda, and Julia Burns. We would also like to thank the resident technicians and photographers, Julien Phalip and Aram Dulyan. Thank you also, to the CCS research group who are consultants to each exhibition, and to the staff at the Powerhouse Museum, without whom we wouldn’t have a space to play.

References Barr, A.H. (1939) “Art in our time: the plan of the exhibition.” Art in Our Time. Ayer Publishing: New York. Bilda, Z. and Turnbull, D. (2008) Beta_Space Manual. Quarterly Report to the Australasian CRC for Interaction Design, QUT, Brisbane. Burns, Julia. (2007) “The Musicians”. Beta_Space exhibition #14, June 2007. Accessed 20 February 2011: Burns, Julia. (2007) “The Gaze”. Firstdraft Exhibition, March 2007. Accessed 20 February 2011: Burns, Julia. (2009) “Drifting”. Beta_Space exhibition # 24, January 2009. Accessed 20 February 2011: Costello, B., Muller, L., Amitani, S. and Edmonds, E. (2005) "Understanding the Situated Experience of Interactive Art: Iamascope in Beta_Space." In Proceedings of Interactive Entertainment CCS Press, Sydney, Australia. D/Lux Media Arts, n.d. Accessed 20 February 2011: DorkBot, Sydney, n.d. Accessed 20 February 2011: Dulyan, A. and Edmonds, E. “AUXie: initial evaluation of a blind-accessible virtual museum tour” OZCHI '10: Proceedings of the 22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction. CHISIG and ACM, New York, USA: 2010. pp. 272-275. ISBN: 978-1-4503-0502-0. Edmonds, E. A., Muller, L, and Connell, M. (2006) “On Creative Engagement”, Visual Communication. 5(3). pp 307-322. Experimenta, n.d. Accessed 20 February 2011: 213

Firstdraft, Sydney, n.d. Accessed 20 February 2011: Gwilt, I., and Turnbull, D. (eds.). (2008) Mundane Traces: essays on transitional art practises. DABdoc 8. DABPublications, Sydney. Hughes, Philip. (2010) Exhibition Design. Laurence King Publishing Press. London, UK. Khut, George Poonkhin and Muller, Lizzie (2005). “Evolving Creative Practice: A Reflection on Working with Audience Experience in Cardiomorphologies”. Vital Signs: Creative Practice & New Media Now Anastasiou, P., Trist, K. & Smithies, R, (Eds). RMIT Publishing, Melbourne , 2005. pp. 21. Leggett, Mike [1]. (2006) . “Brigid Costello: Stimulating Creative Play”. A CCS produced research film. UTS: Sydney, Australia. 2 minutes, 47 seconds in length. Accessed 20 February 2011 at: Leggett, Mike [2]. (2006) “Lizzie Muller: Audience Experience of Interactive Art”. A CCS produced research film. UTS: Sydney, Australia. 3 minutes, 24 seconds in length. Accessed 20 February 2011 at: Muller, L. and Edmonds, E. A. (2006) “Living Laboratories: Making and Curating Interactive Art”, SIGGRAPH 2006 Electronic Art and Animation Catalog. ACM Press, New York. pp 160-163. Muller, L., Edmonds, E., Connell, M. (2006) "Living Laboratories for Interactive Art". CoDesign: International Journal of CoCreation in Design and the Arts, 2(4), pp. 195–207. Nelson, Chris. “The Seven Valleys”. DIMEA '07: Proceedings of the 2nd international conference on Digital interactive media in entertainment and arts. ACM, New York:USA. September 2007. pp. 179. ISBN: 978-1-59593-708-7. Postman, Neil. (1994) “Museum as Dialogue”. Museum Provision and Professionalism (Gaynor Kavanagh, ed). Routledge, London, UK. Seevinck, J., Edmonds, E.A. (2008). "Emergence and the art system `plus minus now'." Design Studies 29 (6). pp. 541-555. Turnbull, D. and Connell, M. (2010) "Protoyping Places": In Conversation. Recorded and transcribed interview from 10 December 2010. © New Media Curation, Sydney, Australia


Asian Heroes in Cultural Heritage Context: An interactive installation for children combining language and mythology. Ng Bee Chin, Fabrizio Galli, Halina Gottlieb College of Humanities, Arts and Social Sciences Nanyang Technological University [email protected]

Abstract In this paper, the authors explore the interdisciplinary process involved in the creation of an interactive puzzle on intangible cultural heritage. The focus is to demonstrate the important roles that intangible heritage such as language and mythology play in transmitting cultural heritage. Through the prototype developed, the researchers highlight how different stake holders can work together in promoting and preserving intangible heritage. One of the features developed for the Asian Heroes exhibition is an interactive puzzle in a multi-touch table. The aim is for children to learn about both tangible and intangible forms of cultural heritage in a fun and interactive manner. Targeted at children and family groups, the game aims to broaden their knowledge of four Asian mythical figures. In the prototype developed, the focus is on the Chinese heroine, Hua Mu Lan. The game is a simple jigsaw puzzle designed in a story mode for children. The objective of the Asian Heroes puzzle is for the player to complete putting together five scenes from the story of Hua Mu Lan and learn more about the legendary heroine and the values and culture that define this character. The colorful visuals, lively music and clear audio narrations all contribute to the aesthetic appeal of the game. The stories displayed on the table combine myth, language and images engaging the visitors in a fun and playful learning process with interactivity as a tool to foster interest in the gameplay. In this paper, we will first discuss the interdisciplinary collaboration, the interactive touch table project, its context of heroes of Asia, the prototype development and the design process. We will also present the evaluation study comparing the Mulan traditional display from the museum to our Mulan interactive touch table carried out at the Asian Civilisations Museum. Finally, we will present our findings and conclusion based on the responses collected in Singapore.




Tangible and intangible heritage

In the last three decades, the research community has come to a better understanding that material or tangible heritage such as monuments, buildings, nature sites alone, cannot adequately capture the history and the heritage of our collective existence. The notion of Intangible Cultural Heritage (ICH) as a significant heritage which needs safeguarding has become topical over the years (see Kurin 2004 and Bouchenaki 2004 for a historical synopsis outlining the discourse surrounding the rising awareness of intangible heritage). The term ‘intangible cultural heritage’ is defined by Kurin as a list of oral traditions and expressions that includes performing arts, folktales and stories, cultural practices and rituals, the knowledge about the world/universe and also sites and places where cultural events are carried out. (See Appendix 1 for the definition of ICH in Article 2 by UNESCO.) Bouchenaki (2004) took a similar view but emphasized the context in which the ICH is located more strongly. He argued that ICH is dependent on the actors and the social and environmental conditions and that it is intricately linked with its owners/creators. ICH has been loosely described as the underlying spirit of the people or community Kurin (2004). Giaccardia (2011) sees ICH as something which is more deeply embedded and which in fact, shapes the people’s identity. As can be seen, what constitute ICH is far from being precise, though there is a profound consensus endorsing the notion of “intangibility”. So far, there are three major formal interventions which are critical in raising the profile of intangible heritage. The first attempt is the Recommendation on the Safeguarding of Traditional Culture and Folklore which largely went unnoticed (Kurin 2004). The next landmark is the Proclamation of Masterpieces of the Oral and Intangible Heritage of Humanity launched in 1998. The proclamation identified 19 oral traditions to be valued. This second UNESCO initiative created a stir and succeeded in drawing the public’s attention to intangible traditions such as festivals, narrative traditions and


performance art. In addition, the Proclamation highlighted the concept of cultural space essential for the transmission of culture, for example, the Jemaa-el-Fna square in Morocco and the Kihnu Cultural Space in Estonia. Since then, this list has increased to 232 by 2010 (UNESCO ‘Lists of Intangible Heritage’.). The list includes the sand drawings from Vanuatu, the Songs from Sanaa, the Pansoric Epic Chant from Korea and the Royal Ballet from Cambodia. Also included are the oral traditions of the Aka Pygmies from the Central African Republic, the oral traditions of the Gelede from Benin, Nigeria and Togo and the expressions of the Wajapi from Brazil. The Proclamation of Masterpieces of the Oral and Intangible Heritage formal recognition of the many forms of intangible heritage is an important precursor to the most significant recognition of ICH to date - the 2003 UNESCO Convention on Safeguarding Intangible Cultural Heritage. This remains the most critical turning point as ICH is critically appraised and recognized by an international community, together with a strong political and economic will to implement many of the recommendations. The convention also marks a mind shift in the way researchers relate to the subject of their enquiry and the notion of “safeguarding” takes on a dimension that is more than just preservation (Smith and Akagawa 2009). UNESCO (2003) spells out that “Cultural heritage is not limited to material manifestations, such as monuments and objects that have been preserved over time. This notion also encompasses living expressions and the traditions that countless groups and communities worldwide have inherited from their ancestors and transmit to their descendants, in most cases orally.” (UNESCO “Intangible Heritage”) There is deep recognition that unlike buildings and paintings which can be preserved through various means, ICH will die and can only be preserved if it is practiced by the people who own it. Up till then, the literature on ICH has focused on what exactly is intangible in ICH and why it should be treated as something separate from tangible heritage. Some researchers consider pinpointing intangibility as the pressing lead to understanding how we can recognise, treat and preserve ICH. Since 2004, the chorus of voices took a discernible turn. Advocates of intangible heritage have argued strongly that intangible heritage provides the larger framework and context for us to interpret tangible heritage (Bouchenaki 2004, Munjeri 2004, Ito 2003). These researchers emphasized the interrelatedness of both intangible and tangible heritage. Munjeri (2004) aptly illustrated that a monument or site is considered worth preserving because it reflects the values or the cultural practices of the community. The symbolic value of these sites are lost or greatly reduced if we have no way of understanding, documenting or preserving the beliefs and values ascribed to the monument. Intangible heritage has also been found to be critical in uncovering the links between the conservation of biological and cultural diversity in Alamos, a Mexican heritage site (Hoekstra 2010). More specifically, language is seen to play a key role in the maintenance and transmission of intangible heritage (Evans 2010). In terms of methods and approach, researchers have also found that that unlike monuments, ICH is more fragile and vulnerable, requiring different modes of safeguarding that have been used for tangible heritage.


Language and mythology as intangible cultural heritage

As pointed out by Kurin (2004) while oral traditions and expressions were clearly identified as intangible heritage in the UNESCO 2003 Safeguarding Intangible Cultural Heritage Convention, the role of language was not so clear. However, in Article 2 (see Appendix 1 for the complete Article) of the convention, there is an unambiguous mention, acknowledging the importance of language (italics inserted). The “intangible cultural heritage”, as defined in paragraph 1 above, is manifested inter alia in the following domains: (a) oral traditions and expressions, including language as a vehicle of the intangible cultural heritage; (b) performing arts; (c) social practices, rituals and festive events; (d) knowledge and practices concerning nature and the universe; (e) traditional craftsmanship.


Linguists working with communities which are endangered have long observed and documented the synchrony of language obsolescence and the loss of cultural heritage. UNESCO has since more publicly acknowledged and endorsed the importance of safeguarding language as a mean of preserving ICH. In answer to the question “Are languages types of ICH?”, UNESCO’s statement unequivocally identifies ‘language’ as a form of intangible heritage. “In relation to the intangible heritage domain of ‘oral traditions and expressions’, language is not only a vehicle of intangible heritage, it is their very essence”. (UNESCO 2010, “What is Intangible Cultural Heritage?”) Though some linguists may debate about the exact relationship between language, culture and identity, there is no doubt that all will agree that language loss is a harbinger of the final and eventual loss and erosion of cultural heritage (see for example, Dorian 1989, Brenzinger 1992, Matsumura 1998, Austin 2002, 2004, De Graaf 2004, Ostler 2005, Grenoble and Whaley 2006 and Evans 2010). By conservative estimates, 50% of the languages spoken in the world (approximately 6000 languages) will die by 2050. This has profound and devastating implications for conservation of ICH. Hence, safeguarding language is a pressing and integral part of safeguarding ICH. Evans (2010) warned about the collective and irrevocable loss faced by our civilization if we do not do more to stem this tidal wave of language death which is surely coming our way.


Safeguarding ICH in the digital era

According to Article 2 of the UNESCO Safeguarding ICH Conventions, “Safeguarding” means measures aimed at ensuring the viability of the intangible cultural heritage, including the identification, documentation, research, preservation, protection, promotion, enhancement, transmission, particularly through formal and non-formal education, as well as the revitalization of the various aspects of such heritage. What is the practice in reality? How do different practitioners/researchers go about safeguarding ICH? The field of ICH is a relatively new with only one journal (Journal of Intangible Heritage) devoted to this area of study. A valuable contribution is the edited volume by Smith and Akagawa (2009) which discussed many emergent issues in the field. The Journal of Intangible Heritage is six years old and a cursory look at the table of contents of each issue shows an eclectic assembly of participants from various fields but noticeably absent in all the discussion is the voice of linguists. Given the critical role that language plays in the transmission of culture, the absence is odd. This is more so when one considers that linguists have long been actively engaged in preserving this very important aspect of ICH. It seems the two camps (cultural/design/media specialists and linguists) have been working alongside each other without much interaction. The next section will outline the contribution from linguists as well as interactive artists and designers with an aim to bridge the concerns from both sides. 1.3.1 Language description and documentation work by linguists In this section, we will briefly describe the work done by linguists in safeguarding language. Describing and documenting languages is a field in linguistics. Traditionally, scholars and PhD students spend long tracks of time (minimum one to two years, often, much longer) working and living in a community where they are doing language description work. More often than not, these linguists choose a community with dwindling number of speakers with the explicit aim of documenting and preserving the language. The documentation process typically involves writing a grammar of the language and often includes the production of a dictionary. This work is laborious and painstaking, entailing many thousand hours of detailed work in the field. Ideally, the linguist usually “apprentices” herself or himself to the speakers in the community and document the language as they learn to speak it. One main aim is the production of a description of the language which provides a record of the language as it is spoken. Most of the time, audio (and later), video recordings are also made. Before the late 90s, the tools for this kind of documentation work were fairly standard involving a recorder and a video camera and at the end, a book detailing the structure and use of the language in question is produced. The documentary “The Linguists” (2008) featuring two field linguists provide an entertaining and realistic portrayal of how linguists do their language description work.


Over the years (especially in the last 20 years), language description has taken on a more ecological turn. Many linguists grimly witness the passing of the languages they work with. With the realization about the fragility of many of the world’s languages, language endangerment has become more topical and with the emergence of new technology and the internet, the way linguists describe and document languages is seeing a dramatic change. Now, linguists are not only concerned with describing a language, they are also more likely to exploit the technology around them to try to preserve and enhance the language using multi-modal means. While many linguists have always been attentive to making sure that there is knowledge transfer to the community they work with, the linguists of today are amply aware of the pressing need to not only engage in documentation work, but the importance of revitalization, as well as the ethical responsibility of providing support to the community they work with as well as actively empowering the community to participate in safeguarding their own heritage. The focus has shifted from the communities being ‘objects’ of study and preservation to an emphasis on building collaboration so that there is continuity in the preservation and transmission of the heritage. Various linguistics organizations (e.g. DoBes ‘Documentation of Endangered Languages’ ELAR ‘The Endangered Languages Archive;; or DELAMAN ‘Digital Endangered Languages and Music Archive Network ‘ have been set up and all are successfully fulfilling the function of documenting and archiving languages. However, critics of documentation process have argued that language is found in active practice and not in “frozen” language records. Even UNESCO has not been consistent in their views on language as they contradicted their support and endorsement of language on their “Oral traditions and expressions” page. Although language underpins the intangible heritage of many communities, the protection and preservation of individual languages is beyond the scope of the 2003 Convention, though they are included in Article 2 as a means of transmitting intangible cultural heritage. Different languages shape how stories, poems and songs are told, as well as affecting their content. The death of a language inevitably leads to the permanent loss of oral traditions and expressions. However, it is these oral expressions themselves and their performance in public that best help to safeguard a language rather than dictionaries, While there is no doubt that oral traditions are important, it remains doubtful that we can preserve language just by preserving oral traditions. Can the ability to recite poems, do a prayer chant or sing a song in a language translate directly into the ability to have meaningful conversation in the same language? It is our contention that ‘oral traditions and expressions” have to be safeguarded together with languages and one is not less important than the other. It is only when we work with this in mind that safeguarding ICH can be successful. 1.3.2 Digital intangible heritage As indicated the last section, the field of intangible heritage is relatively new. The application of digital technology or new media has only just begun to be used for cultural heritage in general and we are beginning to see some innovative work involving the use of digital technology for monuments and heritage sites (Kalay 2008). The volume edited by Kalay et al. (2008) presented many thought provoking projects on how digital media meets cultural heritage but none of the projects presented tackled oral traditions and expressions. There may be about a dozen or so published installations and/or exhibitions which were launched with the explicit aim of featuring ICH. Giaccardia (2011) lists five and Lupo (2007) presents four such installations. Most of these installations tend to focus on stories and memories of the place or community centred on oral history and oral traditions. Some of the installations featured a platform that involves ‘interactive memory sharing’. So far, none of the installations feature language as an ICH. Ng, Stulemejier and Linborg (2008) installation on “Singapore Voices” is the only one that addresses the issue of language endangerment through the voices of eight speakers from eight endangered varieties of languages in Singapore. This fruitful experience has led to the current collaboration with linguist working with researchers with expertise in digital heritage and interactive and product design.



Asian Heroes – Description of the Study

2.1 Interdisciplinary Collaboration "Asian Heroes" is a pilot project of intense interdisciplinary collaboration involving several schools in Nanyang Technological University (NTU) and the Asian Civilisations Museum (ACM) of Singapore. The project is part of the research cluster, Digital Intangible Heritage in Asia (DIHA) and one of the main aim of this cluster is to capture the vibrant cultural and linguistic fusion of Singapore through interdisciplinary lenses. The foremost focus of DIHA is to bridge tangible and intangible heritage through interactive digital media in order to enhance the conservation and interpretation of cultural knowledge. Within NTU, the team came from the Institute for Media Innovation (IMI), the School of Art, Design and Media (ADM) and the Division of Linguistics and Multilingual Studies (LMS) from the School of Humanities and Social Sciences (HSS) and DIHA. Individuals with various competencies collaborated together to work on this project. From ADM, the participants consisted of the interactive programmer, hardware/equipment specialist and graphic designers. The linguists and contents researchers were from LMS. On top of the core group, many other parties such as educators, engineers and curators provided input which helped in the completion of the project.


The Interactive Touch Table Project

The research project focused on a feature developed for the "Asian Heroes" exhibition at the Asian Civilisations Museum; an interactive table-game with a multi-touch interface. The project used simple interactive technology to convey intangible heritage such as culture, mythology and language in the course of the game play. The aim was for children to learn about both tangible and intangible forms of cultural heritage in a fun and interactive manner, and to transmit the social values via cultural stories. The game begins with a display of five carefully selected scenes from the story of Hua Mu Lan. The scenes were chosen based on the significance to the story line and the values portrayed by the scenes. The scenes then dramatically disintegrate leaving the background empty. One by one, a representing silhouette of each scene from the story appears together with the jigsaw pieces and other non-related fragments randomly floating around the silhouette. In accompaniment to music, the child (children) has to drag the pieces to the silhouette to reform the puzzle. Each time the scene is completed, the respondents are rewarded with a story aligned with the scene. This story delivery is done via both visual and audio means in both the English and Mandarin Chinese. The game ends when all five scenes are completed1.


The Context: Heroes of Asia

The four Asian heroes proposed by the Asian Civilisations Musuem for this project are: (1) Hua Mulan Hua Mulan is a Chinese female heroine and legend from 6th century during the Northern Wei dynasty of China that symbolizes the quality of filial piety. The story of her taking over her frail and old father’s place in the Chinese army in the time of war is one told through generations in Chinese households and schools. This character was selected as the focus for the interactive touch table. (2) Hanuman Hanuman is an Indian demigod who has a human body with monkey features. Hanuman symbolizes the quality of loyalty through the story of his commitment to save Lord Rama’s wife Sita. (3) Bahrum Gur Bahrum Gur is a Persian king from West Asia who ruled his kingdom with justice and wisdom. He embodies wisdom and fairness.


For an online demonstration of the Asian Heroes Puzzle, go to this link



(4) Sang Kancil Sang Kancil is a character that appears in Malaysian and Indonesian folktales. He is a mouse deer that uses his intelligence and wit to overcome his enemies.


Prototype Development

The interactive touch table is designed to hide and contain the projector and mirror, video camera, infrared LED and speakers. Figure 1 illustrates the prototype containing indications of the equipment’s location: projection cone in blue and video camera capture cone in green. All the equipments are located within a self-standing structure independent from the table’s body to assure no vibrations during the use.

Figure 1 Executive drawings of the table prototype showing the equipment inside. In the completed table, one side of the table cover is removable for everyday operations on the technical equipment (see Figure 2).

Figure 2 Configuration of the interactive touch table puzzle. 2.5 Design Process In order to research the different styles of artwork that would appeal to children of the 6-14 demographic, we looked into styles that ranged from paintbrush to flat-colour 2D tones. Also, we kept in mind the cultural background of the Asian heroine, Mulan, and the artistic styles typically associated with her. In terms of the illustration style, we gave the character a more Chinese look with the usage of watercolor as the main coloring medium. The outcome was one that was kid-friendly and yet still pays tribute to the tradition. Figure 3 contains examples of the process involved in the graphic and art work.


Figure 3: Examples of processes in graphic and art work in the Asian Heroes puzzle


Field Study



An evaluation study and exhibition was carried out at the Asian Civilisations Museum (ACM) for a three day period from 5th to 7th November 2010. These dates were selected as they coincided with the “Supermighty! Heroes of Asia” exhibition and the “The Little Explorer’s Travels” event held by the museum especially for children and families. The objective of this evaluation study was to assess the responses of visitors to the Mulan interactive touch table and the Mulan panel display (see Figure 4). The interactive touch table was situated at the lobby of ACM near the entrance while the panel display was situated on the second level of the museum. The panel display had a few mechanical interactive elements including drawers for children to open to discover phrases, peepholes for them to locate Mulan as well as movable flaps hiding key elements of the story. The interactive puzzle usually takes a child approximately 3-7 minutes (longer for a younger child). In comparison, an average visitor spends about 10-15 minutes at the static display. Four interviewers were involved in collecting responses from the visitors and two were situated at each of the exhibitions. Participation in the study was voluntary.

Figure 4 The Mulan panel display and the Mulan interactive touch table.




The target respondents were children between four years old to 12 years old. An adult group was also included to assess the reactions from the adult perspective. The majority of the respondents came with their family. The interviewers randomly approached visitors who played at the Mulan interactive touch table and those who viewed the Mulan panel display to fill in the questionnaire. The younger children (those below 8 years of age) were interviewed orally to obtain their responses on the respective exhibitions while the older ones completed the questionnaire on their own (see Figure 5). Magnets of different hero characters and lollipops were given to each participant in the study as a token of our appreciation for their time.

Figure 5 Respondents of the Mulan interactive touch table and a young respondent being interviewed for her responses. In total, 214 respondents took part in the evaluation; 46.7% were respondents for the panel display and 53.3% were respondents for the interactive touch table. The distribution of respondents for each medium of exhibition is illustrated in Table 1 and Table 2. The language background of the participants were noted as there were some questions related to the use of Mandarin Chinese which were more relevant to those participants who did not speak Mandarin Chinese. Table 1 Percentage distribution of respondents who viewed the Mulan panel display across age groups (N = 100)

4-5yrs N = 16 Gender

Citizens hip Speaks Chinese

6-8yrs N = 44

9-10yrs N = 25

11-12yrs N=9

18yrs & above N=6



















Singapore an



















Table 2 Percentage distribution of respondents who viewed the Mulan interactive touch table acros age groups (N = 114).

4-5yrs N = 27 Gender


9-10yrs N = 21

11-12yrs N = 12

18yrs & above N=6



































Foreigner Citizensh Singapore ip an Speaks Chinese

6-8yrs N = 45


A two-page questionnaire was used to collect responses on the Mulan panel display and the Mulan interactive touch table. The questionnaire consisted of a total of 14 questions with eight statements which required responses on a modified Likert-scale for children, five multiple choice questions and one open-ended question. (See Appendix 2 for the questionnaire on the Mulan panel display, and Appendix 3 for the questionnaire on the Mulan interactive touch table.) The statements and questions for both types of exhibition were slightly different based on the context. For example, in rating the fun level of the respective exhibits, the statement for the Mulan panel display is “This Mulan display is good fun. I want to see it again” whereas the statement for the Mulan interactive touch table is “This puzzle is good fun. I want to play this game again.”



Two different analyses were carried out to compare the questionnaire responses on the Mulan glass display and the Mulan interactive touch table. Firstly, a simple frequency count was used for multiple-choice questions. Secondly, the independent samples T-test was used for responses to statements rated on a 5 point Likert-scale. This test provided statistical results for comparison between responses on the panel display and the interactive touch table respectively. In addition, a One-way Analysis of Variance (ANOVA) was carried out to compare the responses among the different age groups on the Mulan panel display and the Mulan interactive touch table respectively.


Panel Display

Figure 6 Fun level of the Mulan panel display and Mulan interactive touch table. When asked to rate the level of fun (refer to Figure 6) for the Mulan panel display and Mulan interactive touch table, respondents rated the interactive touch table (63.16%) as more enjoyable than the panel display (37%). In the first multiple choice question, respondents were asked to choose the characters that are Chinese characters and the majority of the respondents (89.29% from the interactive touch table and 87% from the panel display) from both medium of exhibition answered the question correctly. In the second multiple choice question, respondents were asked to choose the Chinese characters that read ‘Hua Mu Lan’ (花木兰) and there were slightly more respondents from the interactive touch table (88.39%) who answered the question correctly as compared to respondents from the panel display (77%). In the third multiple choice question, respondents were asked to choose the Chinese characters that read ‘filial piety’ and half of the respondents (50% from the interactive touch table and 43% from the panel display) from both medium of exhibition answered the question correctly.

Panel Display

Figure 7 Responses to the question “What is the story of Hua Mu Lan about?”. In the forth multiple choice question, respondents were asked what the story of Hua Mu Lan was about and there responses were mixed (see Figure 7). Of the respondents who viewed the panel display, 37%


chose “Hua Mu Lan is very brave”, 25% chose “Hua Mu Lan is very filial” and another 25% chose “Hua Mu Lan is a good warrior”. Of the respondents who played with the interactive touch table, 38.39% chose “Hua Mu Lan is very filial”, 29.46% chose “Hua Mu Lan is very brave” and 23.21% chose “Hua Mu Lan is a good warrior”. While all the above answers were correct, the core value of the story of Mu Lan is her filial piety, not her bravery or her warrior skills. In this case, the interactive table seems to have transmitted this core value more successfully than the ‘the Panel Display’.

Panel Display

Figure 8 Aspects of the display and puzzle that respondents like. In the fifth multiple choice question with one open-ended choice, respondents were asked to indicate what they liked about the panel display or interactive touch table (see Figure 8). For respondents from the panel display, 38.38% liked the pictures, 22.22% liked other aspects of the panel display and 20.20% liked that the panel display is fun. For respondents from the interactive touch table, 37.50% liked that the interactive touch table is fun, 24.11% liked that they can play it together with their family and friends, and 20.54% liked the pictures. For respondents of the interactive touch table, they were asked to rate the likeliness that they knew how to play the game themselves as they started. 48.25% stated with a ‘100% yes’ followed by 28.95% who stated ‘yes’. When they were also asked to rate their enjoyment level on the game, 70.18% stated with a ‘100% yes’ followed by 24.56% who stated ‘yes’. Table 3 Comparison of Mulan interactive touch table and Mulan panel display


Attention Talk Easy Level Beautiful





Standar Independent Samples T-test d Deviati Result on 1.12 t (202.92) = -3.84, p < .05(twotailed) 1.21



















Medium of Exhibit


t (196.71) = -4.43, p < .05(twotailed) t (152.93) = -3.95, p < .05(twotailed) t (175.16) = -3.43, p < .05(twotailed)


Table 3 compares the two types of exhibit based on results of the Independent Samples T-test. The respondents rated the Mulan interactive touch table higher than the Mulan panel display in the following aspects: 1. 2. 3. 4.

The Mulan interactive touch table gained more attention from visitors compared to the Mulan panel display. Visitors can talk more about the Mulan interactive touch table compared to the Mulan panel display. The Mulan interactive touch table is easier to do as compared to understanding the Mulan panel display. The Mulan interactive touch table is more “beautiful” as compared to the Mulan panel display.

In terms of inducing the respondents to “want to learn”, age was a significant factor in the Mulan panel display (98 [F(4, 95) = 3.54, p < .05]). Those between 8 to 12 were more likely to want to learn more about Mulan than all other groups. In terms of ‘user friendliness’, the Mulan panel display was less easy to understand for 4-5 years respondents (M = 3.50, SD = 1.21) as compared to 9-10 years respondents (M = 4.52, SD = 1.05). Hence, the younger respondents have more difficulty engaging with the Mulan panel display. For the Mulan interactive touch table, there were differences among the different age groups in terms of the perceived ‘educational value’ [F(4, 106) = 3.23, p < .05] only. In this respect, the interactive touch table ‘taught’ the 4-5 years respondents (M = 3.78, SD = 1.22) about Chinese culture more than the 6-8 years respondents (M = 4.42, SD = 0.87). In other words, the 4-5 year olds benefitted more from the interactive display than the older age groups.




Implications from the field study

The analysis showed that over all, the respondents preferred the Mulan interactive touch table more than the Mulan panel display and they received and learnt more information about the Chinese heroine, Mulan through the interactive puzzle and story. The Mulan interactive touch table attracted more attention from visitors, made visitors talk more about it, is more accessible (especially to the under 5 group) and was considered to be aesthetically more pleasing than the Mulan panel display. On the whole, the Mulan interactive touch table proved to be more engaging for the respondents than the Mulan panel display and close to three-quarter of the interactive touch table respondents rated their enjoyment level as 100%. In comparison to the Mulan panel display, twice as many respondents rated the interactive touch table as ‘fun’. The responses to the question ‘What is the story of Hua Mulan? point towards interesting implications about the two different displays. Although all the answers were right since Hua Mu Lan is very brave, very filial, very clever and a good warrior, the actual moral value of Mulan character is that of filial piety, respect and care for parents and patriotism. Despite this, the majority of the respondents for the panel display chose“Hua Mu Lan is very brave” as their answer whereas the majority of the interactive touch table respondents chose “Hua Mu Lan is very filial” as their answer. These answers could be partly related to the information received from the exhibit shown. For the Mulan panel display, Mulan’s warrior armour was displayed and thus the story about Hua Mu Lan being brave maps on well with the exhibit seen. This is despite the fact that the story of Mulan’s filial piety was displayed alongside the exhibit. For the Mulan interactive touch table, the values of filial piety is reflected in the design of the character and the story about Mulan going to war in place of her father conveyed the message of Mulan being filial and thus, most respondents selected this as their answer. Though the interactive table was able to transmit the core value espoused in the story of Hua Mu Lan more effectively, its ability to create more language awareness was not as conclusive. While respondents who interviewed for the interactive puzzle were more likely to recognize the characters for Hua Mulan (花木兰) and filial piety (孝), the difference was not significant. Most interestingly, the


non Chinese group did not show an advantage in recognizing Chinese script from the repeated exposure to the script in the puzzle. One possible explanation is that there were too many stimulating input in the puzzle (music, Chinese script, English translation, romanised Chinese, picture frames, story line etc.) making it difficult for the respondents to focus on a specific cue. The other interesting finding is the general difference in responses to the two exhibits. The majority of the panel display respondents liked the pictures whereas majority of the interactive touch table respondents like the fact that the interactive table is ‘fun’. Not surprisingly, the response to the Panel Display is more visual. In fact, there is some evidence that the visual aspect dominates the children’s experience of the exhibit while the response to the interactive table is more holistic with a focus on the experience. The question as to which mode promotes higher level of learning and for what type of children cannot be answered by this current evaluation but investigation in this direction could be revealing. It may also be necessary to separate the children into different groups reflecting different learning styles. From the findings of age group comparisons, the Mulan panel display is less easy to understand for the 4-5 years old respondents as compared to the 9-10 years old respondents. However, the interactive touch table taught the 4-5 years respondents more about Chinese culture than the older respondents. The current study indicates that children below 5 years of age may have the most to gain from interactive settings whereas this may not make so much of a difference with older age groups. This underscores the need of an exhibit different from the traditional ones to cater for young visitors. From this study, we get a sense of the public’s, especially children’s, learning needs in acquiring information and knowledge in a museum setting. With the help of new technologies and new interactive interfaces, more interactive forms of learning can be created to gain the interest of young visitors and bring them closer to culture. There are great potentials for museums to be an appealing place for young ones to learn about both tangible and intangible forms of cultural heritage in a fun and interactive manner.


The future

In this study we explored the didactic value of an interactive table in transmitting language as well as mythology and cultural values. While the Asian Heroes interactive puzzle did not show a superior effect on recognition and recall of language over the panel display, it was able to transmit the core values of the Mulan story more successfully. In this specific study, Mandarin Chinese is not an endangered language and the story of Hua Mulan is also well-documented but the concept can be transferred to a situation where language survival is an issue, especially in languages without a literary tradition. In such cases, the oral tradition is the only vehicle for the transmission of cultural knowledge. For example, myth that would lend itself very well to this interactive design is the local Singaporean story of the banana spirit. Among the Peranakan Malay speakers (an endangered language), this myth embodies the importance of keeping your promise even if it is made to the spiritual world. In this story, a gambler asked the spirit of a banana tree for lottery numbers and in return, he promised to offer himself to her as a groom. He made the spirit suffer by preventing the tree from fruiting. The spirit finally gave in and gave the gambler the winning number. In his excitement over the winnings, the gambler forgot about his promise to the banana spirit who turned vengeful and came to claim her groom. All cultures have an oral tradition. Some are written now, some are not. However, there may be several stories, myth and folk tales, such as the ‘banana spirit story’ that are orally transmitted. Losing these oral traditions means losing one possibility of understanding the ethos that bind the community together. ‘Filial piety’ sounds almost ‘alien’ translated into English but every Chinese person knows the importance of “filial piety” or “xiao4” 孝 “respect and care for your parents”. This is an integral concept that is woven into the social fabric and the best way to transmit such values is to tell stories where heroes with such traits are valorized and conversely, villains without similar traits are punished. Naturally, it follows that losing the language will deny access to these oral traditions and expressions. So, the question remains as to how central is safeguarding language to ICH? Though we agree with Kirshenblatt Gimblett (2004) that it is reductive to see the solution facing ICH archiving as a simple record of knowledge can never equate a repertoire of knowledge that is anchored in a dynamic social reality. Lupo (2007) is absolutely right to say that the concerns of organizing and classification of ICH


must be followed by transmission and fruition. More importantly, Lupo and other researchers (Kurin 2007, Aikawa-Faure 2009, Munjeri 2009, Blake 2009) all emphasise the importance of the the participatory and collaborative roles of the community. This call for more involvement of ‘tradition bearers’ is now a given in ICH and indeed for many linguists who work on documentation. However, the exact mechanism of how this can be done is still not clear. Other researchers (e.g. Blake 2009) has button-hole the term ‘safeguarding’ to mean more than protection from direct threats but to include actions that will translate into continuing viability. We believe that this is a challenge that still faces researchers and practitioners working on ICH. Finally, as pointed out by Giaccardi (2011), technologies are changing how we share experiences and values in fundamental ways. By capturing and archiving such activities as ICH, we actively change the structure of our collective memories which in turn will affect what we view as heritage. If, as argued by practitioners of ICH, heritage is a “living practice”, then the process of safeguarding ICH itself is a practice which will have an impact on the heritage itself. With the growth of social media, we see rising participation in how we shape the transmission of what we define or identify as ICH2. This platform offers researchers and especially communities a dynamic context to work in. Ironically, while this aspect of globalization is often blamed for rapid erosion of ICH, it can also provide the tools for revitalization – a critical activity in the next few decades. Social media may also be the gateway for more participation from the communities concerned.

Acknowledgements 1. The project was funded by the Institute for Media Innovation (IMI) and co-funded by the School of Art, Design and Media (ADM). 2. We are indebted to the staff of the Asian Civilisations Museum for their collaboration and their input at various point in the project. 3. To the team of student research assistants from the School of Art, Design and Media and the Division of Linguistics and Multilingual Studies, we cannot thank them enough. Without their tireles work, this project would not have seen fruition. The students are: Benjamin Cheah, Dennice Juwono Soesanto, Huang Yanying, Fung Kwok Pan, Ho Danyuan, Eric Lai, Nageswaran Naganandhini, Velda Khoo and Marliana Bte Mohamed Aron (project officer)

References “The Linguists” (2008), a documentary directed by Kramer Seth,. Miller, Daniel A., Newberger, Jeremy, Ironbound Films, USA. Aikawa-Faure, Noriko (2009) From the Proclamation of Masterpieces to the Convention for the Safeguarding of Intangible Cultural Heritage. In L. Smith and N. Akagawa (eds.) Intangible Heritage. London: Routledge. Austin, Peter K. (ed.) (2002, 2004) Language Documentation and Description. The Hans Rausing Endangered Languages Project, SOAS, London. (Vols. 1 and 2). Blake, Janet (2009) UNESCO’s 2003 Convention on Intangible Cultural Heritage: the implications of community involvement in ‘safeguarding’. In L. Smith and N. Akagawa (eds.) Intangible Heritage. London: Routledge. Bouchenaki, Mounir. (2004). The interdependency of the tangible and intangible cultural heri- tage. Paper presented at the Place-memory-meaning: preserving intangible values in monuments and sites, ICOMOS 14th General Assembly and Scien- tific Symposium, Victoria Falls.


A colleague working on an endangered Papuan language was able to post his question related to a grammatical point he is working on facebook which reached the cell phone of a teenager deep in the Papuan jungle. This boy posted his answers and comments on facebook. This level of community participation in documentation is astounding and unheard of a couple of years ago (Frantisek Kratochvil, pc).


Brenzinger, Matthias (ed.) 1992 Language Death. Factual and Theoretical Explorations with Special Reference to East Africa. Berlin, New York: Mouton de Gruyter. De Graaf, Tjeerd 2004 The status of endangered languages in the border areas of Japan and Russia, in A. Argenter and R. McKenna Brown (eds.) On the Margins of Nations: Endangered languages and linguistic rights. Proceedings of the Eight Conference of the Foundation for Endangered Languages, Barcelona, 1-3 October 2004. Pp. 153-159. Dorian, Nancy C. (ed.) 1989 Investigating Obsolescence: Studies in Language Contraction and Death. Cambridge: Cambridge University Press. Evans, Nick (2010)Dying Words: Endangered Languages and What They Have to Tell Us. Oxford: Blackwell. Giaccardi, Elisa. (2011). Things we value. Interactions , 18 (1). Grenoble, Lenore A. and Lindsay J. Whaley 2006 Saving Languages. An Introduction to Language Revitalization. Cambridge: Cambridge University Press. Hoekstra Daan. (2010) Fresco: intangible heritage as a key to unlocking the links between the conservation of biological and cultural diversity in Alamos. International Journal of Intangible Heritage, Volume 5. Ito, Nobuo. (2003). Intangible cultural heritage involved in tangible cultural heritage, in ICOMOS International Scientific Symposium “Place, memory, meaning: preserving intangible values in monuments and sites”, Sub Theme A, Section A3, Victoria Falls, Zimbabwe. Retrieved 8 April 2005 from ICOMOS online database. Kalay, Yehuda (2008) Introduction: Preserving cultural heritage through digital media. In Y. Kalay, T. Kvan and J. Affleck (eds.) New Heritage: New Media and Cultural Heritage. London: Routledge Kalay, Yehuda; Kvan Thomas; and Affleck, Janice eds.(2008) .) New Heritage: New Media and Cultural Heritage. London: Routledge. Kirshenblatt Gimblett, B. (2004) Intangible heritage as metacultural production. Museum International, 56 (1-2), 52-64. Kurin, Richard. (2004), Safeguarding Intangible Cultural Heritage in the 2003 UNESCO Convention: a critical appraisal. Museum International, 56: 66–77. Lupo, E. (2007). Intangible Cultural Heritage Valorisation: A new field for design research and practice. International Association of Societied of Design Research. Matsumura, Kazuto (ed) 1998 Studies in Endangered Languages. Papers from the international symposium on endangered languages, Tokyo, November 18-20, 1995. Tokyo: Hituzi Syobo. Munjeri Dawson. 2004. Tangible and intangible heritage: from difference to convergence. Museum Int. 56:12–20 Munjeri, Dawson (2009) Following the length and breath of the roots: some dimensions of intangible heritage. In L. Smith and N. Akagawa (eds.) Intangible Heritage. London: Routledge. Ng, Bee Chin, Stulemejer, Roeland. and Linborg, PerMagnus, (2008) Singapore Voices: A sound installation of languages. Ostler, Nicholas 2005 Empires of the Word. A Language History of the World. London: HarperCollins Publishers. Smith, Lauranjane, and Akagawa, Natsuko (2009) Introduction. In L. Smith and N. Akagawa (eds.) Intangible Heritage. London: Routledge.


UNESCO (2003) Intangible Heritage., retrieved on 18th April 2011. UNESCO (2003) Proclamation of Masterpieces Site, . retrieved on 18th April 2011. UNESCO (2003) Text of the Convention for the Safeguarding of Intangible Cultural Heritage., retrieved on 18th April 2011. UNESCO (2003) Text of the Convention for the Safeguarding of Intangible Cultural Heritage., retrieved on 18th April 2011. UNESCO (2010) Intangible Heritage Lists. retrieved on 18th April 2011. UNESCO (2010) Oral traditions and expressions including language as a vehicle of the intangible cultural heritage retrieved on 18th April 2011. UNESCO (2010) What is Intangible Heritage?, retrieved on 18th April 2011.

Appendices Appendix 1:

Article 2 – Definitions

For the purposes of this Convention, 1. The “intangible cultural heritage” means the practices, representations, expressions, knowledge, skills – as well as the instruments, objects, artefacts and cultural spaces associated therewith – that communities, groups and, in some cases, individuals recognize as part of their cultural heritage. This intangible cultural heritage, transmitted from generation to generation, is constantly recreated by communities and groups in response to their environment, their interaction with nature and their history, and provides them with a sense of identity and continuity, thus promoting respect for cultural diversity and human creativity. For the purposes of this Convention, consideration will be given solely to such intangible cultural heritage as is compatible with existing international human rights instruments, as well as with the requirements of mutual respect among communities, groups and individuals, and of sustainable development. 2. The “intangible cultural heritage”, as defined in paragraph 1 above, is manifested inter alia in the following domains: (a) oral traditions and expressions, including language as a vehicle of the intangible cultural heritage; (b) performing arts; (c) social practices, rituals and festive events; (d) knowledge and practices concerning nature and the universe; (e) traditional craftsmanship. 3. “Safeguarding” means measures aimed at ensuring the viability of the intangible cultural heritage, including the identification, documentation, research, preservation, protection, promotion, enhancement, transmission, particularly through formal and non-formal education, as well as the revitalization of the various aspects of such heritage. 4. “States Parties” means States which are bound by this Convention and among which this Convention is in force.


5. This Convention applies mutatis mutandis to the territories referred to in Article 33 which become Parties to this Convention in accordance with the conditions set out in that Article. To that extent the expression “States Parties” also refers to such territories.

Appendix 2: Mulan panel display questionnaire (2 pages)



Appendix 3: Mulan interactive touch table questionnaire (2 pages)



Re-Positioning Exhibits in Time and Space Florian Windhager & Eva Mayr Research Center KnowComm, Danube University Krems (Austria) [email protected] , [email protected] Abstract Technologies empower museums to create enriched experiences for their visitors. But they do not necessarily do so. To attain this effect, we suggest to study the visitors’ behaviours and needs first and identify ways how their visit can be advanced – emotionally, socially or intellectually. Often, visitors experience cognitive overload and museum fatigue due to the high amount of information available. Transparent and clear organizing principles can help visitors to cope with the amount of information available and build up a more coherent mental model. But the underlying conceptual structures, like common topics, themes, or geographical origin as well as temporal structures are often not transparent to the visitors. We introduce an advance organizer that can improve conceptual orientation in an exhibition by making these conceptual and temporal structures transparent to the visitors: A space-time-cube can visually re-position exhibits in temporal and topographical space-times and help visitors to orient themselves in the exhibition’s physical and conceptual space. Thereby, a new interactive way of exhibition exploration is created and novel visiting experiences emerge.

1 Introduction Selma Thomas (2007, p. 164) states “that media has transformed the museum as well as the visitor”. But in what way? Is it always intended? Are there positive effects only or negative ones, too? And are we forced to react on these changes only or can we actively steer these new developments to form a better experience for our visitors? Currently, many museums introduce new media applications to their exhibitions, but often it seems that their purpose is mainly to attract new audiences, entertain visitors and polish an old-fashioned image. “They fall short of meeting the intellectual goals set by an exhibition team” (Thomas, 2007, p. 164). An important challenge for media in museums is that they actually support the visiting experience and learning processes (Knipfer, Mayr, Zahn, Schwan, & Hesse, 2009; Wessel & Mayr, 2007). However, most advanced technologies in museums have not been designed to support cognitive processes or knowledge communication explicitly (Knipfer, Mayr et al., 2009). In this paper, we suggest to put the cart before the horse: We think about what kind of visiting experience we want to emerge by introducing conceptual orientation first and in a second step we develop a technological solution suited to meet this requirement.

2 The Exhibition Experience Results from audience research provide a rich ressource to build on, if we want to gain a clearer picture on the kind of experiences visitors have. This knowledge helps us to develop ideas how their experiences could be further improved.

2.1 Museums as Informal Learning Settings Despite its educational mission, the museum is not a pure learning but rather a leisure setting. For many museum visitors the focus of their activity is not on learning. Visitors’ motivations rather include social outings, self-fulfilment, entertainment, or recreation (e.g., Black, 2005). Still, learning is an important part of visiting a museum for most people (Packer, 2006) and the most important part of a museum’s mission for society. Therefore, technology should also aim at improving the intellectual visiting experience.

2.2 Information Overload Exhibitions present vast amounts of exhibits and information that are assembled around various topics and put up high challenges on individual cognition: Visitors see lots of exhibits and associated


information (texts and images). Which information they attend to, depends on different factors like attraction power, personal interest, social cues, etc. Bitgood (2006) assumes that visitors’ circulation and attention depends on a cost-benefit equation, that is, visitors attend towards exhibits they perceive as beneficial (interesting, enjoyable, ...) when the effort of attending towards it is low (time, distance, ...). Just circulating an exhibition does not convey any cultural knowledge. To learn something, visitors have to select relevant information, evaluate this information, integrate it with the other information present and elaborate on it based on their prior knowledge. These cognitive processes happen under aggravating conditions of restricted space, limited visiting time and finite attention spans. As a ubiquitous result museum fatigue shows up: Perceptive and cognitive overload lead to a strong reduction of visitors’ attention towards exhibits, learning motivation and receptivity (Davey, 2005). As this is the basic threat for every exhibition designer and curator, several approaches have been fielded to tackle this issue (storylines, exhibit clusters, interaction, etc.); one of them, which we want to focus on, is the use of an advance organizer.

2.3 Conceptual Orientation by Means of an Advance Organizer Exhibitions want to communicate a storyline and a main message in an effective manner to the visitors. Different exhibition design techniques can support this aim, e.g. conceptual arrangement of similar exhibits (Falk, 1997) supports the visitor in constructing a more coherent mental model about the exhibition. In educational psychology, Ausubel (1960) suggested the use of advance organizers as effective means to improve the construction of mental models: Learners are provided a conceptual structure before a learning unit in which they can integrate the learning material later on. Advance organizers were shown to be especially effective in non-linear, unstructured learning environments (McManus, 2000) and, therefore, have high potential for museums as well. Visitor research shows that advance organizers in the form of texts for conceptual orientation before a museum visit can support informal learning, too (Falk & Storcksdieck, 2005; Patterson & Bitgood, 1987).

3 Designing Technology to Serve as Advance Organizer Exhibits for conceptual orientation are usually included in each exhibition nowadays. However, this information is often presented only textually and cannot compete against the exhibits’ attraction power; consequently it receives only little attention from the visitors (cp. Griggs, 1983). In contrast, new media applications often receive high attention. Therefore, we want to present an application for a visually oriented advance organizer, which at the same time does not compete against the exhibits, but rather refers and points out to them.

3.1 Time Geography The method of time geography was introduced by Hägerstrand (1970) to visualize temporal processes in geographical space, i.e. to visually analyze the spatial dynamics of various entities over time. To achieve this purpose, two classical information visualization methods became conceptually intertwined: cartography as method to map arrangements on geographic surfaces and chronography as method to map arrangements of events along timelines. The resulting basic figures are so called space-time-cubes: Geographic maps, which serve as horizontal (two-dimensional) layers, get orthogonally intersected with timelines, so that three-dimensional cubes are unfolding, where the vertical dimension is open to envision space-time (see Figure 1).


Fig 1: The combination of geographical and chronographical visualization as space-timevisualization.


As such, the resulting space-time-cubes (or space-time-aquariums) are applicative for an integrated visualization method, where geographically and chronologically structured information can visually merge. Depending on the specific scaling of the space or time dimension, space-time-cubes can cover only small sections of space time – such as short regional chains of events – or at the maximum scale they can cover space as world history (see Figure 2).


Wor history minutes hours days months years centuries millennia ld Hist Fig 2: The increasing scaling of spaces and times towards the maximum space time cube of world ory history Within this freely scalable framework, the position of any physical entity can be located and visualized over various time phases (Kraak, 2003). The resulting patterns with regard to a single object are spacetime-trajectories visualizing movements as characteristic traces or tracks. While a non-moving object produces a vertical trajectory, moving objects plot curved paths into the cube, which envision the characteristic space-time behaviour of historical entities – and which could be analytically resolved into more basic visual elements. Whereas time-geography originally aims at the visualization of human life paths, this method can also be applied on artefacts which are exhibited in museums, such as weapons, bones, artworks, etc. Relevant basic elements of their space-time-paths for example encompass the space-time-points of their origin (1) (in case of complex artefacts also the phase of its assembly), their time-geographical movements (2), relevant events (3), including: possibly known effects (4) and in case of losses their point of disappearance and retrieval or reconstruction (5) (see Figure 3).






* *

1755 1) assemblage & 2) time-geographi- 3) involvement 4) causal 5) points of dispoint of origin effects & re-appearance cal movements into events possible visual segmentation of the space time trajectory of a given artefact.

entire trajectory

Fig 3: The

Apart from these options of fine-grained time-geographic imaging – that naturally depend on rich historical artefact data – the most basic coordinates often are time and location of exhibit origin. Given any historically structured exhibition, these artefact data points now can be re-positioned in time and space. The result will be a characteristically distributed point cloud in a characteristically scaled spacetime-cube (see Figure 4, left hand side), which can be further explored and visually analyzed (see chapter 3.3). The effect of such re-positioning procedures for the visitor’s intellectual experience corresponds to the described effects of advance organizers. Before entering the real exhibition spaces, with all its informational diversity, a coherent exhibition model of reduced complexity could be offered. As such, it delivers a well defined overview first and can serve as cognitive schema or skeleton, which is capable to grow and take up the flesh of more detailed information, when zooming into the exhibition by individual walkthroughs.


3.2 Time Topography Before turning towards possible methods of interaction with a space-time-cube, a generalizing extension of the framework as outlined so far has to be ensured. Geographical mapping is not the only method which could help to re-position exhibits in relevant conceptual contexts; various extensions of the time-geographical framework are possible. Aligning with the given scaffold of the space-time-cube, these extensions use the vertical time axis as well, but replace the method of horizontal geographic mapping by other information visualization techniques. For instance by turning to fairly common exhibition topics like natural history, history of arts, or history of nations, the geographic space-time-cube, as conceptualized above, can display valuable exhibit information. Still it cannot be considered to deliver the most appropriate mapping method, when the evolution of whole populations, art forms or dynasties has to be visualized. While the chronological distribution still plays a decisive role, the topographical distribution could be better visualized within a conceptual space of diversity. Now, instead of geographic maps, maps of association, affiliation or social closeness have to figure as ground layer. The evolution of life, fine arts, or any given topic can now be visualized as branching tree, which unfolds over specifically structured topic maps (see Figure 4, right hand side).


Fig 4: Point cloud of exhibit origins in a geographic space-time-cube (left hand side); Visualization of exhibit evolution trees in a topographical space-time-cube (right hand side). As information visualization methods like social, semantic or multi-topographical network analysis have seen significant methodical and technological advancements in recent times (Krempel, 2005; Diesner & Carley, 2010; Windhager, Zenk & Risku, 2010), they can be leveraged and coherently integrated into an extended framework of time topography. With this extended framework, which allows exhibition designers (and visitors) to select various mapping methods to re-position their exhibits in conceptually illuminating space-times, we consider to present an innovative solution to visually structure and organize theoretically unrestricted types of exhibition data.

3.3 Visual Analytics for Interactive Exhibition Exploration Visual Analytics is the science of analytical reasoning facilitated by interactive visual interfaces and appropriate visualisation techniques (Thomas & Cook, 2005). Here, visual and analytical methods are intertwined to support the exploration of data and discovery processes. Within this framework, the user is not merely a passive element who interprets given visualisations and installations, but is instead the core entity who drives the process of analyzing data and interacting with them.


Visual analytics methods are especially suited for application with large data sets to support users in gaining an overview on and exploring these data. An exhibition-based space-time-cube would necessarily include different structuring properties (i.e. horizontal layers, temporal frames, exhibition parts, ...) and, therefore, requires intuitive and easy interfaces for museum visitors. By applying visual analytics methods to an exhibition-space-time-cube visitors are enabled to interactively orient themselves in the exhibitions’ geographical and topographical space-time. They can navigate through these spaces, zoom in and out, filter, select exhibits, retrieve further information, discover connections between different exhibits and build up a cognitive framework which later helps them to integrate the exhibits and the information at the museum in their enacted knowledge structure.

3.4 Interlinking Time Topography and Exhibition Topography Maybe the most demanding challenge, is the task to offer and establish an interlinkage between (the arrangement of) the time-topographical exhibition space and (the arrangement of) the real exhibition space. Time-topographical visualizations allow a translation and re-positioning of given exhibits in meaningful space-time arrangements, but have to be re-translatable into the real (spatial exhibition) arrangements again, to achieve best enduring effects. Thus a public terminal in the entrance area, which allows for time-topographical exploration of the exhibition, has to ensure the link back to the real spatial topography of the exhibits arranged on the exhibition site, where all the details will be discovered on the move. Technically spoken, to interlink the exhibition with the space-time-cube in a one-sided manner, a Linked View should be deployed between the time-topographical exhibition model and an architectural model of the exhibition layout. Linked views offer (at least two) different views on an identical set of elements by varying visual layouts or rules of arrangement. If these views are furthermore set up for interactive exploration (user-driven methods of linking and brushing), the selection of an element in the first view causes the highlighting of the element in the other view. This allows the cognitive translation of arrangements from one layout to the other – and possible re-translation (or multiple cyclic elaborations) in terms of an enriched return to the former way of viewing. As shown in Figure 5, this visual interlinkage of exhibit layouts can allow for the formation of a “bilingual” (“bispatial”) mental model, which can guide and enrich visitors’ experience in the timetopographical and the real-topographical framework. Thus it can help to navigate in the face of cognitive and factual exhibition walkthroughs. As the outlined approach of a visual “exhibit positioning system” could be implemented on various media devices and on various levels of pervasiveness or richness, the following chapter will map out the various options and stages of technical realization.

time-topographical repositioning exhibit G51 exhibit G51

architectural re-positioning

Fig 5: Time-topographical exhibition layout and architectural exhibition layout as linked views.


3.5 Technical Implementation Technical implementations of time-topographical concepts could be enacted on at least two different levels, which would correspond to different media realms: (1) Computer-based visualizations in the museum’s entrance hall allow the introduction of the space-time-cube as advance organizer, whereas (2) additional printed maps or space-based exhibition design can ensure the coherent utilisation of the time-topographical model for the visitors’ experiences of the real exhibit arrangements in the given architectural museum spaces. Whereas the first dimension also could be realized as a supplementing post-hoc measure in every given museum by itself, only a synchronous and narrow intertwining of conceptual information design and physical exhibition design is considered to allow for the best possible development of a coherent mental exhibition model, which again is considered as a precondition for effective cognitive processing in terms of information evaluation, elaboration, and integration. 3.5.1 Time-Topography as Advance Organizer Time-topographical visualization applications can be implemented as interactive advance organizer on large public screens (e.g., in the entrance area), on individual PC terminals (for the purpose of individual exploration), or on websites (for the public communication of science). As various tools already allow for the creation of interactive, time-topographical exhibition visualizations (e.g., Kapler & Wright, 2005), the major challenge for curators and exhibition designers will be data migration and integration into advance organizers. 3.5.2 Coherent Exhibition Design A coherent exhibition design aims on the alignment of the structural and sequential exhibition configuration with the conceptual orientation provided on the advance organizer. If the concepts and structures of the advance organizer are available during the visit continuously and stringently, visitors can more easily integrate the exhibits and the information into the mental model, which they built up in advance. Time-topographical conceptualizations are already included in most coherent exhibition designs. They can be made more explicit and easy to decode by additionally including guiding systems (e.g., signboards and overview maps analogous to the ones used on the advance organizer), showroom design (e.g. colour coding, red threads), or detailed single exhibit information referencing back to the advance organizer. Alternatively (or additionally) the advance organizer can be made accessible on mobile devices too. This would allow visitors to continue to navigate and orient themselves in the exhibition’s conceptual, temporal, and geographical space, make connections to exhibits in other parts of the exhibitions. In addition, visitors could bookmark exhibits and information of personal relevance and interest, annotate thoughts, and take home this information for extended exploration.

4 Conclusion & Outlook Due to its conceptual nature, we consider this article to lay ground to the development of a conceptual and technological framework which can support visitors’ experience in museums with regard to general information integration, informal learning, as well as the relocation of exhibits in space and time. Though only concrete implementations and evaluations will show the extent, to which these effects are achieved, we want to emphasize the relevance of conceptual elaboration as precondition to the successful implementation of various technological measures. As outlined above, the time topographical framework has to be envisioned as freely scalable and adaptable scaffold, which is open to support cognition on any historically structured subject-matter and which can be technologically implemented using a great variety of exhibition design approaches. As long as the basic combination of spaces and time visualizations are implemented by the use of various information design technologies, we are sure that new kinds of coherent visitor experiences will emerge:


The main effect of the space-time-cube is its function as advance organizer. It allows visitors to build up a conceptual framework, in which to integrate the exhibits and the information presented in the exhibition. Thereby, they can more easily orient themselves in the exhibition’s conceptual space, can more effectively evaluate the exhibit information based on the conceptual framework, and finally will acquire more enduring knowledge. Also, the exhibition design principles are made more explicit. Visitors are encouraged to decode the conceptual orientation of exhibits in the museum space, while they integrate the exhibit information into the framework built up by using the advance organizer. In addition, this technology can raise expectations and interests before the visit. Interested visitors are more motivated to attend to the exhibition more closely and more processing capacities will be freed. By situating the advance organizer in the entrance hall, it has different effects: First, it will not directly compete with the exhibits, but rather directs the visitors to them by providing conceptual orientation, as well as raising expectations and curiosity. Secondly, such a technological 3D-installation has high attraction power and motivates visitors to explore it. Thereby, it overcomes the shortcomings of textual advance organizers. As a public terminal it additionally can be explored in groups, allowing for discussion of concepts and conversational elaboration. Last but not least, the space-time-cube opens up a flexible view on exhibitions: Curators are normally restricted to organize the exhibits using one or few organizing principles (e.g. time). In the space-timecube in contrast, exhibits can be organized in their whole complexity, the organizing principles can be interactively changed, and allow visitors to get an idea on the multiple connections and possible conceptual re-positioning of exhibits.

References Ausubel, D. P. (1960), “The use of advance organizers in the learning and retention of meaningful verbal material”, Journal of Educational Psychology, 51:267-272. Bitgood, S. (2006), “An analysis of visitor circulation: Movement patterns and the general value principle”, Curator, 49(4):463-475. Black, G. (2005), The engaging museum, London: Routledge. Davey, G. (2005), “What is museum fatigue?”, Visitor Studies Today, 8(3):17-21. Diesner, J. and Carley, K. (2010), “Relation Extraction from Texts”, in Stegbauer, C. & Häussling, R. (eds), Handbook Network Research (pp. 507-521), Wiesbaden, Germany: VS Verlag für Sozialwissenschaften. Falk, J. (1997), “Testing a museum exhibition design assumption: The effect of explicit labeling of exhibit clusters on visitor concept development”, Science Education, 81:679-687. Falk, J. H. and Storksdieck, M. (2005), “Using the Contextual Model of Learning to understand visitor learning from a science center exhibition”, Science Education. 89:744- 778. Griggs, S. A. (1983), “Orientating visitors within a thematic display”, International Journal of Museum Management and Curatorship, 2:119-134. Hägerstrand, T. (1970), “What about people in regional science?”, Papers Regional Science Association, 24:7-21. Kapler, T., and Wright, W. (2005), “GeoTime Information Visualization”, Information Visualization 4:136-146. Knipfer, K., Mayr, E., Zahn, C., Schwan, S., and Hesse, F. W. (2009), “Computer support for knowledge communication in science exhibitions: Novel perspectives from research on collaborative learning”, Educational Research Review, 4:196-209. Kraak, M.-J. (2003), “The Space-Time Cube revisited from a Geovisualization perspective”, ICC 2003: Proceedings of the 21st International Cartographic Conference: Cartographic Renaissance (pp. 19881996).


Krempel, L. (2005), Visualisierung komplexer Strukturen. mehrdimensionaler Netzwerke, Frankfurt/Main: Campus.




McManus, T. F. (2000), “Individualizing instruction in a web-based hypermedia learning environment: Nonlinearity, Advance Organizers, and self-regulated learners”, Journal of Interactive Learning Research, 11:219-251. Packer, J. (2006), “Learning for fun: The unique contribution of educational leisure experiences”, Curator, 48:329-344. Patterson, D. and Bitgood, S. (1987), “Principles of orientation and circulation”, Visitor Behavior, 1(4):4. Thomas J. J. and Cook, K. A. (2005), Illuminating the Path: The Research and Development Agenda for Visual Analytics, IEEE Computer Science. Wessel, D. and Mayr, E. (2007), “Potentials & challenges of mobile media in museums”, International Journal of Interactive Mobile Technologies, 1(1):32-39. Windhager, F., Zenk, L. and Risku, H. (2010), “Situated organizational mapping”, in Stegbauer, C. (Ed), Netzwerkanalyse und Netzwerktheorie (2nd ed.) (pp. 239-249). Wiesbaden, Germany: VS Verlag für Sozialwissenschaften.


Increasing the Usability of the Museum: Four Studies Pille Pruulmann-Vengerfeldt University of Tartu [email protected]

Pille Runnel Estonian National Museum/University of Tartu [email protected] Abstract This article looks at the changing relations between audiences and museum institutions through the case of the Estonian National Museum. We also bring in the notion of usability as conceptualised by Jakob Nielsen in order to add ICT and marketing related knowledge to the discussion. We argue that in order to activate participation in heritage institutions, publics should not be left to find their own way around, but rather heritage institutions need to make a conscious attempt to create spaces of participation that are user friendly and engaging. This often means combining both online and offline tools and approaches.

1 Introduction This article presents some of the work done in two grant projects – Developing Museum Communication in the 21st Century, and The Problems of Transformation and Reception of Cultural Heritage in the Digital Age, which have both tried to look at the changing relations between audiences and heritage institutions. We will use examples from the Estonian National Museum (ENM) as the basis of our analysis. In this article we will support our argument with empirical case studies and data collected using several methods at different points in time. Rather than being a project report, we hope this will be a more reflexive overview of audience relations in museum institutions and participation. In using the notions of audiences and users interchangeably, we assume that audiences are active despite the communication channels used. In increasing use of ICT solutions in heritage institutions, audiences are placed in even more active roles as users or produsers, as used by Bruns (2006). Benefiting from some of those active users as content providers and active participants in the museum context is a challenge. Coming from a media studies perspective, we approach the publics in the heritage institutions from this angle. In addition, when bringing in notions of usability as conceptualised by Jakob Nielsen in order to add ICT and marketing related knowledge, we try to focus this paper’s analysis around understanding how improved usability can support user generated content and participation in museums. By doing this, we try to place this article on the crossroads of different disciplines, as these add value to the practical applications we are working on. We argue that in order to activate participation in heritage institutions, publics should not be left to find their way around, but rather heritage institutions need to make a conscious attempt to create spaces of participation that are user friendly and engaging.

2 Memory Institution Choices for User Generated Content Participation and active audiences are not entirely new phenomena in the context of heritage institutions (see for instance Carpentier, 2010). Many museums have built their collections using objects and information collected from the people. Historically our home institution, the Estonian National Museum, has regarded the public not only as audiences of their exhibitions and customers of services and consumer products, but individuals and groups have also been approached as subjects of study and as information providers and knowledge co-creators for the collections (related to the research approach of ethnology and folkloristics). The ENM is largely based on collections and archives about folk culture, communities, everyday life. A remarkable archive has been formed on the basis of written contributions and donations from the museum’s network of correspondents (formed more than 70 years ago). However, it can be said that in all these participatory activities, the heritage worker has played the role of the gatekeeper, moderating and limiting the participation for particular purposes. The ENM, similarly to other museums, has been the legitimate producer and guardian of


heritage and social memory and through these roles the museum has also been helper and teacher in developing a sense of collective identity and citizenship. Today, online environments seem to provide ample opportunities for the public to engage in dialogue with heritage institutions. However, two-way communication assumes not only the existence of the communication channel, but also willing parties who are interested in communication. Despite the promising online opportunities, the continuously hierarchical and traditionalist nature of heritage institutions may be part of why audiences/users are not that keen on participating and contributing. In particular the sense of expert power or the consideration of an abstract ‘them’ who know better than the ‘average man’ can become an obstacle to participation, even in the time of these potentially democracy-facilitating technologies. Thus the obstacle of perceived lack of expertise has to be overcome on both sides. All participating parties need to understand that the ‘average man’ is an expert in his own life and his contribution can be valuable for the museum, even when they are less than perfect by the museum’s standard. When we conducted the interviews with cultural heritage institution workers they summarised their passive attitude towards creating online content by claiming that they are not experts to comment or to say, and that their opinion doesn’t matter and nobody cares (Aljas, Pruulmann-Vengerfeldt, 2009). In a way, museum workers ‘lost’ their expertise when facing online environments and became hesitant in providing their contributions. This also indicates the perceived sense of expertise when considering participation from the general public in their field of expertise. In the past, gathering input from the society and mediating information were tools for the memory institutions to maintain an expert position. That position can now, when technology permits potentially very open and wide participation (and content creation) at low cost, become threatened. “The idea of becoming virtual might not be a pleasant one for some museums, especially not for art museums who cherish the ideal of the ‘real thing’ and its aura. However, this development is inevitable because of the increasing digitisation of cultural heritage and the demand to make collections more accessible. Eventually, these trends will blur the differences between cultural heritage institutions, and in the long run these institutions will merge into one memory institution.” (Schweibenz, 2004). These interlinked memory institutions hailed by Schweibenz above, show that the important user motivation comes from the content itself. National museums and ethnographic collections in particular can claim to be ‘living’ museums where communities connected with that particular museum can add to the collection as they experience everyday life. There have been arguments (Carnall, 2009) which say that memory institutions have had significant obstacles that have stopped them from being online to any great extent. These obstacles have included the genuine fear that people would stop coming to museums if they could access museum collections online (Carnall, 2009). At the same time, the virtual museum can extend the ideas and concepts of collections into the digital space and in this way reveal the essential nature of the museum. We have now briefly mapped different kinds of obstacles and considerations related to the changing roles of the museums and museum audiences in relation to the new technologies and user-generated content online: museums being hesitant to lose control; audiences as potential participators facing the still-present barrier of professional expertise; experts being afraid to lose visitors entirely because the content of their collections has been made available and open. Now we will turn to the case study of the Estonian National Museum, which deals with how the museum faces these threats and finds motivation to use online and on-site technologies to support participation by the general public.

3 - Participatory Actions at the Estonian National Museum As mentioned before, the ENM has a long-term tradition of collecting tangible and intangible heritage. Along with the spread of internet technologies, online communication and collection have required conceptualisation and rethinking of collecting in general. Relying mainly on low-tech applications or interactive screens in exhibition rooms, the ENM has also expanded its presence to Second Life, Twitter and Facebook for the sake of dialogue with its audiences. Online and offline participatory options at the ENM are more geared towards facilitating dialogue with users – asking them to comment on and to add to digital collections as a complex body of knowledge. In the context of this article, we will not discuss the effectiveness of those online social networking tools for the museum. We will focus our analysis on four different examples of activities, which combine both online and offline and which have been designed to facilitate user participation and possibly the breaking of the barrier between museum experts and museum audiences on the level of both collections and exhibitions


(See Table 1 for an overview of the cases). We will analyse participatory actions conducted at the Estonian National Museums in recent years from Jakob Nielsen’s usability perspective, starting firstly with a short introduction of these examples. We have summarised the examples following the inspiration of Mariana Salgado (2009) into an overview table, illustrating the initiatives in chronological order and summarising the online on-site dimensions of each of these initiatives. Our own position in analysing these initiatives is that of auto-ethnography and production ethnography as we are initiators of these initiatives as researchers and as museum workers.

Access point


Give the Museum a Day of Your Life (April 2009)

Exhibition comments with pen and paper (autumn/wint er 2010)

Museum night exhibition tagging (May 2010)

Open curatorship exhibition (2010-2011)

Overall Collect stories aim of everyday life of Estonians on 14th of April

Receive comments and clarifications on photo collection displayed in exhibition

Receive stories and comments on permanent exhibition

Hosted exhibitions invitation to general public to display their own collections or cooperation

Museu Collecting m activity





Advertisement of exhibition, no participation possible


Online submission of exhibition proposals,

Advertisement online, e-mail address and online form as submission sites


Online evaluation of exhibition, proposals by public vote


On site

Postal and handdelivered stories

Post-it notes stuck to favourite pictures worth commenting on

A5 papers on washing line and pegs for commenting on specific aspects of the exhibition

Offline submission of exhibition proposals. Offline display and evaluation of proposals. Two winning exhibitions displayed at the main museum building





28 online proposals

No of comments

509 online votes On site




5 on-site proposals 55 on-site votes

3.1 Collecting – Give the Museum a Day of Your Life At 2009, the Estonian National Museum carried out a campaign called Give the Museum a Day of Your Life, which took place both online and offline, aiming to document everyday life in 2009 and asking people living in Estonia to contribute. People were asked to document the April 15, 2009, which was the hundredth anniversary of the museum. The contributions were included in the collections of the ENM. Contributions to this kind of initiatives need dedication and time from the people. They know that the stories and photos or other visuals became part of the museum’s collections and archives. This adds a sense of value and motivation to the audiences to participate. At the same time, the topic remained simple enough for everyone, as all of us can claim to be an expert in one’s own personal everyday life and everyday practices and thus the barrier to participation was low.

3.2 Exhibition comments with pen and paper At the same year, the ENM held a photographic exhibition called With a Thousand Steps, based on an overview of the museum photographic collections and on a display of every 1000th photograph from its collections. Here the user-generated content experiment was carried out. Visitor participation was made easy – people were given the opportunity to add free-form comments on the presented photos by using post-it notes and pens at the exhibition hall. Motivation for this experiment was provided through a prize draw where participants could expect to receive a particular photograph as a print for their personal use. Eighty percent on the comments contributed were expressions of emotion, such as


Many of the offline stories were created with computers - CD-ROMs with digital images, word-processed texts etc, but as they were delivered using regular postal services or brought to the museum by hand, these are counted as offline contributions.


‘beautiful’, ‘great’, ‘I like’, and/or ‘I would like to have that too’. However, from the heritage institution’s point of view, the more valuable were the remarks that indicated the new knowledge that people got from photos or where exhibited photographs activated new interests or questions. In addition, a few corrections were made to the photograph legends the museum had: “It should be Artur Vasiksaat, because the name Vasikraat doesn’t exist in Muhu” (Example from the post-it notes on the photographic exhibition). This kind of ‘tagging’ experiment indicates that in order to have usergenerated content, there is not necessarily a need for high-tech solutions and expensive software. In 2010, the ENM also used pen-and-paper technologies at Museum Night, at which people were invited to tell their stories about the permanent exhibition (or parts of it) of the ENM. In 4-5 hours, 17 stories were collected, ranging from short exclamations to longer personal recollections.

3.3 Open Curatorship exhibition In order to promote the idea of open curatorship, the ENM called audiences to propose exhibition ideas, giving the winners the opportunity to develop a real exhibition. The open curatorship project was thus participatory on three levels: proposing the ideas, selecting the winner and developing the exhibitions was all done by people who, under usual circumstances, might be the more or less passive visitors to the museum. In order to facilitate proposing the ideas, a set of questions was formed for prospective participants. Altogether 33 ideas were proposed and 569 voters participated in selecting the winning project. The winners were selected in two categories – exhibitions based on artefacts from outside the ENM and exhibitions based on the museum’s collections. Although the museum workers expected to have some unusual approaches or choices of topic, of the sort that do not usually make their way to the exhibition halls, the reality was different: the winning project, based on material coming from outside the museum, was dedicated to the changing funeral traditions in Estonia and the winning project using the museum’s own collections displays copies of ethnic dress and folk art produced by a handicraft cooperation called Uku. People were asked feedback to how this kind of participation in the museum’s activities is similar or different to their usual engagement with the museum.

4 Increasing User Motivation for Participation In his analysis, Nielsen (2006) argues that one cannot change the principle that the majority of the users of a particular online environment will remain passive. However, he argues that there are five key possibilities to have users participating in content creation. Nielsen (2006) argues that these are: 1) making the participation easier, 2) providing the possibility to edit rather than create, 3) promoting quality contributors, 4) making participation a side-effect of visiting the site and 5) rewarding, but not over-rewarding, participants; by doing so one can engage more than the 90:9:1 ratio mentioned above. This provides us with additional clues as to what user motivation might be for participating with the heritage institutions (online) and we are going to discuss the experiments of participation from the usability perspective provided by Nielsen to help understand how these ideas can be put in practice from the heritage institution’s perspective.

4.1 Making participation easier Nielsen’s first key proposal is making participation as easy as possible (Nielsen, 2006), which indicates that the usability aspect of the participation environment will be important. By showing people that contributing is easy, one might end up with contributions that might be less valuable for the heritage institution but would help in making participation a habit. Here the offline example of tagging a photographic exhibition with paper post-it notes is an excellent example. Using low-tech solutions and familiar technologies (paper and pen) the participants did not need to learn new practices and could participate while viewing the exhibition. However, experience from the same exhibition indicates that when participation is made too easy, this could also reduce the quality of the contributions. 80% of post-its provided at the exhibitions focused just on utterances like “beautiful” or “I’d like to have this” alongside the rather carefully considered quality contributions. These posts activated the user, but they do not necessarily support the creation of quality contributions relevant for the museum. Also an open curatorship exhibition introduced public voting as an easy method to participate in museum activities. We made online and paper-based voting questionnaires on which everyone could give marks for either all exhibitions or just highlight their favourites. This form of participation was popular, resulting of 569 people casting their votes.


4.2 Providing the possibility to edit rather than create Secondly, the notion that editing is easier than creating (Nielsen, 2006) was taken into consideration when preparing the Give the Museum a Day of Your Life action. Before the public call for participation museum staff made different attempts to collect their own lives and these stories were provided as examples of different styles of participatory content to help people overcome the complications of starting. The idea that editing existing content is easier than starting from scratch is also visible in Trant (2009). The idea supports Carnall’s (2009) observation that online museums are typically very content-light, which makes it difficult to attach the contributions of the public to specific pre-existing structures. Contribution to the online content has been related to the network effect (Liebowitz and Margolis, 1998), meaning that the resource becomes more valuable when there are other people consuming the same good. Thus internet users expect and like to contribute where others are and where some prior content already exists. The more information and opportunities to link, add, comment on and tag the information heritage institutions have online, the more valuable the resource is for individual users. Hence the heritage institutions, like museums, which have so far been very shy of adding their digital information online in the fear of losing their real visitors (Schweibenz, 2004) have to overcome that fear in order to make users interested in generating content on their sites. Nielsen’s second key statement - that editing is easier - was also visible in the open curatorship exhibition. For the exhibition proposal, a set of questions was formed in order to encourage people to start thinking about an exhibition idea. Of the 33 ideas submitted, the majority follows the proposed pattern, although a number of others resisted the provided form and looked for other ways of giving their own ideas. Those who choose to ignore the set questions usually had some previously formed specific ideas and found it difficult to suppress or extend those pre-existing ideas into an exhibition proposal. Thus, when providing editing or clearly pre-formatted ways of participation, it is worthwhile to consider other options for those for whom such pre-formatting might be too limiting. Finding ways to support participation should avoid generating new barriers. The Museum Night pen-and-paper comments experience shows that exhibition items themselves can become the templates for further editing. Our participants used the things they saw as the basis for their stories, connecting objects with their own experiences. These stories made most sense when seen directly next to the relevant parts of the exhibition. In future, sourcing and displaying such stories may become an inherent part of the new permanent exhibition, while in the context of this particular Museum Night the stories became additional focal points of the exhibition.

4.3 Promoting quality contributors Thirdly, Nielsen’s idea (2006) that promoting quality contributors would foster participation is related to what Marlow, Naaman, boyd and Davis (2006) indicate as sociable interests and motivations of the users. Marlow et al. (2006) lists the following aspects as the social motivations for tagging: communicating and sharing, attracting attention, self-presentation and opinion expression. However, they point out that users might not always consider the social (or even altruistic) motivations as the most important ones. People often start for personal organisational reasons and later move on to the social benefits (Marlow et al., 2006). In addition, digital collections, which make user contributions official parts of the heritage institution’s collections, are a good way of promoting participation. This was also part of the Give the Museum a Day of Your Life initiative as potential contributors were assured that their ‘days’ will be part of the museum collections. The open curatorship exhibition was a participatory action in which the promotion was inherently part of the action. Each of the exhibition proposals was displayed on the website and also on a temporary stand in the museum foyer. The winning exhibitions were selected by the voters on the website and in the museum foyer and winners also received the honour and responsibility of actually displaying the exhibition.

4.4 Making participation a side-effect The fourth aspect of participation is more of a technical feature (Nielsen, 2006) in which the system itself points out that if the user found a certain kind of content relevant, they might also find other information helpful. This feature is not used as part of user-generated content (preference data sourced from users) but is used as an expert-provided linking of the materials in which one digital item can be linked to another through an expert-provided relationship. This way of linking the collections has been


used in the Estonian Literary Museum online project, Kreutzwald’s Century4, in which the user can explore history in a non-linear way, thus creating (though not leaving a record of) her or his own trail through literary history. Here one potential application of user-generated content is to store the trails of the digital content users and provide them as potential pathways to those interested in non-linear narration of the literary history. Here, a great future potential also lies in recommendation systems supporting participatory activities, which do go beyond the model of commercial providers (Amazon) of outlining similarities between products. In the Estonian National Museum, the new permanent exhibition, currently being developed, looks for ways of providing a social recommendation system highlighting relevant marked units from the collections or additions from the users, based upon the material in which a user has already expressed an interest. Such social recommendation systems also have the potential to provide recommendations based on both familiarity/similarity while at the same time enabling conscious comparison and connection of objects, stories or comments that perhaps represent a different kind of perspective and use other people’s recommendations to help in the connections.

4.5 Rewarding, but not over-rewarding participants The fifth consideration for increasing user motivation is rewarding active participation. Here the reward can be monetary or giving away things and this way of rewarding has been used at the Estonian National Museum, where a prize draw was included as motivation for both of the discussed usergenerated content initiatives. However, it did have some drawbacks as the promised prize for the post-it note participation involved a print-out of the chosen photo; the many comments were mainly related to the beauty of the photo. Rewarding quality contributions can also potentially invite concerns about how a judge or jury decides on the ‘best’ contribution. This might also have drawbacks when considering that there are groups of people who are less likely to contribute as they might not perceive their contribution to be ‘worthy’ or ‘good enough’ for the competition. Furthermore, there are ethical implications from the perspective of research if the contributions are intended to be used in scholarly work about a particular topic in which one expects to collect material and avoid impact from the researcher’s side (forcing expert power through giving awards) as much as possible. Another example of rewarding participants was the open curatorship project. Here the reward was the opportunity to execute the exhibition proposal later on in museum space. In many ways the reward was intended to be selection criteria for participant – to find those willing and interested in realising their idea. But in this way the reward also worked as a possible barrier.

4.6 Summarising the initiatives In summary of the initiatives above, we have drawn table 2. As it is possible to see, the Give the Museum a Day of Your Life and Open Curatorship Exhibition initiatives have managed to take into account the variety of Nielsen’s recommendations. Here the online and on-site combinations have worked well to provide alternatives and support participation in variety of ways.


Kreutzwald’s Century ($%%&'(()*+,%-./012)3*4,52++() is a web portal of Estonian cultural history, developed in the Estonian Literary Museum, which introduces the cultural history from the begginning o fthe 19th century – 1918, a year, when the Republic of Estonia was established. Its content is primarily the digital archive: archive materials, photos, earlier literatuure in Estonian language, introductions of the figuures in Estonian cultural history in the context of historical events of the time.


Give the Museum a Day of Your Life (April 2009)

Exhibition comments with pen and paper (autumn/winter 2010)

Providing different formats

Familiar technologies, available at multiple points throughout the exhibition

Providing different formats

Edit rather Providing a than create variety of different samples


Providing questions. Providing prefilled papers

Providing structure for the proposals.

Promoting quality

None specifically, all contributions were visible for all visitors

None specifically, all contributions were visible for all visitors

Winners rewarded with exhibition

Make easier

Selection of the best stories, but no public promotion of these

Museum night exhibition tagging (May 2010)

Open Curatorship Exhibition (2010-2011)

Proposing survey templates for voting

Side-effect Not available, although participation becomes an added value for others Reward

Prize draw general

Prize draw related to the exhibition


Exhibition as a reward put some people off

Although Nielsen’s (2006) framework was initially designed to support people attempting to build online communities, it also supports analysis of the participation initiatives for the practitioners, who can then look at the strengths or weaknesses of their activities. While Nielsen talks mainly about online issues and usability in general, the ideas proposed, and the support provided, suit on-site activities because this makes some things simpler. Here in our examples, we have managed to implement many of the recommendations, especially with online initiatives. However, the framework also indicates that there are issues that could have been implemented better and profiles of the participation project could have become more visible, especially if quality contributions could have been better promoted.

5 Conclusion The ways museums use technical measures to support or facilitate participation have to be linked with what museums have set as an aim for that particular participatory action. When participant numbers are sought after, ease of participation is very important, but at the same time, this ease can also become a barrier to more complex or different contributions. No matter if the participation is high-tech or low-tech, technology should not become a barrier in itself. In today’s world this means replicating participatory initiatives online and offline, because some technologies will become more accessible to some groups than to others. In our experience, intertwining works best for the engagement of a diversity of groups. 250

When outlining the different ways in which museums can engage audiences, Simon (2010) stresses the importance of the ‘why’ of participation. Chosen technological means should support the overall aim. Participation should not be an aim in itself, but rather support the other goals and activities of the museum. In conclusion to the particular participatory actions, it could be said that neither of the used participatory activities ‘threatened’ neither the museum nor the audiences/users on the level of expertise and knowledge making. Rather, the audiences received a different kind of approach to museum collections and exhibitions, and to some extent, other visitors. The actions also added reflexivity to museum professionals’ activities. In our examples, the contributions collected through these different measures have raised debates around issues of quality and issues of validity. On the one hand, museum workers consider ‘average man’ to be an expert in his or her own life, while on the other hand they are critical and hesitant when it comes to the idea that everyone should record their lives, provide input to the museum or support the collections with their own stories. For them the question of validity, standards of quality and the value of the contribution needs to be discussed and possibly reevaluated in the light of increasing participatory activities. Combining online and offline technologies is crucial for future museum institutions. In the future of the Estonian National Museum, the variety of participatory activities both online and on site will hopefully help to form a strong network of people around the museum. We expect that in the future technology will support museums and audiences, for example through re-making the ‘museum visit’ into the ‘museum engagement’: with content-rich ICT solutions, open to participation, with which a potential museum visit would start long before the real visiting experience and would not end with leaving the museum. Participatory contributions would be seen as an integral part of the museum and discussion around their value would be open with the participation.

Acknowledgements The authors are grateful for the support of the Estonian Science Foundation, research grants Nos. 7162 and 8006, and target financed projects SF0180002SO7 and SF0180017SO7.

References Bruns, A. (2006). Towards produsage: Futures for user-led content production. In F. Sudweeks, H. Hrachovec, & C. Ess (Eds.), Cultural attitudes towards technology and communication 2006 (pp. 275– 284). Murdoch: Murdoch University. Carnall, M. (2009),”Chasing the online audiences”, University Museums and Collections Journal 2, , available online at , last accessed 19/02/2011: 37-39. Carpentier, N. (2010), A short history of cultural participation. A. Aljas et al (eds) Transforming culture in the digital age international conference in Tartu 14-16 April 2010: Papers presented at the conference in Tartu, 14-16 April 2010.: Transforming culture in the digital age international conference; Tartu; 14-16 April 2010. Tartu: Eesti Rahva Muuseum: 11 - 19. Liebowitz, S. J. and Margolis, S. E. (1998), “Network Externalities (Effects)”. in: The New Palgrave's Dictionary of Economics and the Law, MacMillan, available online at, last accessed 19/02/2011 Marlow, C., Naaman, M., boyd, d., & Davis, M. (2006), Position Paper, Tagging, Taxonomy, Flickr, Article, To Read World Wide Web 2006 (WWW2006): Collaborative Web Tagging Workshop, Edinburgh, Scotland, available online at, last accessed 19/02/2011 Nielsen, J. (2006), Participation Inequality: Encouraging More Users to Contribute, Jakob Nielsen's Alertbox, available online at, , last accessed 19/02/2011


Pruulmann-Vengerfeldt, P.; Aljas, A. (2009). Digital cultural heritage – challenging museums, archives and users. Journal of Ethnology and Folkloristics, 3(1), 109 - 127.

Salgado, M. (2009) Designing for An Open Museum, An Exploration of Content Creation and Sharing through Interactive Pieces, available online at, last accessed 15/04/2011 Schweibenz, W. (2004), Development of Virtual Museums. ICOM News 3, available online at, last accessed 19/02/2011: 3. Simon, N (2010), The Participatory Museum. Santa Cruz (Calif.): Museum 2.0 Trant, J. (2009), Tagging, Folksonomy and Art Museums: Early Experiments and Ongoing Research. Journal of Digital Information 10(1) , available online at, last accessed 19/02/2011


A Virtual World Installation in an Art Exhibition: Providing a Shared Interaction Space for Local and Remote Visitors Spyros Vosinakis & Ioannis Xenakis Department of Product and Systems Design Engineering, University of the Aegean, Greece [email protected], [email protected] Abstract Visiting museums and exhibitions represented in multi-user 3D environments can be an efficient way of learning about the exhibits in an interactive manner and socialising with other visitors. The rich educational information presented in the virtual environment and the presence of remote users could also be beneficial for the visitors of the physical exhibition space. In this paper we present the design and implementation of a virtual exhibition that allowed local and remote visitors coexist in the environment, access the interactive content and communicate with each other. The virtual exhibition was accessible to the remote users from the Web and to local visitors through an installation in the physical space. The installation projected the virtual world in the exhibition environment and let users interact with it using a handheld gesture-based device. We performed an evaluation of the 3D environment with the participation of both local and remote visitors. The evaluation results indicate that the virtual world was considered exciting and easy to use by the majority of the participants. Furthermore, according to the evaluation results, virtual museums and exhibitions seem to have significant advantages for remote visitors compared to typical museum web sites, and they can also be an important aid to local visitors and enhance their experience.

1 Introduction Virtual Museums and exhibitions are considered an effective alternative to an actual visit, especially for people whose physical access to the site is restricted. A virtual museum is a digital representation of an artefact collection presented in a single- or multi-user environment, in which visitors navigate, observe the exhibits, learn related information about them, and in some cases interact with them (Sylaiou et al, 2009). Besides the obvious advantage of serving as information placeholders without time and space limitations, navigating within a virtual museum can generate rich user experiences that may partially satisfy the educational, recreational and social aspects of a museum visiting experience (Dede & Ketelhut, 2003). This holds especially in the case of Virtual Worlds with realistic 3D representations of the space and artefacts and with the ability to view and discuss with other users within the digital environment. People who cannot physically access a museum collection, e.g. for reasons related to distance, cost or kinetic disability, may benefit from the existence of a virtual museum that presents the same collection in a rich, interactive way. One significant drawback of today’s virtual museums is that the virtual space is completely separated from the physical space it represents. Therefore there are two independent social groups that browse the same collection and are not aware of each other: the visitors of the physical museum space, referred in this paper as “local visitors”, and the visitors of the virtual museum, referred as “remote visitors”. Although virtual museums may be enhanced with interactive content generated both by the curator and the community, the local visitors are not aware of it. E.g. exhibits may have attached hypermedia information, discussion groups may emerge, comments and tags may be added by the visitors, etc. Furthermore, although visitors of the virtual museum may discuss with each other about the exhibits based on their in-world experience, they cannot exchange opinion with people who have visited the actual museum and have had eye contact with the physical collection. In this paper we present the design and implementation of a virtual exhibition and its installation in a physical exhibition space, which allowed local and remote visitors be aware of each other and communicate within the 3D environment. The authors have set up a virtual world with a fairly accurate representation of the actual exhibition space enhanced with additional information about the exhibits, which could be accessed by Internet users through their browsers. During the exhibition, remote visitors were able to join the virtual exhibition space, browse the exhibits, view related information, add comments and chat with other visitors. Furthermore, the virtual world was also presented within


the actual exhibition space as a wall projection and local visitors had the chance to view the virtual museum users, chat with them and navigate within the environment using a low-cost gesture based interface. Remote visitors were also aware of the presence and motion of the person using the installation in the physical environment, if any, as her avatar representation indicated that she was located the physical exhibition space. The authors have performed an evaluation of the virtual world and its installation in the physical space in order to assess the usability of the environment and to discover critical issues concerning the shared experience of visitors. The virtual exhibition environment has been evaluated by a number of local and remote visitors using questionnaires and follow-up discussions, where possible. The analysis of the results has revealed that the environment was considered exciting and easy to use by the majority of the participants. Furthermore, the evaluation results have highlighted some strong points of interacting with virtual worlds within the exhibition space or remotely, as well as some issues that need to be further considered. The rest of the paper is structured as follows. Section 2 presents the related approaches in designing and implementing virtual museums, augmenting museums with virtual worlds, and sharing museum visits through technology. In Section 3 we describe the design and implementation of the virtual exhibition and its installation in the physical space. Section 4 describes the user evaluation process and presents the results. Section 5 presents the implications for design that we have identified based on the evaluation results. Finally, Section 6 presents our conclusions and future work.

2 Background 2.1 Virtual Museums and Exhibitions The main objectives of exhibitions are to provide a public space for the presentation of artefacts and to serve as a centre of knowledge specialized in a thematic area (Economou, 2004). 2D interfaces (e.g. museum Web sites) limit the user experience to simple page viewing and sequential browsing and leave no room for any immersive experience. On the other hand, a 3D representation of an exhibition environment places the artefacts in a natural-looking setting and may offer a much more realistic and entertaining experience (Lepouras & Vassilakis, 2004). The notion of virtual museums and exhibitions has been introduced by Tsichritzis and Gibbs (1991) as a means to overcome the limitations of the physical space and to provide a vivid experience to remote visitors. A synthetic collection of artefacts, which incorporates multimedia and virtual reality technologies, alleviates the problem of storing, preserving and protecting the real artefacts and allows virtual spaces to contain a limitless number of exhibits, to which users have access at any time and from any place. Furthermore, it may introduce new forms of presentation and interactivity that stand beyond the passive viewing of the artefacts and reading the accompanying legend, which is typically the case with traditional museum and exhibition visits. Digital artefacts may be presented using a combination of various forms of media, such as 3D representations and rich hypermedia annotations and may also let the user interact with them in many intuitive and creative ways in order to learn and entertain themselves. Various Virtual Exhibitions have been implemented as commercial or research projects during the last decade, taking advantage of the tremendous increase in the efficiency of graphics rendering and processing of modern PCs and of the availability of 3D scanning hardware. In some cases these applications serve as complementary information source to existing exhibitions and they are usually found in the web pages of museums or galleries and contain artefacts that belong to their collection. On the other hand, there are Virtual Exhibitions that host collections of artefacts that may be abstract, imaginary, restorations of damaged objects, hypothetical models of real artefacts that no longer exist, etc. (Ciabatti, 1998). There is a great diversity in terms of visualization and user interactivity in the available approaches, as a multitude of different technologies has been employed. As per the presentation of artefacts, the approaches include simple images, panoramic views, video and hypermedia presentations, and detailed 3D models. In terms of user interface, a variety of systems have been presented, ranging from a sequence of 2D pages containing the artefacts to immersive 3D environments in which users can navigate, explore the virtual space and get haptic feedback from the exhibits (Kim et al, 2006), using specialized hardware. The interaction modalities vary from simply viewing the artefacts to letting users have rich educative interactions with them (Petridis et al, 2005).


Concerning implementation technology, virtual museums have been developed either as standalone multimedia applications or as web-based environments using various standards, such as Flash, Quicktime VR, VRML (Virtual Reality Modeling Language) and X3D. They may run remotely in the users’ home computers or in public installations inside existing exhibition spaces. Virtual Exhibitions may also be presented in multi-user environments, where users are represented with avatars and may communicate using text or voice chat. In this case, users can have a collaborative visiting experience by meeting people with similar interests, commenting on the exhibits, exchanging information about the area of interest, making new friends, etc. Virtual Museums and Galleries are also being built inside persistent Virtual Worlds, such as the popular Second Life platform (Marty et al, 2007), thus making the exhibition accessible to a large, already established, user community, with no need to install any additional software or plug-in. An overview of Virtual Museum technologies is presented in (Sylaiou et al, 2009).

2.2 Augmenting Museums with Virtual Worlds Virtual museums can provide rich interactive content, from which the visitors of an actual museum may also benefit. The idea of installing a virtual world inside a museum is proposed by Charitos et al (2001) as a means to augment the visitors’ experience. According to the authors, this solution would overcome the problem of limited space and would provide rich, lively and multisensory exhibit presentations. Furthermore, it would serve as a navigational aid for visitors inside vast museums, as the virtual representation can provide a good overview of the exhibits and their arrangement in the exhibition space. Kwon et al (2003) propose the architecture of a virtual museum, which will be accessible from the real museum through an information docking station equipped with Mixed and Virtual Reality technology and through handheld PDAs that visitors can carry with them. Robertson et al (2009) describe a museum installation that presents a photorealistic virtual environment in a large wall projection. Museum visitors can collaboratively explore the environment and browse its multimedia content using handheld devices. A number of installations try to go beyond the simple projection of a 3D environment on a flat surface by generating more immersive experiences. Immersion in virtual worlds is supported through the use of wearable devices that provide high fidelity visualization with stereo vision, and more natural interactivity of the environment, e.g. using head tracking, hand and arm gestures, etc. (Biocca & Delaney, 1995). A number of immersive virtual reality applications have been installed in the Foundation of Hellenic World (Roussou, 2001) for educational and recreational purposes. They present reconstructions of historical cities and buildings, which visitors can explore using special handheld devices and wearing stereo glasses. A classification for immersive virtual reality installations on the basis of the hardware used for displaying and interacting with the environment is proposed in (Carrozzino & Bergamasco, 2010). The authors present a number of existing approaches and classify them using their proposed scheme. Finally, Papagiannakis et al (2005) propose the use of Augmented Reality technology to mix real and virtual scenes in the site of ancient Pompeii. Their system can generate a 3D storytelling environment with virtual actors superimposed in the real site, provided that the visitors are wearing the appropriate display devices. Immersive and augmented reality may be a promising technology for museum installations. However, it has been criticized for the feeling of nausea often caused to visitors in the extensive use of such environments, a symptom also called cybersickness (LaViola, 2000).

2.2 Shared Museum Visits The museum visit is also a social encounter and technology can serve as a communication medium between visitors. Falk and Dierking (1992) claim that the visitors’ perspective is strongly influenced by the social context and they consider it as one of the major aspects that shape the visiting experience. Additionally, one of the four museum visitor types identified by Umiker-Sebeok (1994) is the Utopian, who is interpreting the exhibition as an encounter session and her main goal is the social interaction. In a study of the motivational factors of exhibition visitors using questionnaires in three different sites, the results showed that a significant percentage of visitors had social interaction as one of their goals (Parker & Ballantyne, 2002). Paolini et al (2000) identified the need for interaction and collaboration between museum visitors and propose a number of cooperation metaphors that can be applied in virtual museums. These metaphors involve the distribution of the shared state, the grouping mechanism, the flow of information, the user and scene visualization, the user movement and the avatar representation.


Various systems have been developed to support synchronous or asynchronous communication between museum visitors. Sotto Voce (Grinter et al, 2002) was a handheld guide that supported shared listening of audio descriptions; groups of visitors could use it to listen to each other’s guidebook. An installation presented by Fraser et al (2004) allowed users to record their opinion about the exhibits and listen to other users’ opinions, thus serving an asynchronous communication mechanism between visitors. Cosley et al (2009) present a system for social tagging of museum exhibits using a handheld mechanism. Visitors can read each others’ tags, vote about them or add new tags to the exhibits. Dini et al (2007) propose the use of synchronous collaborative games to facilitate interaction between museum visitors. They present five games played on handheld devices that are relevant to the museum content. The games support mutual awareness and can be played collaboratively. Finally, Brown et al (2003) present a system that allows local and remote visitors of a museum share their visit in real time and communicate with each other. Local visitors use a handheld device, whilst remote visitors are using either a Web or a Virtual Reality interface. This co-visiting system shares some similarities to the installation presented in this paper, in the sense that both systems allow local and remote visitors communicate with each other, and that remote visitors are immersed in a VR environment. However, there are also a number of significant differences: a) our approach is not using an immersive VR technology, but a Desktop VR environment which is multi-user and accessible to remote users via the WWW. On the other hand, the system of Brown et al allows only one visitor to be immersed in the VR environment, who has to use special equipment, and Web visitors are interacting using a non-VR interface (Web page browsing). b) Our approach does not provide voice communication channel and does not track the actual position of the local visitors. However, it is a low-cost solution that provides a natural interaction with the environment. c) Our study has been set up during an actual exhibition and has been evaluated by the exhibition’s visitors, whilst the system of Brown et al has been tested using a series of trial sessions with three concurrent visitors per session.

3 The Virtual World Installation The authors designed and implemented a virtual world accessible by local and remote visitors of an art exhibition in order to facilitate visitor communication and to augment the exhibition with interactive content. The aim of this research was to provide a space in which local visitors can be aware of the activity taking place in the virtual world and to take active part in it. This perception of how live remote visitors are moving and experiencing the space that local visitors have just visited or are about to visit is expected to give them additional social navigation cues (Cosley et al, 2009) and enhance their sense of co-presence. This will also allow them to acquire extra information about the exhibits and to exchange opinions about them with the remote visitors. The communication between local and remote visitors taking place both synchronously as live discussions and asynchronously as commentary is expected to facilitate the sharing of the visiting experience and to widen the social context of the exhibition visit.

3.1 System Overview The virtual world was accessible during the exhibition of the artist Ioannis Xenakis5, which took place between the 14th and 23rd of August 2010 in the Cultural Center of Hermoupolis in Syros island. Remote visitors could access the virtual exhibition through the artists’ Web page, in which a hyperlink to the client application was provided. The virtual world was also projected in the actual exhibition space and visitors could browse the activity in real time and use the facility provided, i.e. a handheld device and a keyboard, to interact with the environment. The installation actually attracted the majority of the exhibition visitors and a significant percentage of them interacted with it. Furthermore, a lot of remote visitors joined the virtual world, especially people familiar with the artist’s work who could not physically access the exhibition. Figure 1 shows a visitor using the handheld device to navigate in the virtual world.




Fig 1. A local visitor interacting with the virtual exhibition

The key design choices of the virtual exhibition installation are the following: -





Exact representation of the actual exhibition. Although a lot of 3D virtual museums use larger virtual exhibition spaces and include more exhibits that the actual exhibition, the authors decided to design the virtual world as a copy of the physical space. One reason for this choice was that remote visitors could get a better understanding of the actual exhibition space and exhibit arrangement. The other was that the instant mapping between physical and virtual space would be beneficial to local visitors; they would easily recall the exhibits they had just visited when they navigate in the 3D space, and they would be able to instantly re-visit a physical exhibit that captured their attention or has been referred by others in the virtual world. Additional exhibit information. All exhibits are enhanced with additional content that provides important information to visitors, especially the ones wishing to get further insight into the artist’s work and related artistic movements. The content provided may also help visitors to better understand the context of the works of art and get richer interpretations of them. Avatar-based user representations, discussion and comments. Fostering visitor communication was one of the main goals of the installation. The virtual world uses humanlike avatar representations of users that move in real time so that other viewers are aware of their position and point of view. Furthermore, local and remote visitors can communicate with each other using text chat and they can exchange opinions about the exhibits by reading each others’ comments and adding new ones. Natural user interface for local visitors. The traditional desktop user interface, i.e. mouse and keyboard, requires from the user to be in a sitting position, or at least to use a supporting surface, which novice computer users may need to frequently look at. We decided to let local visitors navigate and interact with the virtual world in a more natural way and intuitive way using a handheld device and minimise the use of keyboard only in case she needs to post a comment or talk to the remote visitors. Internet-based, cross platform application. We wanted the virtual world to be accessible to the majority of the Web visitors without any need for additional installations. Therefore, we selected an implementation platform that allowed the 3D environment to be installed and executed directly from the Web browser and could run successfully in various platforms including Windows, MacOS and Linux.


Fig 2.System architecture The implementation of the virtual world is based on a client-server architecture. A virtual world server is responsible for the storage and update of the environment’s data. It loads the models of the exhibits, their associated information and their position and orientation in the 3D environment. It also maintains a list of the online users and their actions, e.g. navigation, communication, interaction with exhibits, and records the user comments on the exhibits. Remote visitors are connected to the virtual exhibition using a client application which is downloaded and running from their browser. The virtual world server communicates with the clients via TCP/IP and transmits all changes in the environment so that all users view a consistent version of the environment. On the exhibition room there was a computer with a slightly different client application running; the difference was that the user input was adjusted to communicate with the handheld controller and interpret its actions. The virtual world was projected in the exhibition room and users could navigate using the handheld device, which communicated with the computer via Bluetooth, and send messages and comments using a wireless keyboard placed on a surface in front of them. Figure 2 presents the architecture of the system.

3.2 Virtual World Interface and Functionality The client interface is a desktop multi-user virtual environment. Each remote visitor initially enters a user name and selects his avatar. There are six human-like avatars available: three male and three female. The virtual world is rendered from a first person perspective and, therefore, users don’t see their own avatar. Although it is common in virtual worlds to use a third person perspective, e.g. in Second Life, we preferred to use the first person view, because we wanted users to be able to observe the works of art without their avatar model blocking the view. Remote users can navigate freely in the environment with the keyboard or the mouse using a navigation technique, which is common in 3D environments and first-person computer games. During their navigation they can observe the exhibits and view the motion and actions of other users in real time. As a navigation aid we have added a minimap, which presents a small top-down view of the environment and is always rotated towards the direction the user is looking at. The location and orientation of the users in the environment are marked on the mini-map. Users can get more information about each exhibit by clicking on it. The environment checks the position and orientation of the user with respect to the exhibit she clicked and, if she is not standing close enough, it automatically moves the user towards the exhibit to an appropriate viewing position. Then, the user can select from a menu-based interface the type of content she wishes to view. The information associated with each exhibit in the virtual world is: a) background information about the content, artistic movement and materials used, b) related artwork of the artist not included in the exhibition, c) related artists and their indicative works (Figure 3a) and d) links to similar works in the exhibition. The latter is visualized in the environment as follows: routes are drawn on the ground from the user’s current position to the target exhibits, the exhibits are labelled as ‘similar’ and their location is also marked on the mini-map (Figure 3b). Finally, users can browse other users’ comments on the exhibit they selected and post their own ones.


Fig 3. Screenshots of the virtual world. a.Information about relevant artists b.Routes to similar exhibits Communication is taking place in the environment using text messages. Whenever a user sends a message, it pops up and stays visible for a few seconds above his avatar’s head. User messages are also permanently displayed on a message window allowing users to keep track of the chat history. The message window additionally displays other user actions such as a user entering or leaving the environment, the posting of a comment, etc. Finally, if a user is using the exhibit menus to view further information or comments, a status message is displayed on his avatar so that other users can be aware of her activity.

3.3 Installation Interface Local exhibition visitors view the same multi-user 3D environment and can perform all available actions. Their avatar appears in the environment with a status message that the user is “located in the physical interaction space”. They can interact with the environment using a Nintendo Wiimote, i.e. the controller of the Wii game console which is capable of sensing motion and acceleration, and a keyboard for sending messages and comments.

Fig 4. User interactions with Nintendo WiiMote The Wiimote can be used in the following interaction modes: -

Mouse mode. This mode can be used for clicking on the exhibits and interacting with the menus. If the user points the Wiimote on the screen and moves his arm, the mouse pointer is positioned accordingly. Pressing the “A” button is the equivalent of a left mouse click. Look around mode. If the user presses and holds the “B” button of the Wiimote and slowly moves his arm to any direction, her viewpoint in the 3D environment will rotate accordingly. This mode is useful for observing the environment from a fixed position. Walking mode. If the user presses and holds the “Up” direction arrow, her avatar will move forward in the environment. Moving the device to the left or right while holding the button will change the walking direction accordingly, thus allowing the user to steer her motion. The “Down” direction arrow causes the user avatar to move backwards and can be steered as well. 259

Finally the “Left” and “Right” direction buttons make the avatar walk sideways to the left or right side accordingly, a technique known in first-person computer games as strafing. Figure 4 presents all available user interactions in the virtual world using the Wiimote controller.

3.4 Implementation Details The implemented system is integrating a number of technologies. The client applications have been implemented in Java and run over the Web using the Java Web Start technology. They utilise the Java3D library for the construction, visualization and real-time interaction of the virtual environment. The 3D models of the exhibits have been built in VRML format using high resolution photographs of the actual exhibits and the exhibition space has been modelled based on its ground plan and using photographs as textures in some parts of the space for added realism. We intentionally avoided putting too much detail on the surrounding environment because we wanted the application to run in adequate refresh rates even in older computers. User avatars have been built using low-resolution models of Curiouslabs Poser and are also stored in VRML format. The VRML models are imported in the client applications with the use of the java3d-vrml97 loaders. The virtual world server is a Java application that communicates with the clients using standard TCP/IP sockets and a dedicated communication protocol that supports the message exchange between them. Finally, the Wiimote interface with the client application running at the exhibition room is supported by the WiiUseJ library.

4 Evaluation We performed a user evaluation of the virtual world and its installation in the exhibition in order to assess the usability of the proposed system, to gain empirical observations and to discover critical issues concerning its usage. Our evaluation method included monitoring of the visitors’ behaviour during their interaction with the environment, follow up questionnaires and discussion. Remote visitors were also asked to fill the same questionnaires via a Web form. 43 people participated in the evaluation, out of which 25 were local visitors (11 male and 14 female) and 18 were remote (12 male and 6 female). Concerning the age distribution the majority of the participants were between 18 and 35 (39.5% aged 18-25 and 32.6% aged 26-35). Six users were under 18, five users were between 36 and 45, and only one user was over 45. The questionnaire asked users to rate their experience with 3D environments (e.g. 1st person games) using a 1-5 scale (1: not at all - 5: very experienced) and the average was 3.23. The majority of the users found the virtual world exciting and easy to use. In a question asking users to rate how stimulating was their experience from the use of the application in a 1-5 scale (1: very boring – 5: very exciting), the average value was 4.4 (local visitors: 4.48, remote visitors: 4.29). The average visitor rating of how easy was their navigation in the 3D environment (1: impossible – 5: very easy) was 4.35 (local visitors: 4.4, remote visitors: 4.29). An interesting observation is that the local visitors navigating with the Wiimote, which they probably used for the first time, did not have significant difficulties using it. Furthermore, all users that reported having little or no experience with 3D environments rated the ease of navigation with 4 (easy) or 5 (very easy). Finally, the users were asked to rate their awareness of the presence of other users in the environment (1: not at all, 5: very aware). The average value was 4.16 (local visitors: 4.13, remote visitors: 4.22). The evaluation results concerning the visitors’ experience of the virtual world are summarised in Table 1. Local Visitors

Remote Visitors











Stimulation (1 very boring – 5 very exciting) Ease of navigation (1 impossible – 5 very easy) Awareness of other users (1 not at all – 5 very aware)

Table 1. User rating of their experience with the virtual world


Users were also asked about the advantages of 3D virtual exhibitions compared to typical exhibition Web sites. They could select more than one answers from the following: a) the presence of other users and the ability to chat with them, b) the presence of visitors from the physical exhibition space, c) the placement of the exhibits in a representation of the physical space, d) a more natural way of navigating and browsing the exhibits. Respectively, they were asked about the disadvantages of virtual museums compared to Web sites and the possible answers were: a) worse presentation quality of the exhibits, b) less chances to present extra information about the exhibits, c) difficulties in navigation, d) incompatibility with some platforms (e.g. cell phones, older computers, etc.). The results are presented in Figure 5. One may note that the advantage of browsing the exhibition in a more natural way has been selected by over 70% of the local visitors, a percentage significantly higher compared to remote visitors. This could be related to the fact that local visitors were using a gesture based interaction device instead of the typical mouse / keyboard input of the remote visitors and the environment was projected in a significantly larger area compared to a typical computer monitor. One user (remote visitor, male, 26-35) added the advantage that the natural size of an exhibit is easier to understand in virtual museums, because the user can see its analogy with the other exhibits and the room. Regarding disadvantages of virtual worlds, the navigation difficulties and the limited information about the exhibits were selected by only a few visitors. On the other hand, the presentation quality of the exhibits and the incompatibility with some platforms seem to be considered important drawbacks by the visitors.

Fig 5. Advantages (left) and disadvantages (right) of virtual museums and exhibitions compared to museum web pages There were also a couple of questions addressed only to local visitors, who had the experience of visiting both the physical and the virtual space. We asked them how similar was the visual representation of the exhibition presented in the virtual world compared to the physical exhibition space (1: not at all, 5: very similar) and the average value was 4.62. Another question was whether they the additional information presented in the virtual world was useful for their interpretation of the works of art (1: not at all, 5: very useful) and the average value was 3.93. The observation of local visitors’ behavior, the discussions with them and their free text answers in the questionnaire comply with the aforementioned results and reveal some interesting points for further consideration. Visitors were curious to watch and use the installation and needed little or no assistance


using the Wiimote. They were especially excited when remote visitors were present and they could communicate with them. The most common communication utterances were about their opinions of the exhibition and of specific exhibits. In several cases people moved their avatars in front of exhibits to highlight the ones they were referring to, e.g. in discussions about which they liked most. In some cases remote visitors were asking general questions about aspects of the exhibition that they could not observe through the virtual world, such as the number of local visitors, the ‘feeling’ of the actual exhibition space, the general impression of visitors, etc. Finally, there were also cases in which users were asking for assistance on the use of the virtual world interface or were starting more generic conversations with each other. Concerning their general impression of the installation, some users wrote that they would like to see more information about the artist and his works within the environment, e.g. to see his artistic evolution, his sources of inspiration, etc. One user (local visitor, male, 18-25) specifically noted that it would be interesting to include more information for users without significant background in the history of arts to assist them in interpreting the works. One local visitor and two remote would like the environment to offer additional navigation techniques usually found in computer games. Another local visitor (female, <18) wrote that “it is amazing that Wii is being used for moving in the 3D world”. There were a number of participants that highlighted the expected benefits of using virtual representations of museums and exhibition. They mentioned that they would be especially useful for people with kinetic difficulties and for people who do not have the time or money to visit all the museums they would like to. They also noted that the use of a virtual world inside a large exhibition space would be useful for visitors to see a preview of the environment and plan the route of their visit. Finally, there were a couple of visitors who said that the actual visit to the exhibition generates much stronger emotions compared to a virtual visit and expressed their concern whether virtual worlds would reduce museum visits in the future.

5 Implications for Design Individuals differ in how they construct their aesthetic experience, and visitors without significant background knowledge in art history usually require a lot of care or feeding in an exhibition (Smith & Wolf, 1996). Virtual worlds can help towards that way by augmenting the works of art with further information addressed to various visitor types, as same participants suggested. Considering that virtual worlds offer a more natural and intuitive way of presenting and interacting with the environment, a challenge for the interaction design community would be to discover novel metaphors for enhancing virtual museums with rich, interactive and adaptive information. A significant drawback for designing and developing interactive 3D environments is the notable lack of protocols and standards, a fact, which prevents their reusability and interconnectedness. The popular virtual world of Second Life may help towards that direction as it has a very large user base and a lot of content has already been developed in its worlds. However, its user interface is not customizable, one cannot install a private server, and the land ownership requires monthly subscription. Another critical issue regarding virtual worlds is that they require high processing power to be rendered in acceptable frame rates and, therefore, they are not suitable for computers with less capabilities, such as smartphones and netbooks, which are nowadays being used for browsing the Internet. On the other hand, visitors of virtual museums expect as much realism as possible. Therefore, designers should reach a good balance between visual quality and minimum system requirements. The interaction of exhibition visitors with a virtual world is expected to give further insight about the motion and preferences of visitors, e.g. whether they they prefer a linear organization in exhibitions, or a more holistic or global presentation (Veron and Levasseur, 1983), and may aid the development of adaptive content presentation (Bonis et al, 2009) targeted to various visitor types.

6 Conclusions and Future Work This paper presented the design implementation and evaluation of a virtual world that allows local and remote visitors of an art exhibition co-exist in a shared 3D environment, access additional information about the exhibits and communicate with each other. The evaluation results were promising. The virtual world installation captured the attention of visitors and they were quite excited from using it despite the fact that it was based on low-cost hardware. Furthermore, the evaluation revealed some important advantages and drawbacks of virtual worlds, as well as some issues that need to be considered by the research community.


In the future we are planning to expand our implemented system to a generic platform for designing virtual museums and use it to build a network of interconnected museum environments. Furthermore, we aim to experiment with mixed reality environments that will enhance the communication between local and remote visitors with voice and real-time video in order to offer more rich and vivid co-visiting experiences.

References Biocca, F. and Delaney, B. (1995), “Immersive virtual reality technology”, In Frank Biocca & Mark R. Levy, (Eds.), Communication in the age of virtual reality , Hillsdale , NJ : Lawrence Erlbaum Associates, 57–124. Bonis, B., Stamos, J., Vosinakis, S., Andreou, I. and Panayiotopoulos, T. (2009), “A Platform for Virtual Museums with Personalized Content”, Multimedia Tools and Applications, 42(2): 139-159. Brown, B., MacColl, I., Chalmers, M., Galani, A., Randell, C. and Steed, A. (2003), “Lessons from the lighthouse: collaboration in a shared mixed reality system”, Proceedings of the SIGCHI conference on Human factors in computing systems, 577–584. Carrozzino, M. and Bergamasco, M. (2010), “Beyond virtual museums: Experiencing immersive virtual reality in real museums”, Journal of Cultural Heritage, 11(4): 452-458. Charitos, D., Lepouras, G., Vassilakis, C., Katifori, V., Charissi, A. and Halatsi, L. (2001), “Designing a virtual museum within a museum”, Proceedings of the 2001 conference on Virtual reality, archeology, and cultural heritage. Ciabatti, E., Cignoni, P., Montani, C., Scopigno, R. (1998), “Towards a Distributed 3D Virtual Museum”, Proceedings of Working Conference on Advanced Visual Interfaces, pp 264-266. Cosley, D., Baxter, J., Lee, S., Alson, B., Nomura, S., Adams, P. et al. (2009), “A tag in the hand: supporting semantic, social, and spatial navigation in museums”, Proceedings of the 27th international conference on Human factors in computing systems, 1953–1962. Dede, C. and Ketelhut, D. (2003), “Designing for Motivation and Usability in a Museum based MultiUser Virtual Environment”, Proceedings of the American Educational Research Association Conference, Chicago. Dini, R., Paternò, F. and Santoro, C. (2007), “An environment to support multi-user interaction and cooperation for improving museum visits through games”, Proceedings of the 9th international conference on Human computer interaction with mobile devices and services - MobileHCI ’07, 515521. Economou, D. (2004), “Virtual reality and Museums: The development of a collaborative virtual environment for learning about ancient Egypt”, Museology - International Scientific Electronic Journal, 14(2). Falk, J.H. and Dierking, L.D. (1992), “The museum experience”. Washington, DC: Whalesback. Fraser, M., Bowers, J., Brundell, P., O’Malley, C., Reeves, S., Benford, S., et al. (2004), “Re-tracing the Past: Mixing Realities in Museum Settings”, Proceedings Of ACM ACE 2004. Grinter, R. E., Aoki, P. M., Szymanski, M. H., Thornton, J. D., Woodruff, A., and Hurst, A. (2002), “Revisiting the visit: understanding how technology can shape the museum visit”, Proceedings of the 2002 ACM conference on Computer supported cooperative work, 146–155. LaViola, J.J. (2000), “A discussion of cybersickness in virtual environments”. SIGCHI Bull.32, 1:4756. Kim, Y.S., Kesavadas, T. and Paley, S. (2006), “The Virtual Site Museum: A Multi-Purpose, Authoritative, and Functional Virtual Heritage Resource”, Presence 15(3):245-261.


Kwon, Y. M., Hwang, J. E., Lee, T. S., Lee, M. J., Suhl, J. K., and Ryu, S. W. (2003), “Toward the Synchronized Experiences between Real and Virtual Museum”, Proceedings of APAN 2003 Conference, 1-7. Lepouras, G. and Vassilakis, C. (2004), “Virtual museums for all: employing game technology for edutainment”, Virtual Reality, 8(2):96-106. Marty, P. F., Twidale M. and Urban R. (2007), “A Second Life for Your Museum: 3D Multi-User Virtual Environments and Museums”, Proceedings of Museums and the Web. Packer, J. and Ballantyne, R. (2002), “Motivational factors and the visitor experience: A comparison of three sites”, Curator: The Museum Journal, 45(3):183-198. Paolini, P., Barbieri, T., Loiudice, P., Alonzo, F., Zanti, M. and Gaia, G. (2000), “Visiting a museum together: How to share a visit to a virtual world”, Journal of the American Society for Information Science, 51(1): 33-38. Papagiannakis, G., Schertenleib, S., O Kennedy, B., Arevalo-Poizat, M., Magnenat-Thalmann, N., Stoddart, A., et al. (2005), “Mixing virtual and real scenes in the site of ancient Pompeii”, Computer Animation and Virtual Worlds, 16(1): 11-24. Petridis, P. et al (2005) “Exploring and Interacting with Virtual Museums”, Proceedings Of Computer Applications and Quantitative Methods in Archaeology (CAA). Robertson, S., Jones, B., O’Quinn, T., Presti, P., Wilson, J. and Gandy, M. (2009), “Multiuser Collaborative Exploration of Immersive Photorealistic Virtual Environments in Public Spaces”, Virtual and Mixed Reality, 235–243. Roussou, M. (2001), “Immersive Interactive Virtual Reality in the Museum”, Proceedings of TiLE (Trends in Leisure Entertainment). Smith, J. K. and Wolf, L. F. (1996), “Museum visitor preferences and intentions in constructing aesthetic experience”, Poetics, 24(2-4): 219-238. Sylaiou, S., Liarokapis, F., Kotsakis, K. and Patias, P. (2009), “Virtual museums, a survey and some issues for consideration”, Journal of Cultural Heritage, 10(4):520-528. Tsichritzis, D. and Gibbs, S. (1991), “Virtual museums and virtual realities”, Proceedings of International Conference on Hypermedia and Interactivity in Museums, pp 17-25. Umiker-Sebeok, J. (1994), “Behavior in a Museum: A Semio-Cognitive Approach to Museum Consumption Experiences”, Signifying Behavior. Journal of Research in Semiotics, Communication Theory, and Cognitive Science, 1(1):52-100. Veron, E. and Levasseur, M. (1983), “Ethnographie de l'Exposition”. Bibliothque publique d'Information, Centre Georges Pompidou, Paris.