Living lab

Articleissues|article=September 2008
cleanup = September 2008
OR = September 2008
peacock = September 2008
advert = September 2008

Living Lab is a new research paradigm integrating both:
* a user centred multidisciplinary research approach
* a user community driven innovation based on real life experiments

It is intended to:
* increase the understanding of occurring phenomena
* explore and evaluate new ideas, concepts and related ICT artefacts
* confront new ideas, concepts and related ICT artefacts with users' value model
* enable re-usable experiments (i.e. dataset, research protocols and methods,..)
* result in more accurate and reliable products and services
* speed-up concepts to market and promote viral adoption
* contribute to initiate potential lead markets
* contribute to bring science and innovation closer to the citizen

"Living Lab is more than experimental facility as its philosophy is to turn users, from being traditionally considered as a problem, into value creation"

Introduction

Nowadays, ICT users are creating value in massively populating web applications and online role-playing games with their own activities and related data (i.e. eBay, Amazon, Linked-In, FaceBook, SecondLife, World-of-Warcraft). Over the last 10 years, ICT has dramatically progressed and is more and more embedded into people’s everyday life in such a way that they have at hand more open, powerful and easy to use ICT devices and applications (essentially web-based) than the ones they have at work that are close, limited and complex to use.

Several years ago research studies were demonstrating that the media naturalness of human beings was making almost impossible to create interpersonal relationships beyond the use of natural senses such as sight and touch. Nowadays, when looking at the global success of online social networking and other popular web applications, one realises that almost everyone is willing to become the friend of everyone else around the world and share with them things he likes (i.e. friends, music, movies, books) whatever is the level of media naturalness. Should this phenomenon be considered as a multiplayer online game attracting more fans every day or people willingness to experience something new which is not limited by geographical and time constraints or even a prelude towards collective intelligence Pierre Levy, group behaviour, cognition (i.e. “The Wisdom of Crowds”) and consciousness into the cyberspace?

Would that be possible to imagine people developing new behaviour and senses in the cyberspace to perceive relevant existing information and knowledge resources which would normally be out of reach?

Our knowledge and understanding of this phenomenon is currently quite limited and subject to many possible interpretations. However, the JOHARI Window Model (Luft and Ingham, 1969) is telling us that anyone exposing on the web can potentially receive massive feedback in return. Looking at YouTube and DailyMotion posted videos it becomes obvious that the Internet and web technologies have already significantly unleashed people creativity for the best as well as for the worst. Are the Internet and Web transforming people behaviour? Or are individuals adapting their behaviour to the new opportunities delivered by mobile devices almost permanently connected to the Internet? How is ICT going to evolve in the near future to meet fast evolving citizen/individuals’ needs?

How are we going to better understand this phenomenon? Is this a continuous phenomenon or will the increasing communication and information overload lead to an overkill reaction or a division of our societies between the multi-information-network-tasking users and those who are not capable to deal with the new situation? What could be the most appropriate research approach and environment to investigate these crucial questions?In this article, we are arguing that one possible way to answer the above questions is to adopt a “User-centred Multidisciplinary Research” approach, as being an extension of the Experience and Application Research. This new research approach could be implemented through a range of emerging technologies assembled into User Experience Prototyping Environments as being a new ICT research field enabling users and scientists from complementary disciplines to explore new innovative concepts.

Dr William Bainbridge, head of Human-Centred Computing at the US National Science Foundation, wrote in the journal Science: “Online worlds offer great potential to social scientists because they overcome some of the problems these researchers encounter when gathering subjects in the real world, For instance, social scientists often face problems finding subjects fast enough or securing funds to carry out the research. The popularity of online worlds such as Second Life and World-of-Warcraft meant there was a ready pool of subjects that could be recruited over long periods of time for little cost. The game worlds also gather huge amounts of data about what players do that could easily be analysed by social scientists.”Human-centred computing appears to be a potential EAR/Living Lab related ICT research domain, especially because it is an emerging multidisciplinary research field devoted to computing and computational artefacts addressing human aspects. It involves researchers from different disciplines such as computer science, sociology, psychology, cognitive science, anthropology, science and technology studies, and industrial design.This is an opportunity for Living Labs to lead the way towards a new innovative ICT field which can help to introduce a more creative and intuitive style into research while at the same time bringing science closer to the citizens through the engagement of communities of users.

EAR and Living Labs

Experience and Application Research (EAR) has been proposed by the ISTAG committee in 2004 as a means of addressing the challenge of creating a human-centred approach to R&D in ICT Ambient Intelligence for supporting integrated research and concurrent assessment of Ambient Intelligence technologies and systems. EAR involves research, development and design by, with and for users. In fact, ISTAG suggested that Ambient Intelligence research increasingly needs “to allow people to live in their own future” in order to bring that research closer to the needs of citizens and businesses.EAR technologies should enable prototyping of novel interaction concepts while resembling natural environments of use. These ‘experience prototyping’ environments should also be equipped with observation technologies that can capture and analyse the behaviour of people that interact with the experience prototypes.

Several EU research projects, funded under the 6th Framework Programme ( [http://www.ami-communities.eu/wiki/C@R Collaboration@Rural] , [http://www.ami-communities.eu/wiki/CoSpaces CoSpaces] , [http://www.ami-communities.eu/wiki/ECOSPACE ECOSPACE] , [http://www.ami-communities.eu/wiki/LABORANOVA LABORANOVA] and [http://www.ami-communities.eu/wiki/WearIT@Work WearIT@Work] ), have implemented a User-centred Research approach, named either “Experience and Application Research” (EAR) or “Living Lab”, in order to involve end-users at the earlier stage of the R&D process. Following this EAR or Living Lab approach, end-users are engaged into User Experience Prototyping Environments where scientists from complementary disciplines are expected, on the one hand, to reach a higher level of understanding of occurring phenomena (such as the one described above) and on the other hand, to create new concepts that will lead to radical innovations in terms of both ICT products and/or services.

Beside the fact that a Living Lab is gathering a network of complementary organisations and stakeholders involved in the R&D process in combining technology push and application pull, the main concept is to turn users, from traditionally being considered as a problem, into a value. There are currently 51 Living Labs across the EU that are members of the European Network of Living Labs (ENoLL). According to the nature and scope of each Living Lab, different scientific disciplines might be represented. For example, a City driven Living Lab could be more orientated towards Social sciences, Humanities, Computer science and policy development (i.e. e-government, e-democracy) and Environmental sciences as well as Energy for the benefit of citizens while a Regional Competitiveness Cluster driven Living Lab is more orientated towards Computer science, Biomedical and Life sciences, Physics in aeronautics as well as Engineering for the benefit of regional business activities.

In the past, a great guy, named Mason Cooley, said "Every path to a new understanding begins in confusion.", which explains why people may have difficulties in perceiving the differences between testbed and Living Lab. Someone else, Albert Szent-Gyorgyi, famous for his Nobel prize said "Research is to see what everybody else has seen, and to think what nobody else has thought" which is exactly where Living Lab is standing now in arguing that building up User Experience Prototyping for exploring new innovative concepts and better understanding users’ behaviour and their real need is radically different than building-up testbed for testing functionalities against requirements.

An American Astronomer, Writer and Scientist, Carl Sagan, said “Somewhere, something incredible is waiting to be known.”, which makes sense for people willing to get more powerful observation instruments to be lightening something still in the darkness waiting to be discovered. This is exactly what is needed for EAR/Living Lab but instead of lightening space objects, the new observation instruments (i.e. new ICT sensors) should combine multiple heterogeneous data streams for lightening new innovative concepts and related users’ behaviour. This will result into a considerable increase of collected data amount and complexity. However, it is more than coping with massive amount of data as there is also a need for multidisciplinary research combining human expertise and knowledge between Domain scientists and ICT scientists for being able to understand and interpret the data from different perspectives.

Why is it vital to have a User-centred Multidisciplinary Research Approach?

Recent research studies shows that 70% to 80% of new product and service development that fails does so not for lack of advanced technology but because of a failure to understand real users’ need. Beside those studies there are other examples such as this dramatic aircraft crash which led to the conclusion, after analysing all elements that the crash was mainly due to the change of analogical displays in the cockpit by numerical ones which has turned into a miss-interpretation of parameters by the pilot during a stressful situation. Since this time, there is a higher motivation to involve users at the earlier stage of the R&D process in order to better understand the relationship between new innovative concepts and related users’ behaviour within specific situations as well as potential cognitive workload in interpreting received signals.A another good illustration of the need to take into account users at the earlier stage of the R&D process concerns the place of trust in man machine communication. The previous example of aircraft accident is an example of bad mapping between human tasks/goals and system design. The user interface is ill-designed and it is also a matter of task allocation based on confidence: the pilot trusts that the technical system will reach the expected performance and will give alert in case of problems. Beyond the trust in a technical system, the web has brought another level of conscience as the devices are used to connect people for share activities. In many web applications, it is the case: for instance in commercial transactions (e-commerce) or in car-sharing. Clearly, to deal with these situations and support relationships, it is necessary to understand trust building mechanisms "in the real life". In fact, if a technical system is intended to support social relations, it has to provide more than good response time, clear user interface and optimized code. It has to provide hints and clues that the other people on line will follow implicit rules and will behave as it is expected by other stakeholders of an interaction situation.

One can also imagine that User-centred Multidisciplinary Research Approach through the development of specific User Experience Prototyping Environments (such as virtual/mixed reality environments) could also be used for the treatment of Phobia where patients and psychologists are immersed together into a simulated environment reproducing stressful situations. This kind of Experience Environments could also be used to design social friendly public spaces in order to reduce the level of stress and increase the ambiance friendliness.

The Multidisciplinary nature of a User-centred Research approach

Within a User-centred Research approach, several disciplines such as Computer science, Ergonomics, Economics, Cognitive, Psychology, Social sciences, Environmental sciences, Humanities and Life sciences are necessary for designing and building-up User Experience Prototype environments, exploring new concepts and related artefacts, making proper observations and evaluation on different aspects according to the context of the specific research projects.The "observed end-users" are not necessarily immersed individually but could also be immersed as a group or even as a community which is leading to richer observations and greater quantity of collected data which are increasing the reliability of the resulting analysis.

Living Labs Research Challenges

There are current gaps of different natures related to both Research Infrastructure (RI) and ICT Research and Technology Development (RTD).

EAR or Living Labs Research Infrastructure

There is a need for a specific Living Lab RI at the European level providing access to the Science community to the harmonised user-centred research and innovation services (best practices, methods, tools and platforms) supporting the early involvement of users within the R&D process. This is relevant to the Capacities programme of the DG Research and more specifically addressing the Research Infrastructure WorkProgramme.

User Experience Prototyping Environments

There is also a need to carry on research dedicated to the development of specific User Experience Prototyping Environments designed to experience new ICT paradigms (i.e. Ambient Intelligence, Web2.0, Web3.0, Perceptics), innovative concepts and related artefacts that will lead into innovative ICT based products and/or services. This is relevant to the ICT Work programme but not yet addressed by one of the current objectives. There is another need for an agreed catalogue or library or even handbook of research methods that serves both as a resource and on the same hand as a standardisation media to harmonise and to compare experimental research. Such a library can be extended by a set of use case to inform the experimentation within the community.This kind of ICT based research environment would be valuable to most of the ICT research fields as it enables the systematisation of the User-centred Research approach and access to a wider scope of collected usage data across the EU.

ICT RDT need: User Experience Prototyping Environments

While usability tests have already been successfully implemented within the downstream side of the R&D process, very few has been done so far regarding the earlier involvement of end-users within the upstream side of the R&D process. This is especially true during the exploration of new innovative concepts and observation of related users’ behaviour.It is important to note that as user experience widens the concept of usability (see for instance: Norman 1998, 2004, 2007; Garett , 2002). User experience methods to assess user experience have to tackle wider dimensions as done by traditional usability evaluation methods. Methodologies for studying user experience are, admittedly, still in their infancy, although many ingredients are already available. Contrary to assessment methods several approaches try to understand experiences and their context of origin. Examples are cultural probes (Gaver et al 1999; 2004) which are collections of materials that are designed to provoke inspirational responses from people in diverse communities or the experience sampling method (Csikszentmihalyi et al 1987; 1997). This is based on the repeated assessment of individual or group behaviour and experience in the daily context. Other methods refer to experience prototyping to simulate experiences if different situations or idea generation methods for specific experiences.

Various scientific communities are interested in better understanding the behaviour of individual users as well as group or community of users on the internet, notably Human Computer Interaction research community (HCI), Social sciences and Data mining which provides tools to process massive and evolving data set. While there is such a common interest, very little effort has been done to share resources, to combine point of views, and in most cases each discipline simply ignores the others. The new approach of "User Experience" encompasses all aspects of end-users’ interactions, widen the scope of previous research fields and engages researchers from different disciplines to create new Meta-heuristic (i.e. genetic algorithms, ant colony algorithm, differential evolution algorithm) based analysis tools.

A User Experience Prototyping Environment is a kind of exploration-of-concepts/proof-of-concept/adoption-of-concepts platform which is composed of three distinctive parts:
* Designing user experience prototypes around new innovative concepts to be explored;
* Immersing individual or group of users into specific situations for designing and experiencing new innovative concepts;
* Evaluating usage, impact and potential adoption: Evaluate the cognitive workload level of individuals and groups with a proposed concept design to be explored. Visualising combined heterogeneous collected data and making multidisciplinary observations on the usage of concepts.

This kind of environment encompasses the fields of human factors, human-computer interaction, socio-emotional interaction, cognitive psychology, cognitive ergonomic, computer science, artificial intelligence and other related fields. As in the case of Socio-Cognitive or Cognitive Engineering, users and observed usages are considered as being the central drivers for new innovative concept design and exploration.

ICT RTD should provide the following elements:

The main objective is to researching emerging models, methods and tools coming from new scientific and technological derived paradigms to increase the quality of User Experience and Observation (quantity as well as quality of collected data) of new innovative concepts leading to new innovative ICT products and/or services.

1. New models

Research and develop new models incorporating socio-cognitive, cognitive ergonomic, socio-emotional and economic aspects that will enable new usage analysis methods. Models have to be developed from the perspective of meta-systemic knowledge and socio-cognitive values involved in the management of large complex heterogeneous intelligent socio-technical systems integrating human, technological and environmental elements.
* Scenario and Session Models: Context, user-centred sessions, Defining interaction steps between users and their experience environment.
* User Model: Collecting Usage Data/Experience, Pre-processing Data for collective usage data, Clustering users and concepts, individual and collective behavioural aspects, user or session profiling, Mining online social and concept networks especially for adaptive tools.
* Cooperation Model: networks, patterns, and forms of cooperation.
* Trust Model: Evaluation criteria, mining of users interactions through different channels and implication on interpersonal relationships.
* Forecast Model: Measure Internet browsing behaviour, cognitive overload and particularly switching costs and searching costs for users (i.e. user learning costs).

2. New techniques/methods To go further in these fields, that are addressing growing data set, refining users or network modelling, new multidisciplinary methods articulating usage mining approach and human factors expertise are required. Researchers will have to improve their clustering techniques and use more advanced techniques to process usage data streams. These different techniques are studied by researchers from the Data Mining/KDD community, though a new specific field has emerged, namely Web Usage Mining (WUM):
* Data acquisition methods: synchronizing heterogeneous data, cognitive radio, based on new instruments and able to get structured data inside collective dynamic situations.
* Data Mining and Benchmarking methods: some ICT research could be very relevant for User Experience context and should be validated in this context: for example usage data is a kind of very large data sets and could be a good context of validating algorithms in mining data streams. Another example, contrary to document mining field where there is a lot of benchmarking initiatives (challenge TREC, INEX offering large documents collection - XML or not), it is needed to do the same for usage mining communities.
* Knowledge Elicitation/Discovery and Evaluation methods: The most crucial aspect for ICT research is to formulate research ICT problems inside the EAR context. Indeed some very crucial notions in EAR such as "trust", "social networks", "usage" ... should be included in the formulation itself of the ICT research problems with a clear description of the way to validate such researches. Multi-disciplinary researches with a pragmatic, cognitive and socio-cognitive approach are of a crucial importance nowadays. For example: - coupling objective data (logs, usage traces) with subjective data (questionnaires) to answer questions such as "how to measure the successful and failed user sessions "how to measure trust in usage data according to a given application ? " - taking into account multiple points of views in a Knowledge Discovery process from Databases ( KDD), process which is usually decomposed in three steps: pre-processing, mining and interpretation/ validation. Cognitive and socio-cognitive approach are crucial in the pre-processing of usage data in terms of data cleaning, data structuring as they help to understand what are the useful analysis units, construction of new aggregated variables as well as variables that will be related to the notion of trust, selection of relevant variables or relevant distances for the mining step.... There is the same need for the interpretation step.
* User Experience research methods: constituting a catalogue of research methods that could be combined for being able to understand and interpret phenomenon from different perspectives. It should go beyond the current socio-cognitive and related methods such as Applied Cognitive Task Analysis, Cognitive Function Model, Cognitively Oriented Task Analysis, Interacting Cognitive Subsystems, Unstructured and Structured Interviews, Group Interview, Walk-through, Cognitive Walk-through, Talk-through, Formal Usability Studies, Ergonomics Checklists, Situated and Distributed Cognition, not mentioning Ethnographic research.

3. New tools

* User Experience modelling tools (i.e. User dynamic profiling, behavioural and trust profiling)
* Virtual/ Mixed User Experience prototyping tools: Prototyping of new participatory communication/collaboration paradigms often involving very targeted and selected communities, addressing small or large groups of users such as team or community, providing the appropriate community environment beyond classic 2D based for supporting user group experience in a 3D immersion space. The resulting elements will constitute an Experience Environment that will be applicable in most of user group experience situation. These virtual worlds may look primitive still, but citizens are already living in an enriched world where people interactions with companies, banks, institutions, universities, cities and public services, are no longer just based on a physical communication paradigm. Instead they have become highly mediated by technologies. This will continue to grow. People interactions will not only become more mobile but also more involving, more three-dimensional, and more experiential.
* Simulated User Experience prototyping tools: Prototyping of new interaction paradigms through simulated environments to handle extended multidisciplinary simulations with multi-source acquisition for supporting user group experience into a shared simulated space instead of the traditional individual user experience space. Researchers can be getting insights into real life by studying what people do in virtual worlds, suggesting that virtual worlds could help scientists studying ideas of government and even concepts of self, while other researchers are looking at how behaviour peculiar to online worlds differs from that in real life.
* Usage Analysis tools: New analysis tools based on a more global approach of the user tasks both for a better ICT research or ICT based services evaluation and for improving the understanding of User Experience. Widen the scope of previous research fields and engages researchers from different disciplines to create new Meta-heuristic based analysis tools.
* Living Lab out of the box tools that enable the set-up of an experiment anyplace on demand. Experimentation in an unconventional digital environment as Virtual laboratories to better understand human behaviour.

4. New Infrastructure Paradigms

* Researching new User-centred/Participatory Infrastructure Paradigms where users are setting up and maintaining their own infrastructures (e.g. Wifi networks, Web communities) which requires a fundamental different design of the infrastructures, their user interfaces, operations and maintenance.
* Experiment new User-centred/Participatory Infrastructure Prototypes which means that said paradigmatically new user-centred infrastructures have to be conceived, engineered, built technically tested and validated in a user participatory approach.

References

* Csikszentmihalyi, M. and Larson, R. (1997). Validity and reliability of the Experience Sampling Method, Journal of Nervous and Mental Disease, 175. 526-536
* Csikszentmihalyi, M., Larson, R. and Prescott, S. (1987). The ecology of adolescent activity and experience, Journal of Youth and Adolescence, 6. 281-294
* Garett, J (2002). The element of user experience, Paperback
* Gaver, B., Dunne, T. and Pacenti, E. (1999). Design: Cultural probes, interactions, 6 (1). 21-29
* Gaver, W.W., Boucher, A., Pennington, S. and Walker, B. (2004). Cultural probes and the value of uncertainty.
* Kock, N. (2005). Media richness or media naturalness? The evolution of our biological communication apparatus and its influence on our behavior toward e-communication tools. IEEE Transactions on Professional Communication, 48(2), 117-130.
* Luft, Joseph, (1969). "Of Human Interaction", Palo Alto, CA: National Press.
* Norman , D. (1998). The Invisible Computer, Why Good Products Can Fail, the Personal Computer Is So Complex, and Information Appliances Are the Solution, Cambridge MA, MIT Press
* Norman , D. (2004). Emotional Design. : why we love (or hate) everyday things, NY : Basic Books.
* Norman , D. (2007). The Design of Future Things, Basic BooksOther links regarding User Experience:
* http://www.uxnet.org/
* http://www.uxmag.com/

* Core Labs (2006), http://www.ami-communities.net/wiki/CORELABS.
* The European Network of Living Labs, http://www.openlivinglabs.eu
* de Ruyter, B. & Pelgrim, E. (2007). Ambient Assisted Living research in CareLab, ACM Interactions, Volume 14, Issue 4, July + August 2007.
* de Ruyter, B., van Loenen E. & Teeven, V. (2007). User Centered Research in ExperienceLab, European Conference, AmI 2007, Darmstadt, Germany, November 7-10, 2007. LNCS Volume 4794/2007, Springer.
* Andrew Kusiak, The University of Iowa, "Innovation: The Living Laboratory Perspective", Computer-Aided Design & Applications, Vol. 4, No. 6, 2007, pp 863-876
* ISTAG EAR working group report "Involving users in the development of Ambient Intelligence" on http://www.cordis.lu/ist/istag.htm
* Kusiak, A.; Tang, C.-Y.: Innovation in a requirement life-cycle framework, Proceedings of the 5th International Symposium on Intelligent Manufacturing Systems, IMS’2006, Sakarya University, Sakarya, Turkey, 2006, 61-67.
* Niitamo, V.-P.; Kulkki, S.; Eriksson, M.; Hribernik, K. A.: State-of-the-art and good practice in the field of living labs, Proceedings of the 12th International Conference on Concurrent Enterprising: Innovative Products and Services through Collaborative Networks, Milan, Italy, 2006, 349-357.
* Schumacher, J.; Feurstein, K.: Living labs – a new multi-stakeholder approach to user integration, Presented at the 3rd International Conference on Interoperability of Enterprise Systems and Applications (I-ESA'07), Funchal, Madeira, Portugal, 2007.

ee also

* Cognitive science
* Context awareness
* Human-computer interaction
* Information science
* Social computing
* Ubiquitous computing
* User-centered design
* User experience