Book launch: Grounded Innovation by Lars Eric Holmquist

At the Museum of the Weird in Austin Lars Erik Holmquist hosted a book launch party for his book: Grounded Innovation: Strategies for Creating Digital Products. The book uses a good number of research examples to highlight the challenges and approaches for digital products. The book has to parts: Methods and Materials and shows how both play together in the design of digital products. There is a preview for the book at Amazon.

Over 10 years back I worked together with Lars Erik on the European Project Smart-Its (http://www.smart-its.org/), where we created sensor augmented artifacts. The book features also some of this work. To get an overview of the project have a look at [1] and [2]. The concept of Smart-Its Friends is presented in [3]. Smart-Its friends proposed the idea, that products can be linked by sharing the same context (e.g. connecting a phones and a wallet by shaking them together).

[1] Lars Erik Holmquist, Hans-Werner Gellersen, Gerd Kortuem, Albrecht Schmidt, Martin Strohbach, Stavros Antifakos, Florian Michahelles, Bernt Schiele, Michael Beigl, and Ramia Maze;. 2004. Building Intelligent Environments with Smart-Its. IEEE Comput. Graph. Appl. 24, 1 (January 2004), 56-64. (PDF) DOI=10.1109/MCG.2004.1255810 http://dx.doi.org/10.1109/MCG.2004.1255810

[2] Hans Gellersen, Gerd Kortuem, Albrecht Schmidt, and Michael Beigl. 2004. Physical Prototyping with Smart-Its. IEEE Pervasive Computing 3, 3 (July 2004), 74-82. (PDF) DOI=10.1109/MPRV.2004.1321032 http://dx.doi.org/10.1109/MPRV.2004.1321032

[3] Lars Erik Holmquist, Friedemann Mattern, Bernt Schiele, Petteri Alahuhta, Michael Beigl, and Hans-Werner Gellersen. 2001. Smart-Its Friends: A Technique for Users to Easily Establish Connections between Smart Artefacts. In Proceedings of the 3rd international conference on Ubiquitous Computing (UbiComp '01), Gregory D. Abowd, Barry Brumitt, and Steven A. Shafer (Eds.). Springer-Verlag, London, UK, UK, 116-122. (PDF)

TEI2011 Keynote: Bounce Back by Gilian Crampton Smith

Gilian Crampton Smith is one of the people that massively influenced the field of interaction design from the very beginning. She is currently working in Venice: www.interaction-venice.com. Nearly 20 years back in London Durrell Bishop was one of her students. She pointed out that the central insight of the early work was "We know how to control objects by physical means". This insight has since become a basic motivation for tangible interfaces and embodied interaction. Closely linked to this is the complexity of interaction. To me one great point with designing tangible user interfaces is that you are forced to come up with a simple solution. Overall the argument was that embodied interaction works because it draws on knowledge we have. An example is that two physical things cannot be in exactly the same place and another one is that things stay where they are if there is no force moving them. There are clear limitations to the interaction with physical objects that give indications how to use it; she referenced a paper on an exploration of physical manipulation [1].

As one example Gilian showed the marble answering machine concept video from 1992. To me this video is a great example that shows the power of a concept video. Using a simple animation in this concept video enables viewers to understand the idea and interactions very quickly. I like the simplicity and clarity of this video - it would be very useful for teaching. Hopefully it will made publicly available; see below the bad-quality copy I took with my mobile. A good quality version ist at http://vimeo.com/19930744

One interesting point was a reflection on the shortcomings of the traditional view of interaction being based on input and output. Gilian argued that output should be separated in feedback and results. Input is categorized in according the interaction required, such as simple (e.g. text, minimal movement), medium (e.g. GUI), maximum (e.g. musical instrument, movement of the whole body). The feedback is related to the sense human have, and includes modalities such as visual, auditory, tangible, kinesthetic, proprioception. Results are related to feedback but are oriented on the outcome. Results may be visual (e.g. symbolic, words, icon, films); auditory (e.g. sounds, words or music) or physical such as touch (e.g. massage machine) or movement.

Gilian went on to the question: "Why aren't tangible, embedded and embodied interfaces out there in our everyday world?" For me the most important answer she gave is that it is hard to do them. It is very difficult to created sensor based interaction in a way that users understand it and that it feels natural to use. Counter examples where people got it wrong are those many lights that go on when you are already half way up the stairs or the automatic doors where people wave their hands in order to open them. As much of the interaction technology in such systems is not visible people have a hard time figuring out how things work. Based on their (incomplete) experience base on a few interactions they will create their mental modal … which is often wrong and potentially leads to frustration. A further point she made is that creating physical interactive objects and things is difficult and involves a lot of different skills. In combination with the function it is difficult to get the aesthetics right. One very good example is how hard it is to the aesthetics of feel is right, as it includes a well balanced design taking into account touch, weight, balance, movement, and sound. Overall this requires a great passion for detail to get the quality of engineering right; Gilians examples for this was the AUDI advert where an Engineer opens and closes the car door and listens carefully. I think I have seen in 8 years back in the UK as part of the ad-campain "Vorsprung durch Technik". If anyone has a link to a copy please post it.

When I was in 2002 (or around this time) visiting her at Ivrea near Torino I was very much impressed by the creativity and inventiveness. In her talk she explained one approach to teaching I really like. The students get a device (e.g. an alarm clock, answering machine) without manual with the task to figure out how it works. While exploring the functionality they film themselves. Based on this they reflect on the design and then go on to do a completely new design for a device with a similar functionality (with the constraint that it cannot have buttons).

I like a reference to something Bruce Steering wrote. In short it basically says research is like crime; you need Means, Motivation and Opportunity.

Some further example she showed:

• A communication device based candle: ceramic liaison. It is a bi-directional 1-bit communication device, that is esthetically pleasing. The state of the candle (real wax with fire) on either side is reflected by lighting up some ceramics on the other side.
• A full body experience interaction device for controlling games: Collawobble. Using two bouncing balls a packman game can be controlled. One user controls with the bouncing X and the other Y.

update:
Ellen Do posted 2 clips that were shown in the keynote

• http://vimeo.com/11802060
  L'età del fuoco - The age of fire
• www.youtube.com/watch?v=LE9t98Gox60
  Jaques Tati - Mon Oncle (1958) - Kitchen Scene - The struggle of a simple man with a "highly modern" kitchen and a bouncing jug

and suggested a further one: http://www.youtube.com/watch?v=oS6pwQqSY70

the marble answering machine video on http://vimeo.com/19930744

[1] Andrew Manches, Claire O'Malley, and Steve Benford. 2009. Physical manipulation: evaluating the potential for tangible designs. In Proceedings of the 3rd International Conference on Tangible and Embedded Interaction (TEI '09). ACM, New York, NY, USA, 77-84. DOI=10.1145/1517664.1517688 http://doi.acm.org/10.1145/1517664.1517688

Interviewing with Rikke Friis Dam and Mads Søgaard

Rikke Friis Dam and Mads Søgaard are currently working on a re-launch of the website interaction-design.org. The side has over the last years involved in a useful resource for researchers and practitioners in human computer interaction and interaction design. There is a very comprehensive calendar that includes most relevant events in HCI.


With their current work Rikke and Mads pursue a mission to create a new and free resource for teaching and learning interaction design and HCI. In a first step they work with researchers (like myself) around the world that are experts on a certain topic (in my case context-awareness and implicit interaction) to create new teaching materials. This includes a chapter (about 3000 words) that has tutorial character and interviews in which specific topics are discussed in more details.

It was great fun to work with them and I look forward to seeing the new material online.

Opening Keynote of Mobile HCI by Patrick Baudisch

Patrick Baudisch presented the opening Keynote. He gave an interesting overview of work on interaction with small screen devices. It is very apparent that currently the form factor of devices is nearly exclusively determined by the I/O components. Throughout the examples it became very explicit, that the usefulness of a device for certain tasks it determined by the options for interaction. E.g. by increasing the input precision or by reducing the fat finger problem new tasks (and eventually new applications) will become possible on mobile devices. Patrick showed an example where he asks his students, to design a UI for a device as small as a one Euro coin (e.g. digital jewelry) - I think I borrow this idea for our next course :-)

How showed many examples of his work - too many to write it in a blog entry - but well documented on his web page. It is a page worthwhile to check regularly as his group is outstandingly creative and productive.

Patrick made an insightful analogy between small screens and theater stage, arguing that the both have a similar problem: the lack space to show everything you would like to show due to a constraint space. The solutions in the theater can give a good inspiration for designing mobile UIs. He mentioned three typical solutions used in the theater:

• Partially out of the frame - you show something only to a small part and the user's imagination will fill in for the rest. In the theater an example is a ocean liner where you only see a very small part but people imagine the rest. The halo visualization technique [1] is one example for this approach on a mobile device.
• In and out points - people enter and leave the stage at given points providing an illusion of a comprehensible world around the stage (and in the UI case around the screen)
• Direction of interaction to invisible targets - by talking/pointing/looking into a certain direction actors can create an impression of interacting with someone at a certain location - who is not really there and who's position is off the stage.

Technically I found the video he showed on using a depth cam extremely exciting. He speculated that this may solve many problems we have currently with using computer vision in difficult light conditions. In his example he showed the image recorded by a depth cam (based on time of flight) "filming" into direct sunlight - and as one would expect the sun does not matter. Looking forward to seeing some more results on this…

Looking at the current re-appearance of watch based computers and phones the idea of back side interaction becomes more and more interesting. In one design he showed how one can interact with a watch size device by interacting on buckle of the watch band. For more on back of device interaction see [2].

[1] Baudisch, P. and Rosenholtz, R. 2003. Halo: a technique for visualizing off-screen objects. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA, April 05 - 10, 2003). CHI '03. ACM, New York, NY, 481-488. DOI= http://doi.acm.org/10.1145/642611.642695

[2] Baudisch, P. and Chu, G. 2009. Back-of-device interaction allows creating very small touch devices. In Proceedings of the 27th international Conference on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009). CHI '09. ACM, New York, NY, 1923-1932. DOI= http://doi.acm.org/10.1145/1518701.1518995

Creativity Workshop at NRC Tampere

Creativity is a key issue of creating novel applications and interaction methods and techniques. Over two days we ran a hands on prototyping workshop on physical user interaction. We setup teams of 3 people, each team including at least one person with design and one with programming skills. Within about 5 hours the teams had to create a multiplayer interactive game - using a mouse or several mice as basic sensors. We discussed how novelty and the learning curve of interaction technologies relate to physical interaction. The results of the workshop were most impressive… and I think some of them could be really pushed further.
In 2008 there was an interesting issue of Interactions Magazine on innovation. The cover article by Dubberly provides a good conceptual models of innovation [1]. In my lecture on HCI I tyically also introduce the TRIZ ("The theory of solving inventor's problems") methodology, introduced by the Soviet engineer and researcher Genrich Altshuller. TRIZ is interesting to me, as it is in contrast to many other creativity approaches,a systemtic (algorithmic) approach for generating innovative ideas and solutions for problem solving. However I am not sure how well it works in the real world...
During the workshop our approach was very practical. We did also a speed invention exercise - the task was to create 3 game concepts (related to physical penny games) within 10 minutes. The results were pretty impressive - perhaps someone has the time to implement them.

And being in Finland we got the real sauna experience - inside +79°C and then outside into the snow -23° - this is cool (in the very literal sense!).

[1] Dubberly, H. 2008. Toward a model of innovation. interactions 15, 1 (Jan. 2008), 28-36. DOI= http://doi.acm.org/10.1145/133

Visiting TU-Berlin and T-Labs

We have a number of student projects that look at novel applications and novel application platforms on mobile phones. As Michael Rohs from T-Labs is also teaching a course on mobile HCI we thought it would be a good opportunity to meet and discuss some application ideas.

I gave a talk in Michael's lecture discussing the concept of user interfaces beyond the desktop, context as enabling technology, and future applications in mobile, wearable and ubiquitous computing. We had an interesting discussion - and in the end it always comes down to privacy and impact on society. I see this as a very positive development as it shows that the students are not just techies but that they see the bigger picture - and the impact (be it good or bad) they may have with their developments. I mentioned to books that are interesting to read: the transparent society [1] and total recall [2].

In the afternoon we discussed two specific projects. One was an application for informal social while watching TV (based on a set iconic communication elements) that can be used to generate meta data on the program shown. The other is a platform that allows web developers to create distributed mobile applications making use of all the sensors on mobile phones. It is essential a platform an API that provides access to all functions on the phones available in S60 phones over a RESTful API, e.g. you can use a HTTP call to make a photo on someone's phone. We hope to release some of the software soon.

In the coffee area at T-labs was a printout with the 10+1 innovation principles - could not resist to take a photo ;-) Seems innovation is really trival - just follow the 11 rules and you are there ;-)

[1] David Brin. The Transparent Society: Will Technology Force Us to Choose Between Privacy and Freedom. Basic Books. 1999. ISBN-13: 978-0738201443. Amazon-link. Webpage: http://www.davidbrin.com/transparent.htm

[2] Gordon Bell, Jim Gemmell. Total Recall: How the E-Memory Revolution Will Change Everything. Dutton Adult. 2009. ISBN-13: 978-0525951346. Amazon-link. Webpage: http://totalrecallbook.com/

Land of ideas - Award for the innovation factory at the University of Duisburg-Essen

Every day per year one place or institution is Germany is awarded "a place in the land of ideas". This initiative is quite completive (each year well over 2000 institutions apply and the year has only 365 days…).

The innovation factory at the University of Duisburg-Essen has a really interesting vision and received the award for its concept of converting ideas into innovations and products. It promotes thinking in product rather and by this challenges researchers to look how their research results could enable new products or enhance existing products. I find this a most useful exercise - especially when having basic research results that are not directly linked to an application. To facilitate thinking in products they have a number of industrial designers that act as innovation scouts.

Such events always offer the official speeches (not my favorite part but I see one needs them) but also the chance to meet interesting people. I enjoyed the discussions - in particular one question I found interesting - whether or not the ebook reader based on eInk (now just entering the marked on massive scale) is already dead… and replace by the next generation of multimedia extra-thin wireless tablets.