Golden Doctorate – 50 years since Prof. Gunzenhäuser completed his PhD

It is 50 years now that Prof. Rul Gunzenhäuser, my predecessor on human computer interaction and interactive systems at the University of Stuttgart, defended his PhD. Some month back I came across his PhD thesis “Ästhetisches Maß und ästhetische Information“ (aesthetic measure and aesthetic information) [1], supervised by Prof. Max Bense, and I was seriously impressed.

He is one of the few truly interdisciplinary people I know. And in contrast to modern interpretations of interdisciplinary (people from different working together) he is himself interdisciplinary in his own education and work. He studied Math, Physics and Philosophy, worked while he studied in a company making (radio) tubes, completed a teacher training, did his PhD in Philosophy but thematically very close to the then emerging field of computer science and became later a post-doc in the computing center. He taught didactic of mathematics in a teacher training University, was a visiting professor at the State University of New York and finally became in 1973 professor for computer science at the University of Stuttgart staring the department of dialog systems. This unique educational path shaped his research and I would expect his whole person. Seeing this career path I have even more trouble accepting the streamlining of our educational system and find it easier to relate to a renaissance educational ideal.

Yesterday evening we had a small seminar and gathering to mark the 50th anniversary of his PhD. Our colleague Prof. Catrin Misselhorn, a successor on the chair of philosophy held by Max Bense, talked about “Aesthetic as Science?” (with a question mark) and started with the statement that what people did in this area 50 years ago is completely dated, if not largely wrong. I found the analysis very interesting and enlightening as it highlights that scientific results, to be relevant, do not have a non-transient nature. For a mathematician this may be hard to grasp, but for someone in computing and especially in human computer interaction this is a relief. It shows that scientific endeavors have to be relevant in their time but the lasting value may be specifically in the fact, that they go a single step forward. Looking back a human computer interaction a lot of the research in 70ties, 80ties, and 90ties looks now really dated, but we should not be fouled, without this work we would not be in interactive systems where we are now, if this work would not have been done.


Prof. Frieder Nake, one of the pioneers of generative art and a friend and colleague of Prof. Gunzenhäuser, reflected on the early work of computers and aesthetics and on computer generated art. He too argued the original approach is 'dead', but the spirit of computer generated art is stronger now than ever, with many new tools available. He described early and heated discussions between philosophers, artists, and people who made computer generated art. One interesting approach to solve the dispute is is that the computer generated art is “artificial art” (künstliche Kunst).

The short take away message from the event is:
If you do research in HCI, do something that is fundamentally new. Question the existing approach and creates new ideas and concepts. Don’t worry if this will last forever, accept that your research will likely be 'only' one step along the way. It has to be relevant when it is done, it matters less that it may have little relevance some 20 or 50 years later.

[1] Rul Gunzenhäuser. Ästhetisches Maß und ästhetische Information. 1962.

Share your digital activities on Android - AppTicker

If you share an apartment with a friend you know what they do. There is no need to communicate “I am watching TV” or “I am cooking” as this is pretty obvious. In the digital space this is much more difficulty. Sharing what we engage with and peripherally perceive what others do is not yet trivial.

Niels Henze and Alireza Sahami in our group have made a new attempt to research how to bridge this gap. With the AppTicker for Android they have released a software, that offers means to share usage of applications on your phone with your friends on Facebook. You can choose that whenever you start a certain app (e.g. the web browser, the camera, or the public transport app) this is shared in your activities on Facebook. In the middle screen you can see the means for control.

The app provides additionally a personal log (left screen) of all the apps that were used. I found that feature quite interesting and when looking at it I really started to reflect on my app usage patterns. If you are curious, have an android phone and if you use Facebook, please have a go and try it out.

The App homepage on our server: http://projects.hcilab.org/appticker/
Get it directly from Google Play or search for AppTicker in Google Play.

To access it directly you can scan the following QR-Code:

Our Research at CHI2012 - usable security and public displays

This year we have the chance to share some of our research with the community at CHI2012. The work focuses on usable security ([1] and [2]) and public display systems [3]. Florian got together with the researchers from T-Labs a best paper award for [3].

Please have a look at the papers… I think it is really worthwhile.

Increasing the security of gaze-based graphical passwords [1]
"With computers being used ever more ubiquitously in situations where privacy is important, secure user authentication is a central requirement. Gaze-based graphical passwords are a particularly promising means for shoulder-surfing-resistant authentication, but selecting secure passwords remains challenging. In this paper, we present a novel gaze-based authentication scheme that makes use of cued-recall graphical passwords on a single image. In order to increase password security, our approach uses a computational model of visual attention to mask those areas of the image that are most likely to attract visual attention. We create a realistic threat model for attacks that may occur in public settings, such as filming the user's interaction while drawing money from an ATM. Based on a 12-participant user study, we show that our approach is significantly more secure than a standard image-based authentication and gaze-based 4-digit PIN entry." [1]

Assessing the vulnerability of magnetic gestural authentication [2]

"Secure user authentication on mobile phones is crucial, as they store highly sensitive information. Common approaches to authenticate a user on a mobile phone are based either on entering a PIN, a password, or drawing a pattern. However, these authentication methods are vulnerable to the shoulder surfing attack. The risk of this attack has increased since means for recording high-resolution videos are cheaply and widely accessible. If the attacker can videotape the authentication process, PINs, passwords, and patterns do not even provide the most basic level of security. In this project, we assessed the vulnerability of a magnetic gestural authentication method to the video-based shoulder surfing attack. We chose a scenario that is favourable to the attacker. In a real world environment, we videotaped the interactions of four users performing magnetic signatures on a phone, in the presence of HD cameras from four different angles. We then recruited 22 participants and asked them to watch the videos and try to forge the signatures. The results revealed that with a certain threshold, i.e, th=1.67, none of the forging attacks was successful, whereas at this level all eligible login attempts were successfully recognized. The qualitative feedback also indicated that users found the magnetic gestural signature authentication method to be more secure than PIN-based and 2D signature methods."[2] There is also a youtube video: http://www.youtube.com/watch?v=vhwURyTp_jY

Looking glass: a field study on noticing interactivity of a shop window[3]
"In this paper we present our findings from a lab and a field study investigating how passers-by notice the interactivity of public displays. We designed an interactive installation that uses visual feedback to the incidental movements of passers-by to communicate its interactivity. The lab study reveals: (1) Mirrored user silhouettes and images are more effective than avatar-like representations. (2) It takes time to notice the interactivity (approx. 1.2s). In the field study, three displays were installed during three weeks in shop windows, and data about 502 interaction sessions were collected. Our observations show: (1) Significantly more passers-by interact when immediately showing the mirrored user image (+90%) or silhouette (+47%) compared to a traditional attract sequence with call-to-action. (2) Passers-by often notice interactivity late and have to walk back to interact (the landing effect). (3) If somebody is already interacting, others begin interaction behind the ones already interacting, forming multiple rows (the honeypot effect). Our findings can be used to design public display applications and shop windows that more effectively communicate interactivity to passers-by." [3]



References
[1] Andreas Bulling, Florian Alt, and Albrecht Schmidt. 2012. Increasing the security of gaze-based cued-recall graphical passwords using saliency masks. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 3011-3020. DOI=10.1145/2208636.2208712 http://doi.acm.org/10.1145/2208636.2208712
[2] Alireza Sahami Shirazi, Peyman Moghadam, Hamed Ketabdar, and Albrecht Schmidt. 2012. Assessing the vulnerability of magnetic gestural authentication to video-based shoulder surfing attacks. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 2045-2048. DOI=10.1145/2208276.2208352 http://doi.acm.org/10.1145/2208276.2208352
[3] Jörg Müller, Robert Walter, Gilles Bailly, Michael Nischt, and Florian Alt. 2012. Looking glass: a field study on noticing interactivity of a shop window. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 297-306. DOI=10.1145/2207676.2207718 http://doi.acm.org/10.1145/2207676.2207718

Introduction to the special issue on interaction beyond the desktop

After coming back from CHI2012 in Austin I found my paper copy of the April 2012 issue of IEEE Computer magazine in my letter box. This is our special issue on interaction beyond the desktop. Having the physical copy is always nice (it is because I probably grew up with paper magazines ;-).

This guest editors’ introduction [1] is an experiment as we include photos from all papers on the theme. The rational is, that probably most people will not have the paper copy in their hand. When having the digital version the overview of the papers is harder to manage, that is why we think including the photos helps to make readers curious to look at the papers in the issue. Please let us know if you think this is a good idea…

[1] Albrecht Schmidt and Elizabeth Churchill. Interaction Beyond the Keyboard. IEEE Computer, April 2012, pp. 21–24. (PDF). Link to the article in Computing Now.

Book launch: Grounded Innovation by Lars Eric Holmquist

At the Museum of the Weird in Austin Lars Erik Holmquist hosted a book launch party for his book: Grounded Innovation: Strategies for Creating Digital Products. The book uses a good number of research examples to highlight the challenges and approaches for digital products. The book has to parts: Methods and Materials and shows how both play together in the design of digital products. There is a preview for the book at Amazon.

Over 10 years back I worked together with Lars Erik on the European Project Smart-Its (http://www.smart-its.org/), where we created sensor augmented artifacts. The book features also some of this work. To get an overview of the project have a look at [1] and [2]. The concept of Smart-Its Friends is presented in [3]. Smart-Its friends proposed the idea, that products can be linked by sharing the same context (e.g. connecting a phones and a wallet by shaking them together).

[1] Lars Erik Holmquist, Hans-Werner Gellersen, Gerd Kortuem, Albrecht Schmidt, Martin Strohbach, Stavros Antifakos, Florian Michahelles, Bernt Schiele, Michael Beigl, and Ramia Maze;. 2004. Building Intelligent Environments with Smart-Its. IEEE Comput. Graph. Appl. 24, 1 (January 2004), 56-64. (PDF) DOI=10.1109/MCG.2004.1255810 http://dx.doi.org/10.1109/MCG.2004.1255810

[2] Hans Gellersen, Gerd Kortuem, Albrecht Schmidt, and Michael Beigl. 2004. Physical Prototyping with Smart-Its. IEEE Pervasive Computing 3, 3 (July 2004), 74-82. (PDF) DOI=10.1109/MPRV.2004.1321032 http://dx.doi.org/10.1109/MPRV.2004.1321032

[3] Lars Erik Holmquist, Friedemann Mattern, Bernt Schiele, Petteri Alahuhta, Michael Beigl, and Hans-Werner Gellersen. 2001. Smart-Its Friends: A Technique for Users to Easily Establish Connections between Smart Artefacts. In Proceedings of the 3rd international conference on Ubiquitous Computing (UbiComp '01), Gregory D. Abowd, Barry Brumitt, and Steven A. Shafer (Eds.). Springer-Verlag, London, UK, UK, 116-122. (PDF)

CHI2012 opening Keynote by Margaret Gould Stewart – Empowerment, Disruption, Magic

Margaret Gould Stewart, a highly regarded user experience designer currently leading UX design at YouTube, presented the opening keynote at CHI2012.  She started her talk with reminding us that humans are story tellers – they always have been and probably always will. What is not constant is the medium – as technologies change so do means for storytelling and sharing.

The topic started out with talking about video connects the world. It extended to a larger view – changing the world through experience design (in the context of video). I often wonder what designers are and she added another quite interesting explanation: designers are humanist. By putting up the definition for humanism she made her point clear that this could apply to good people in design, essentially it is down to caring for humans in their works.

To show the power of video in connecting people she used the following example: the film “Life in a Day” and as it said in the credits “a movie filmed by you”. I have not seen it yet, but the trailer made me curious to look at this one (see the film on YouTube).

By asking the question: what are the things that make sites like YouTube have impact? she introduced 3 principles. Sites have to be:

• Empowering
• Disruptive
• Magical

She outlined what these 3 principles mean for user experience design.

For empowering she had very strong examples: how photo sharing, video sharing, and social networks changed what we see of natural disaster and the effect on people. It also changed way we see it and how we can respond to it. The concrete example was the information coverage on the Hurricane Katrina 2005 (pre-video-sharing age) and the recent flood in Asia. Empowering = helping people to share their stories.

Disruption is in this context the change in use of media and especially how it changes how we perceive the ubiquitous technology of TV. The capabilities of video sharing platforms has, are very different than those of TV – at the same time it is disrupting TV massively. She had a further example of how such technology can disrupt: The Khan Academy (basically sharing educational videos) is challenging the education system. As a further step she had an example where a teacher encourages students to make their own instructional videos as means for them to learn. Disruption = finding new ways that are challenging / overthrowing the old approach.

Magic is what makes technology exciting. There is a quote by Arthur C. Clarke “Any sufficiently advanced technology is indistinguishable from magic”. The term “magic” has a long tradition in human computer interaction. Alan Kay talked about it with regard to graphical user interfaces. We had some years back a paper  a paper on Magic beyond the screen [1]. In the talk Margaret Gould Stewart used as another example Instagram, as software that provides magical capabilities for the person using it. Another example of magic she discussed is the GPS based “moving dot” on a map that makes navigation in mobile maps easy. Even without navigational skills people can “magically” find their way. Her advice is “do not get in the way of magic” – focus on the experience not technology in the back ground. In short she summarized:  “Magic disrupts the notion of reality”.


She combined the principles in one example in the design of YouTube. She discussed the page design using an analogy to a plate.  A great plate makes all food presented on it look more attractive and the design goal of the YouTube page is to be such a plate for video. It should make look all videos look better.

Another example used to highlight how to empower, disrupt, and create magic is the http://www.thejohnnycashproject.com/. Each participant can manipulate one frame of the video (within given limits) and the outcome of the whole video is amazing. Cannot be described, you have to watch it.

Related to the example above an interesting question comes up: How much control is required and what type of control is applied. Here one example is twitter, which limits how much you can write but it does not limit what you post (limiting the form but not the content). She made an interesting argument about control. If you believe that democracy works and is good you can assume that people in general will make the right decisions. One further indicator is, that positive things go viral much more often than negative things. One of the takeaway messages is to believe in people an empower them.

To sum up, there are three questions to be asked when designing an experience:

• How to empower people?
• How to disrupt
• How to create magic?

A final and important point is that there are things that cannot be explained and she argued that we should value this.

[1]  Albrecht Schmidt, Dagmar Kern, Sara Streng, and Paul Holleis. 2008. Magic Beyond the Screen. IEEE MultiMedia 15, 4 (October 2008), 8-13. DOI=10.1109/MMUL.2008.93 http://dx.doi.org/10.1109/MMUL.2008.93

Keynote at Percom 2012: Andy Hopper from Cambridge on Computing for the Future of the Planet

In his Keynote "Computing for the Future of the Planet" Andy Hopper brought up 4 topics and touched shortly on each of them: (1) Optimal digital infrastructure – green computing, (2) Sense and optimize – computing for green, (3) Predict and react – assured computing, and (4) Digital alternatives to physical activities.

In the beginning of his talk he discussed an interesting (and after he said it very obvious) option of Green Computing: move computing towards the energy source as it is easier to transmit data than to transmit power. Thinking about this I could imagine that Google’s server farms are move to a sunny dessert and then the calculations are done while the sun is shining… and using the cold of night to cool down… This could be extended to storage: storing data is easier than storing energy – this should open some opportunities.

As a sample of an embedded sensing systems Andy Hopper presented a shoe with built-in force sensing (FSR) that allows to measure contact time and this helps to work out speed. There initial research was targeted towards athletes, see Rob Harle's page for details. It is however easy to imagine the potential this has if regular shoes can sense movement in everyday use. He hinted to think about the options if one could go to doctor and analyze the change in walking pattern over the last year.

In various examples Andy showed how Ubisense is used in commercial applications, production, and training. It seems that medium resolution tracking (e.g. below 1 meter accuracy) can be reliably achieved with such an off the shelf systems, even in harsh environments. He mentioned that the university installations of the system at an early product stage were helpful to improve the product and grow the company. This is interesting advices, and could be a strategy for other pervasive computing products, too. For close observers of the slides there were some interesting inside in the different production methods between BMW and Austin Martin and the required quality ;-)

Power usage is a central topic in his labs work and he showed several examples of how to monitor power usage in different scenarios. On example is monitoring power usage on the phone, implemented as an App that looks at how power is consumed and how re-charging is done. This data is then collected and shared – at current over 8000 people are participating. For more details see Daniel T. Wagner’ page. A further example is the global personal energy meter. He envisions that infrastructure, e.g. trains and building, are broadcasting information about the use of energy and that they provide information about one individuals share of this.

With an increasing proliferation of mobile phones the users’ privacy becomes a major issue. He showed in his talk an example, where privacy is provided by faking data. In this approach fake data, e.g. for calendar events, location data, and address book, is provided to apps on the phone. By these means you can alter what an application sees (e.g. location accuracy).

For more details and papers see the website of the digital technology group: http://www.cl.cam.ac.uk/research/dtg/www/

Opening talk at the Social Media for Insurances Symposium

I was invited to Leipzig to talk about social networks and in the context of insurance companies (http://www.versicherungsforen.net/social-media). The main focus of the talk was to show what people currently do in social networks and to speculate why they do it (and  I used a picture of the seven deadly sins as an illustrations…) Additionally I discussed some prototypes of activity recognition and their potential once integrated into social media.

My talk was entitled “500 Freunde (auf Facebook): Wozu noch eine Versicherung?“ – „500 friends (on Facebook) – Is there still need for insurance?“ and discussed how ubiquitous capture and social media may shape the next community [1]. The slides in are in German.

The event was very interesting and I would expect that there is a great potential out there for insurance companies to tap into. Looking back at the original idea of insurance (e.g. old fire insurance communities) or sharing the risk of hail in farming communities can give interesting inspiration for peer-2-peer insurance models. It will be exciting to see if there a new products and services that come out of the “big players” or if new players will come to the game. To me the central issue to address is how to make insurance products more visible – and I think a user centered design approach could be very interesting…

In the future I would expect that finding the right value mix (privacy, price, safety, etc.) will be essential as we argued for other services in [2]. Some years back we wrote in an article about RFID [3] “privacy is sacred but cheap” and the more services we see the more I am convinced that this is more than a slogan. If you can create a service that is of immediate value to the user I would I expect that privacy will be a lesser concern to most? On the other hand if you reduce privacy without any value in exchange there is always an outcry…

[1] “500 Freunde (auf Facebook): Wozu noch eine Versicherung?“ - Ermöglichen allgegenwärtige Aufzeichnungs-technologien und digitale soziale Netze die nächste Solidargemeinschaft? Slides as PDF (in German)
[2] Albrecht Schmidt, Marc Langheinrich, Kristian Kersting, "Perception beyond the Here and Now," Computer, vol. 44, no. 2, pp. 86-88, Feb. 2011, doi:10.1109/MC.2011.54 (final version at IEEE, free draft version)
[3] Schmidt, A.; Spiekermann, S.; Gershman, A.; Michahelles, F., "Real-World Challenges of Pervasive Computing", Pervasive Computing, IEEE , vol.5, no.3pp. 91- 93, c3, July-Sept. 2006. 10.1109/MPRV.2006.57

Facebook – a platform to spot when companies go bankrupt? Real world example.

In the Germany the drug store chain Schlecker announced to be insolvent, see the Reuter news post. If you look at the company’s Facebook page and scan the comments from the last 4 weeks it is apparent that some people in the crowd and employees expected it already last year.
Schlecker is a large drug store chain with probably over 10.000 outlets in Europe and more than 30.000 employees.

The following screen shots show some selected examples I took from the following page: http://www.facebook.com/schlecker.drogerie 
The posts are in German – the minimal summary should give you some idea…

This one the company wishes a happy Christmas and reminds people of a chance to win a car. The first replies echo the holiday greetings but then one complains that they let their shops bleed out (run empty) and that the order good do not arrive (probably posted by an employee). One further speculates that the company is close to bankruptcy. (over 3 weeks before the official note of insolvency)





The company announces a 2 euro discount on a product. Then employees post that they would like to sell the goods to the customers but that they do not get the goods for their shops. Additionally they complain that the goods they get from other closed down shows are not what they need. One says we want to work but we cat (as they are running out of stock). (over 2 weeks before the official note of insolvency)


The company announces price reductions in some goods. Some says that is great – but would be much better if these goods would be in the shops to buy them. (9 days before the official note of insolvency)


Overall I think this is an instructive real world example of the information that can be found in social networks about the health/value of companies. In particular the mix of customers and employees posting makes it a good example to study. I would expect that companies will learn lessons from this with regard to guidelines for the employees… and about transparency / openness…to understand how reliable such posts are we probably need to do some more research? let us know if you are interested in working this with us.

Congratulations to Frau Doktor Dagmar Kern for a great PhD defense (No. 5)

Dagmar Kern has successfully defended her PhD on “Supporting the Development Process of Multimodal and Natural Automotive User Interfaces” in Essen. External examiner was Antonio Krüger from University of Saarbrücken. Her dissertation will be available online soon. The core contribution of the thesis is the investigation of how to improve a user centered design process for automotive user interfaces. In order to systematically assess user interface designs in cars she developed a design space (inspired by Card et al [5]). In various cases studies she create novel in-car user interfaces and explored experimentally the implications on driver distraction.

Dagmar started working with me as a student of Media Informatics at the LMU Munich in 2005, then jointed my group at Fraunhofer IAIS/BIT in Bonn and move in 2007 with the group to Essen. She was for a short research stay in Saarbrücken and Milton Keynes and was extremely productive over the last years – 18 publications she co-authored are listed in DBLP and here are some highlights of here research:

• exploration of how to present navigation information (e.g. vibra tactile steering wheel) [1]
• gazemarks - an approach to aid attention switching between the road and an in car display using eye gaze date [2]
• a multi-touch steering wheel, that reduced driver distraction [3]
• a design space for automotive user interfaces [4]

Additionally to the publications one of the side products of here thesis is the CARS open source driving simulator. It is a configurable low cost simulator that can be used to measure driver distraction, e.g. as an alternative to LCT.

Dagmar’s defense brought us back to Essen and it was great to meet many colleagues again. We finally managed to have a group photo taken with nearly all the team (Elba is missing in the Photo).

The doctoral hat may look strange to non-Germans but it has some funny tradition. It is hand crafted by the colleagues and each of the items on the hat tells a story – usually known to the group but in the best case hard to guess for outsiders. Besides others Dagmar’s hat included a scrap heap of cars, a giraffe, a personal vibration device, a yoyo, a railway station building side, and a steering wheel cover.

[1] Dagmar Kern, Paul Marshall, Eva Hornecker, Yvonne Rogers, and Albrecht Schmidt. 2009. Enhancing Navigation Information with Tactile Output Embedded into the Steering Wheel. InProceedings of the 7th International Conference on Pervasive Computing (Pervasive '09). Springer-Verlag, Berlin, Heidelberg, 42-58. DOI=10.1007/978-3-642-01516-8_5 (free PDF)

[2] Dagmar Kern, Paul Marshall, and Albrecht Schmidt. 2010. Gazemarks: gaze-based visual placeholders to ease attention switching. In Proceedings of the 28th international conference on Human factors in computing systems (CHI '10). ACM, New York, NY, USA, 2093-2102. DOI=10.1145/1753326.1753646 (free PDF)

[3] Tanja Döring, Dagmar Kern, Paul Marshall, Max Pfeiffer, Johannes Schöning, Volker Gruhn, and Albrecht Schmidt. 2011. Gestural interaction on the steering wheel: reducing the visual demand. In Proceedings of the 2011 annual conference on Human factors in computing systems (CHI '11). ACM, New York, NY, USA, 483-492. DOI=10.1145/1978942.1979010 (free PDF)

[4] Dagmar Kern and Albrecht Schmidt. 2009. Design space for driver-based automotive user interfaces. In Proceedings of the 1st International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '09). ACM, New York, NY, USA, 3-10. DOI=10.1145/1620509.1620511 (free PDF)

[5] Stuart K. Card, Jock D. Mackinlay, and George G. Robertson. 1991. A morphological analysis of the design space of input devices. ACM Trans. Inf. Syst. 9, 2 (April 1991), 99-122. DOI=10.1145/123078.128726

Auto-UI 2012 in the US, looking for hosts for 2013

The next and 4rd international conference on Automotive User Interfaces and Vehicular Applications (AutoUI 2012) will be in Portsmouth, New Hampshire in the USA. The dates for the conference are 17-19 of October 2012. The first day for workshops and tutorials and 2 days for the main conference. Portsmouth is about an 1 hour drive from Boston and the timing is great (fall foliage – the photos of the colorful forests looked good ;-)

The steering committee (sc@auto-ui.org) is inviting proposals for Auto-UI 2013 from the community of researchers in the field. The conference was 2009 in Essen (Germany), 2010 in Pittsburgh (USA), 2011 in Salzburg (Austria), and it will be in 2012 in Portsmouth (USA). Keeping this cycle between Europe and North America 2013 should be in Europe.

Paper and demo in Salzburg at Auto-UI-2011

At the automotive user interface conference in Salzburg we presented some of our research. Salzburg is a really nice place and Manfred and his team did a great job organizing the conference!

Based on the Bachelor Thesis of Stefan Schneegaß and some follow-up work we published a full paper [1] that describes a KLM-Model for the car and a prototyping tools that makes use of the model. In the model we look at the specific needs in the car, model rotary controllers, and cater for the limited attention while driving. The prototyping tool provides means to quickly estimate interaction times. It supports visual prototyping using images of the UI and tangible prototyping using Nic Villar´s VoodooIO. Looking forward to having Stefan on our team full-time :-)

We additionally had a demo on a recently completed thesis by Michael Kienast. Here we looked at how speech and gestures can be combined for controlling functions, such as mirror adjustments or windscreen wipers, in the car. This multimodal approach combines the strength of gestural interaction and speech interaction [2].

The evening event of the conference was at Festung Hohensalzburg - with a magnificent view over the twon!

[1] Stefan Schneegaß, Bastian Pfleging, Dagmar Kern, Albrecht Schmidt. Support for modeling interaction with in-vehicle interfaces. (PDF) Proceedings of 3rd international conference on Automotive User Interfaces and Vehicular Applications 2011 (http://auto-ui.org). Salzburg. 30.11-2.12.2011

[2] Bastian Pfleging, Michael Kienast, Albrecht Schmidt. DEMO: A Multimodal Interaction Style Combining Speech and Touch Interaction in Automotive Environments. Adjunct proceedings of 3rd international conference on Automotive User Interfaces and Vehicular Applications 2011 (http://auto-ui.org). Salzburg. 30.11-2.12.2011

Bryan Reimer: Opening keynote at Auto-UI 2011 in Salzburg

Bryan started his keynote talk the automotive user interface conference (auto-ui.org) in Salzburg with reminding us that having controversial discussions about the HMI in the car is not new. Quoting a newspaper article from the 1930s on the introduction of the radio in the car and its impact on the driver he picked an interesting example, that can be seen as the root of many issues we have now with infotainment systems in the car.

The central question he raised is: how to create user interface that fit human users? He made an important point: humans are not “designed” to drive at high speed in complex environments; perception has evolved for walking and running in natural environment. Additionally to the basic limitations of human cognition, there is a great variety of capabilities of drivers, their skills and cognitive ability (e.g. influence of age). A implication of the global change is demographics is that the average capabilities of a drivers will be reduced – basically as many older people will be drivers…

Over the last 100 years cars have changes significantly! Looking more closely Bryan argues that much of the chance happened in the last 10 years. There has been little change from the 1950s to the 1990s with regard to the car user interface.

It is apparent that secondary tasks are becoming more important to the user. Users will interact more while driving because the can. It is however not obvious that they are capable of it.

Even given these developments it is apparent that driving has become safer. Passive safety has been improved massively and this made driving much safer. There seems to be a drawback to this as well, as people may take greater risks as they feel safer. The next step is really to avoid accidence in the first place. Bryan argues that the interaction between driver, environment, and vehicles is very important in that. He suggests that we should make more of an effort to create systems that fit the drivers.

The Yerkes-Dodson Law helps to understand how to design systems that keep peoples attention in the optimal performance. He made an important point: there are certain issues that cannot be solved, e.g. if someone is tired we can do only very little – the driver will need to rest. We should make sure that we take these things into account when designing systems.

Visual distraction is an obvious factor and much discussed in the papers at the conference – but Bryan argued that “eyes on the road” is not equal to “mind on the road”. I think this is really a very important point. Ensuring that people keep their eyes on the road, seeing things is not enough. The big resulting question is how to keep or get people focused on the street and environment. It seems there is some more research to do…

The variety of interfaces and interaction metaphors build into cars opens more choices but at the same time creates problems, as people need to learn and understand them. A simple question such as: How do you switch the car off? may be hard to answer (Bryan had the example of a car with a push button starter, where you cannot remove the key). I think there are simple questions that can be learned from industry and production machines… add an emergency stop button and make it mandatory ;-)

If you are interested more about Bryan’s work look at his webpage or his page at the MIT agelab or one of his recent publications [1] in the IEEE Pervasive Computing Magazine’s special issue on automotive computing, see [2] for an introduction to the special issue.

Sorry for the poor quality photos ... back row and an iPhone...

[1] Joseph F. Coughlin, Bryan Reimer, and Bruce Mehler. 2011. Monitoring, Managing, and Motivating Driver Safety and Well-Being. IEEE Pervasive Computing 10, 3 (July 2011), 14-21. DOI=10.1109/MPRV.2011.54 http://dx.doi.org/10.1109/MPRV.2011.54

[2] Albrecht Schmidt, Joseph Paradiso, and Brian Noble. 2011. Automotive Pervasive Computing. IEEE Pervasive Computing 10, 3 (July 2011), 12-13. DOI=10.1109/MPRV.2011.45 http://dx.doi.org/10.1109/MPRV.2011.45

Automotive UI in Salzburg

Manfred Tscheligi opend the Automotive UI conference in Salzburg. The conference is now in its 3rd year after 2009 in Essen and 2010 in Pittsburgh. The conference is growing – there were well over 130 people registered :-)

The proceedings of the conference series are online available at http://auto-ui.org

3rd Auto-UI Proceedings 2011 (soon in the ACM DL)
2nd Auto-UI Proceedings 2010 (ACM DL)
1st Auto-UI Proceedings 2009 (ACM DL)

Guests in my multimodal interaction class

Today I had brought 3 more professors with me to teach the class on multimodal interaction (I learned from Hans). As we have the pd-net project meeting Nigel Davies, Marc Langheirich, and Rui Jose were in Stuttgart and ‘volunteered’ to give a talk.

Nigel talked about the work in Lancaster on the use of mobile computing technology to support sustainable travel. He explained the experiments they conducted for collecting and sharing travel related information. In the 6th Sense Transport project they look beyond looking at understanding the current context into predictions and eventually ‘time travel’ ;-)



Marc presented a one hour version of his tutorial on privacy introducing the terminology and explaining the many facets this topic has. We discussed the ‘NTHNTF’ argument (Nothing To Hide Nothing To Fear) and Marc used an example of AOLstalker.com to show the weaknesses of this argument. Marc suggested some reading if you want to dive into the topic, see [1,2,3,4].

Rui focused in his lecture on pervasive public displays. He gave an overview of typical architectures for digital signage systems and the resulting limitation. The pd-net approach aims at creating an open platform that allows many different applications and use cased. He showed once concept of using virtual pin-badges to trigger content and to express interest in a certain topic.

There is more information on the pd-net project on http://pd-net.org

[1] David Brin. The Transparent Society. Perseus Publishing, 1999.
[2] Simson Garfinkel: Database Nation – The Death of Privacy in the 21st Century. O’Reilly, 2001.
[3] Lawrence Lessig: Code and Other Laws of Cyberspace. Basic Books, 2006. http://codev2.cc/
[4] Waldo, Lin, Millett (eds.): Engaging Privacy and Information Technologygy in a Digital Age. National Academies Press, 2007.

Call for Papers: Symposium on Pervasive Display Networks

Rui José and Elaine Huang are chairing an international symposium on pervasive displays in Portugal. The conference will be held June 4-5 2012 in Porto. The submission deadline for full papers is January 16th, 2012.

With our research in the PD-net project we encounter many interesting research questions and met with many other researchers interested in the topic. It seems that the many real deployments of electronic displays is fueling ideas and makes it obvious that research is required to understand the properties of this new upcoming media. The call states: “As digital displays become pervasive, they become increasingly relevant in many areas, including advertising, art, sociology, engineering, computer science, interaction design, and entertainment.”

We hope with this symposium we will bring together researchers and practitioners as well as users to share research results and generate new ideas.

Submissions that report on cutting-edge research in the broad spectrum of pervasive digital displays are invited, ranging from large interactive walls to personal projection, from tablets and mobile phone screens to 3-D displays and tabletops. Topics include:

• Novel technologies
• Architecture
• Applications
• Domains and formative studies studies
• Evaluations and deployments
• Interfaces and interaction techniques
• Content design

Have a look at the webpage and the call for paper at http://pervasivedisplays.org/cfp.php

Hippy, purple hair, piercing, ..., facebook? How to rebell

Inspired by some discussion on the implication of information sharing at AMI2011 I wanted to put this up for discussion here...

If you read newspapers you find a lot of people have major concerns about how young people are using social networks, and especially facebook. I believe many of them are well meaning when the speculate about the lasting damage young people do to themselves if the post too much too openly. Some of the concerns I share but I think there is another dimension to it, too.

Being young (e.g. teenager) is not about being sensible, reasonable, rational - it is about exploring the world and rebelling. Probably most of us looked for way to provoke reactions from parents and society in this phase of live. Some examples to remember... Hippies and sex in 70’s, green, blue, red and orange hair in the 80’s, in then in 90’s it was piercing. Now what can the young people do today? Granny had a skirt that was really short; parents had piercings in places where you don't want to think about…. It is easy - sharing a picture on facebook where you wear to little or nothing - and you get all the reactions. Especially you will get the same reaction that has been around for many decades (and your parent and grandparents got, too): if you do this no one will ever give you a job ;-)

I don't want to deny the risks of sharing information online, but I think we should analyze things a bit more deeply ...

Closing Keynote at AMI2011, Beyond Ubicomp – Computing is Changing the Way we Live

On Friday afternoon I had the privilege to present the closing keynote at AMI2011 in Amsterdam with the title ‘Beyond Ubicomp – Computing is Changing the Way we Live’. The conference featured research in Ambient Intelligence ranging from networking and system architecture to interfaces and ethnography. It brought an interesting set of people together and it was good to see many students and young researchers presenting their work.

In my closing keynote at talked about my experience of the last 13 years in this field and about a vision of the future. My vision is based on a basic technology assessment - basically looking what technologies will (in my view) definitely come over the next 20 years and looking at the implications of this. I stared out with a short reference to Mark Weiser's now 20 year old article [1]. The upcoming issue of IEEE Pervasive Magazine will have a in-depth analysis of the last 20 years since Weiser' article - we have also an article in there on how interaction evolved.

The vision part of the talk looked "Perception beyond there here and now" [2] from 3 different angles:

• Paradigm Shift in Communication
  Here I argue that the default communication in the future will be public communication and only if something is secret we will try to use non public channel. First indicators of this are a switch from email to twitter and facebook. I used a cake baking example to highlight the positive points of this shift.
• Steep Increase in media capture
  The second angle is just observing and extrapolating the increase in capture of media information. If you go already now on youtube you will information about many things (backing a cake, repairing a bike, etc.). The implication of this increase in media capture will be virtually unlimited access to experience other people share
• Transformation of experienced perception
  The final angle is that this creates a new way of perceiving the world. We will extent perception beyond the here and now and this is bringing a completely new way of creating and accessing information. I used the example of enquiring about buying an international train ticket at the station in Amsterdam. If you can look there through other people’s eyes the question becomes trivial.

My overall argument is that we are in for a major transformation of our knowledge and information culture. I would expect that this shift is as radical as the shift from an oral tradition to the written societies - but the transition will be much quicker and in the context of a globalized and competitive world.

The main conclusion from this is: Ethics and values are the central design material of this century.

Looking at twitter it seems it got across to some in the audience ;-) If your are interested, too have a look at the slides from the keynote.

[1] Mark Weiser. The computer for the 21st century. Scientific American, Vol. 265, No. 3. (1991)
[2] Albrecht Schmidt, Marc Langheinrich, and Kritian Kersting. 2011. Perception beyond the Here and Now. Computer 44, 2 (February 2011), 86-88. DOI=10.1109/MC.2011.54 http://dx.doi.org/10.1109/MC.2011.54

Opening Keynote at AMI 2011: Margaret Morris

Margaret (Margi) Morris presented the opening keynote at the 2011 conference on ambient intelligence in Amsterdam (AMI2011) with the title “Left to our own devices”.

Margaret brought up an interesting point on motivation: Showing people what they lose is a stronger motivator than the prospective of gain. She made the point in order to implement this the depicted loss has to be very specific. She showed a facebook application “With a little help from my friends”, where this basic concept is applied.   I had recently seen a bill board adverting campaign for safe driving on motorways in Germany using this approach (basically showing the risk of loss of family).

In the talk several examples of devices and applications were presented. To learn more about her work I recommend the following two papers: at tool to improve emotional self-awareness [1] and an investing in social networks and their utility to promote health [2].

Another point that made me think was the question of how we design interventions. One conceptual example was about an obesity campaign. The official UK campaign starts out with the statement that obesity is a problem for 9 million kids. Her alternative is to provide instead of the information a specific hint about an opportunity for action for an individual (e.g. telling the kid when it leaves school in the afternoon: now is probably a good time to play soccer with your friends, as 16 of them like to play soccer). An open research question that relates to this seems to me to investigate the impact of information about the norm, e.g. how will it affect my behavior if I know that 70% of my friends think driving too fast is OK vs. if I know that only 20% find it acceptable. I think this could be further explored in the context of social networks to create interesting persuasive technologies.


There has been an interesting discussion after the talk. Norbert Streitz questioned if it is a good idea to ask people to engage more with digital devices (e.g. self monitoring one’s mood). The question is hinting that the engagement with the digital device keeps us from interaction in the “real” world. I think this separation is disappearing fast – making a phone call, listening to MP3, chatting with friends on facebook is for many of us real, we live in a world that is augmented by technology and the boundaries are bluring...

[1] Morris ME, Kathawala Q, Leen TK, Gorenstein EE, Guilak F, Labhard M, Deleeuw W. Mobile Therapy: Case Study Evaluations of a Cell Phone Application for Emotional Self-Awareness. Journal of Medical Internet Research 2010;12(2):e10. URL: http://www.jmir.org/2010/2/e10/

[2] Margaret E. Morris. 2005. Social Networks as Health Feedback Displays. IEEE Internet Computing 9, 5 (September 2005), 29-37. DOI=10.1109/MIC.2005.109 http://dx.doi.org/10.1109/MIC.2005.109

Opening at AMI2011: Emile Aarts

Today the conference on ambient intelligence 2011 (AMI2011) has started in Amsterdam. Emile Aarts reflected on what Ambient Intelligence is and was.

He looked back of some of the concepts that were well received in the research community but did not make it to the market (e.g. iCat). He made a very important point on how to involve users in the research: Don’t ask the customer –look at what he is doing. (he attributed it to someone from booking.com but it has been around for a while, e.g. in Jakob Nielsen's Alertbox 2001). Even if this is well known I think it was great to remind us again! It is still the most important points in user centered research that is often completely missed!

He suggested that the most important topic in ambient intelligence is lighting. It seems very simple but he had some very interesting example (e.g. a hospital or an airport) where light has a measureable impact on people’s experience.

For some more impressions of his talk, see the random slide selection below.

Human Computer Confluence consultation in Brussels

FET Procative is running consultation meetings for research calls in 2013. Today (Nov, 11 2011) the meeting is on Human Computer Confluence. This term is interesting and I like the vision, but in general I have an issue with such new terms; It seems this is an European approch to allow a lot of people to be experts... if we would use an established term (e.g. Human Computer Interaction, Cognitive Science, or Artificial Intelligence) it would be easy to see who has expertise in this domain and who does something else or who is not in science at all ;-)

My approach would be to identify the established scientific fields that matter to the research we envision (such as human computer confluence) and find the experts in these fields and give them incentives to collaborate. My best experience in interdisciplinary research is with people who are undisputed experts in one field and work to together with others who are experts in their field. I don't believe in getting together "experts" on a new topic which does not yet exist. As one cannot tell if they are or not (and many are not)...

This meeting on Human Computer Influence seems OK as there are at least a number of experts in established fields who are interested in this new field, hence it is fun to listen to some of the visions.

My vision is based around the idea of "Perception beyond the here and now" [1]. I argue that the transition that can be expected in capture and perception technology is massive. I think the change we are heading for is as fundamental as the transition from oral cultures to written cultures. Having access to all experiences people have ever had are seems to me extremely exciting and at the same time extremely scary.

One central issue I feel strongly about is to give incentives to do high risk and high gain research. So far FET has not been good at allowing risk. None of the 100 projects of the last 5 years explicitly failed. It is clear that not all of them were highly successful - I would expect that typically project, even though the proposals are high risks, they do not carry out the high risk research. Typically the research that is actually done is standard research where researchers are sure that they get enough out to be OK in the review. There is not incentive to do high risk while carrying out the project. I strongly argue that we create incentives to not penalize high risk research while carrying out the research.

[1] Albrecht Schmidt, Marc Langheinrich, and Kritian Kersting. 2011. Perception beyond the Here and Now. Computer 44, 2 (February 2011), 86-88. DOI=10.1109/MC.2011.54 http://dx.doi.org/10.1109/MC.2011.54