Visiting TU-Berlin and T-Labs

We have a number of student projects that look at novel applications and novel application platforms on mobile phones. As Michael Rohs from T-Labs is also teaching a course on mobile HCI we thought it would be a good opportunity to meet and discuss some application ideas.

I gave a talk in Michael's lecture discussing the concept of user interfaces beyond the desktop, context as enabling technology, and future applications in mobile, wearable and ubiquitous computing. We had an interesting discussion - and in the end it always comes down to privacy and impact on society. I see this as a very positive development as it shows that the students are not just techies but that they see the bigger picture - and the impact (be it good or bad) they may have with their developments. I mentioned to books that are interesting to read: the transparent society [1] and total recall [2].

In the afternoon we discussed two specific projects. One was an application for informal social while watching TV (based on a set iconic communication elements) that can be used to generate meta data on the program shown. The other is a platform that allows web developers to create distributed mobile applications making use of all the sensors on mobile phones. It is essential a platform an API that provides access to all functions on the phones available in S60 phones over a RESTful API, e.g. you can use a HTTP call to make a photo on someone's phone. We hope to release some of the software soon.

In the coffee area at T-labs was a printout with the 10+1 innovation principles - could not resist to take a photo ;-) Seems innovation is really trival - just follow the 11 rules and you are there ;-)

[1] David Brin. The Transparent Society: Will Technology Force Us to Choose Between Privacy and Freedom. Basic Books. 1999. ISBN-13: 978-0738201443. Amazon-link. Webpage: http://www.davidbrin.com/transparent.htm

[2] Gordon Bell, Jim Gemmell. Total Recall: How the E-Memory Revolution Will Change Everything. Dutton Adult. 2009. ISBN-13: 978-0525951346. Amazon-link. Webpage: http://totalrecallbook.com/

Ephemeral User Interfaces - nothing lasts forever and somethings even shorter

Robustness and durability are typical qualities that we aim for when building interactive prototypes and systems. Tanja and Axel explored what user experience we can create when we deliberately design something in a way that is ephemeral (=not lasting, there is a good German word "vergänglich"). Ephemeral User Interfaces are user interface elements and technologies that are designed to be engaging but fragile [1]. In the prototype that we showed at TEI 2010 in Cambridge the user can interact with soap bubbles to control a computer. Axel has some additional photos on his web page.


There is a short video of the installation on the technology review blog: http://www.technologyreview.com/blog/editors/24729

[1] Sylvester, A., Döring, T., and Schmidt, A. 2010. Liquids, smoke, and soap bubbles: reflections on materials for ephemeral user interfaces. In Proceedings of the Fourth international Conference on Tangible, Embedded, and Embodied interaction(Cambridge, Massachusetts, USA, January 24 - 27, 2010). TEI '10. ACM, New York, NY, 269-270. DOI= http://doi.acm.org/10.1145/1709886.1709941

AmbiViz Project Presentation at DoCoMo Eurolabs in Munich

In the AmbiViz project we investigate how context information (explicitly or implicitly collected) can be provided to the user. The aim is that the display of context is meaningful to the user but at the same time abstract and privacy preserving. For context collection we integrated the IYOUIT platform as well as an application that runs as an app in facebook. Examples of abstract presentations are implemented for the Nabaztag and for digital picture frames. More details will be available soon in a WIP paper at Percom 2010. Currently we run a longer term study and are already curious about the results.

Recently launched TVs with Web browsing capabilities include special widgets for certain services (e.g. Philips Net TV offers widgets for youtube, ebay, etc.). This is a first step in the market towards the switch-over of traditional TV we discussed some time ago in Seoul. I wonder what other information services may become hype as widgets or Web channels on TVs.

Book chapter on Teaching Beyond the Classroom

Kati Mäkitalo-Siegl, Jan Zottmann, Frederic Kaplan and Frank Fischer organized a workshop that resulted in the book: Classroom of the Future. The book gives a very good overview and is in my view very well suited to run a seminar

We have one chapter in the book that talks about teaching beyond the classroom and outside a typical classroom using pervasive computing technologies [1]. Our chapter includes some of the work on tangible interaction we did in Munich, e.g. [2] and earlier experience where we did a school garden blog (in German only http://www.hcilab.org/albrecht/flachs-web/).

Details about the book and a free preview are available at the publisher's side. The free preview includes an article on the classroom of the past - which I found quite interesting. If you are interested in our article, drop me a mail and I may find the draft.

[1] Paul Holleis, Albrecht Schmidt, Heiko Drewes, Richard Atterer, Petra Dollinger. Teaching Beyond the Classroom: Pervasive Computing Technologies for the Classroom of the Future. Classroom of the future. SensePublisher 2010, pp 63-85. ISBN:978-9460911026 (book at amazon)

[2] Terrenghi, L., Kranz, M., Holleis, P., and Schmidt, A. 2006. A cube to learn: a tangible user interface for the design of a learning appliance. Personal Ubiquitous Comput. 10, 2-3 (Jan. 2006), 153-158. DOI= http://dx.doi.org/10.1007/s00779-005-0025-8

Multi-touch on Ice

I thought I have seen all possible multi-touch installations ;-) but here is a new one: Ubice installation.

Jonna and her group at NRC in Finland have built a multi-touch wall made from ice (frozen water - they seem to have plenty of it up there).

Even though it is on youtube (http://www.youtube.com/watch?v=bbtrI6GjBsk) we still hope to see a paper on this project - do the user study before it melts ;-)

The Interactive Tabletops and Surfaces 2010 conference is this year in Germany in November 2010: http://www.its2010.org/

Energy Harvesting - wireless power?

Since a wireless charger (RCA Airnergy) was announces at CES that harvests energy from wifi signals that are around the topic is everywhere in the (tech) news. It is an exciting prospective to imagine mobile devices that will recharge themselves from the environment they are in. However I am not sure how well harvesting energy from RF will worked in regular environments - not saturated with radio signals.

The topic is really exciting - and in the end people probably do not care much if the energy is harvested from their movement, from light, temperature or RF - as long as it works. There is a introductory article to energy harvesting by Joe Paradiso and Thad Starner [1] which is worthwhile to read.

[1] Joseph A. Paradiso, Thad Starner, "Energy Scavenging for Mobile and Wireless Electronics," IEEE Pervasive Computing, vol. 4, no. 1, pp. 18-27, Jan.-Mar. 2005, doi:10.1109/MPRV.2005.9

Essen - European Capital of Culture 2010

This year RUHR.2010 - Essen and the Ruhr area - is European Capital of Culture. If you ever dreamed of visiting Essen - this is the year!

Article on Driving Automotive User Interface Research in IEEE Pervasive

We wrote an article on the automotive user interface conference that took place in Essen in September 2009. In the current issue of the IEEE Pervasive Computing Magazine this paper is published and gives an overview of the conference [1]. We hope our article highlights the key ideas that were presented at the conference.

Abstract. Cars offer an interesting but challenging microcosm for pervasive computing research and, in particular, for interaction with pervasive computing systems. Increasingly, researchers are looking at interactive applications in the car and investigating human-car interaction from a computer science-rather than an ergonomics or mechanical engineering-perspective. This article reports on the International Conference on Automotive User Interfaces and Interactive Vehicular Applications, wherein participants shared presentations on topics such as aesthetics, user interaction and distraction, safety, and driver monitoring.

[1] Albrecht Schmidt, Wolfgang Spiessl, Dagmar Kern, "Driving Automotive User Interface Research," IEEE Pervasive Computing, vol. 9, no. 1, pp. 85-88, Jan.-Mar. 2010, doi:10.1109/MPRV.2010.3.

Seminar on Pervasive Display Networks in Dagstuhl

From the 4th to the 8th of January we (Nigel Davies, Antonio Krüger, Marc Langheinrich, Martin Strohbach, and I) organized a Seminar on Pervasive Display Networks in Dagstuhl. The timining was less not perfect but we go a really good team together.

Working on the topic we identify an interesting set of dimensions to classify public display, collected a large set of scenarios, and compiled a bibliography as well as a reading list. As soon as the material is publicly available I will post it.

The first time I have been to Dagstuhl was in September 2001 (the seminar was on during 9/11) on Ubiquitous Computing (http://www.vs.inf.ethz.ch/events/dag2001/). This previous seminar really helped to create a vibrant community. Perhaps Pervasive Display Networks will grow in a similar way.

Since then I have been many times to Dagstuhl - but this year I was skiing while there for the first time. There is a small "skiing resort" close by: Erbeskopf…

PS: if you rent skis it is always good to remember what skis (e.g. color, make, size, etc.) one got ;-)
I learned just taking the next pair of skis is sub-optimal...

Books - Christmas break reading

While travelling I came across two very different books. On one of the airports I came through I came across Superfreakonimics by Levitt and Dubner [1] I had also read their earlier book (Freakonomics) as well as The Undercover Economist from Tim Harford - so I got this one. It is funny to read and I enjoyed most of it. The Geo-Engineering statements in the book received quite some critism on the net. So don't by it for its discussion on climate ;-). Reading the books it seems one gets a good explanation of certain things in the world (and economics) - not sure if this is really true, but it is great fun to read nevertheless. I particular like the argument why emancipation leads to a lower quality of teaching in schools :-)

A very different book (also with regard to the price; its more on a library budget than a casual airport buy), but not less interesting, is "Awareness systems" by Panos Markopolous, Boris de Royter and Wendy Mackay [2]. So far I have had only a quick look at the book but this could be the basis for a seminar in a term to come.

[1] Steven D. Levitt, Stephen J. Dubner. Superfreakonomics. 2009. http://freakonomicsbook.com/ ISBN 978-0-7139-9991-4

[2] Markopoulos, Panos; De Ruyter, Boris; Mackay, Wendy (Eds.). Awareness Systems. Advances in Theory, Methodology and Design. 2009, ISBN: 978-1-84882-476-8

PS: I came across another book that takes an extreme - but still to some extent interesting - perspective on the German society. The book is called "Die verblödete Republik" (the republic that went gaga). In some parts I was reminded of the movie "wag the dog" - but the book is much more serious about it - providing a lot of references...

TPC meetings in Atlanta, Palo Alto and Boston

December was filled with travel - twice to the US and several trips in Europe - and for the first time in two years I did not really get around to write my blog...

I am still wondering what technology we require that could make physical meetings less important. Video conference is getting better and I use it a lot - but it still does not facilitate a discussion between 30 or more people well. Besides the work that is in reviewing I really emjoy that part of my job - I find it really exciting to see so much (somehow) novel work in a very short time.

Academics often complain about a lot of travel - but sometimes we need a reality check. Walking around Atlanta airport and seeing the large number of soldiers I felt that I should not complain about my travels… At the same time I asked myself what we will have first: "remote only" wars or "remote only" critical business meeting.

In the first two weeks of December I had the privilege to be the CHI, Pervasive and PerCom program committee meeting. Having seen more than 100 papers being discussed made the effort in reviewing worthwhile. The overview of the field one gets is amazing. And with the insight view I am looking forward to three very interesting conference programs to come in 2010. By the way Geraldine Fitzpatrick (this years paper co-chair at CHI) has move to the Vienna University of Technology.

Besides the PC meetings there was some time to visit labs. In Palo Alto at Nokia Research we were shown a communication appliance that is designed to facilitate remote interaction and communication around a book. Looks interesting and they promised there will be a paper about this soon.

In Boston I went to the new MediaLab building (and met Leah Buechley and Joe Paradiso) - really exciting - research between boxes. Seeing some a Leah's work motivated again to look more into wearables. If you are curious too, have a look at Lilypad Arduino [1] and at the 2008 CHI paper [2].


[1] Buechley, L. Lilypad Arduino | build something. http://web.media.mit.edu/~leah/LilyPad/build.html

[2] Buechley, L., Eisenberg, M., Catchen, J., and Crockett, A. The Lilypad Arduino: using computational textiles to investigate engagement, aesthetics, and diversity in computer science education. In CHI '08: Proceedings of the twenty-sixth annual SIGCHI conference on Human factors in computing systems (New York, NY, USA, 2008), ACM, pp. 423-432.

Enrico Rukzio visits us in Essen, projections everywhere

On Wednesday and Thursday Enrico visited our group in Essen. He gave a part of my lecture on user interface engineering talking about mobile interaction with the real world. He include interesting examples, such as QR-code/NFC/RFID use in Asia, SixthSense project (camera projection system to wear around the neck) and handheld mobile projections. Enrico also explained some of the multi-tag work he does at Lancaster University [1].

In the lecture we talked briefly about future devices and interfaces. I mentioned one example: projection in the large - on building scale. The 3D visualization overplayed on buildings seem impressive - at least when looking at the video. NuFormer (http://www.projectiononbuildings.com/en) has created several interesting projections - but I never have seen one in the real - so far...

Looking at the 6th sense project and on the building projections we wondered how important in may become in the future to make research results in ubicomp/HCI understandable and accessible to a wide audience. Will this replace papers in the future?

[1] Seewoonauth, K., Rukzio, E., Hardy, R., and Holleis, P. 2009. Touch & connect and touch & select: interacting with a computer by touching it with a mobile phone. In Proceedings of the 11th international Conference on Human-Computer interaction with Mobile Devices and Services (Bonn, Germany, September 15 - 18, 2009). MobileHCI '09. ACM, New York, NY, 1-9. DOI= http://doi.acm.org/10.1145/1613858.1613905

MUM 2009 in Cambridge, no technical solution for privacy

The 8th International Conference on Mobile and Ubiquitous Multimedia (MUM 2009) was held in Cambridge, UK. The conference is fairly specific and had an acceptance rate of about 33% - have a look at the table of content for an overview. Florian Michahelles presented our paper on a design space for ubiquitous product recommendation systems [1]. Our work contributes a comprehensive design space that outlines design options for product recommendation systems using mobile and ubiquitous technologies. We think that over the next years mobile recommendation systems have the potential to change the way we shop in the real world. It probably will be normal to have access in-depth information an price comparison while browsing in physical stores. The idea has been around for a while, e.g. the pocket bargain finder presented at the first ubicomp conference [2]. In Germany we see also a reaction of some electronics stores that asked users NOT to use a phone or camera in the shop.

The keynote on Tuesday morning was by Martin Rieser on the Art of Mobility. He blogs on this topic on http://mobileaudience.blogspot.com/.
The examples he presented in his keynote concentrated on locative and pervasive media. He characterized locative media as media that by social interaction that is linked to a specific place. He raised the awareness that mapping is very important for our perception of the world, using several different subjective maps - I particular liked the map encoding travel times to London . A further interesting examples was a project by Christian Nold: Bio mapping - emotional mapping of journeys. QR or other bar code markers on cloth (large and on the outside) have a potential ... I see this now.

In the afternoon was panel on "Security and Privacy: Is it only a matter of time before a massive loss of personal data or identity theft happens on a smart mobile platform?" with David Cleevely, Tim Kindberg, and Derek McAuley. I found the discussion very inspiring but in the end I doubt more and more that technical solutions will solve the problem. I think it is essential to consider the technological, social and legal framework in which we live. If I would need to live in a house that provides absolute safety (without a social and legal framework) it would be probably not a very nice place… hence I think here we need really interdisciplinary research in this domain.

[1] von Reischach, F., Michahelles, F., and Schmidt, A. 2009. The design space of ubiquitous product recommendation systems. In Proceedings of the 8th international Conference on Mobile and Ubiquitous Multimedia (Cambridge, United Kingdom, November 22 - 25, 2009). MUM '09. ACM, New York, NY, 1-10. DOI= http://doi.acm.org/10.1145/1658550.1658552

[2] Brody, A. B. and Gottsman, E. J. 1999. Pocket Bargain Finder: A Handheld Device for Augmented Commerce. InProceedings of the 1st international Symposium on Handheld and Ubiquitous Computing (Karlsruhe, Germany, September 27 - 29, 1999). H. Gellersen, Ed. Lecture Notes In Computer Science, vol. 1707. Springer-Verlag, London, 44-51.
http://www.springerlink.com/content/jxtd2ybejypr2kfr/

PhD Defense of Elina Vartiainen

Finland is one country in Europe were it seems pretty hard to get to in the morning from Germany or Austria. If you have a meeting before 11 am you have to fly the day before.

I was invited to Helsinki to be opponent (together with Kaisa Väänänen-Vainio-Mattila) for the PhD defense of Elina Vartiainen at Helsinki University of Technology. The first time I came across work she was involved in was at CHI 2006 in Montreal. She worked with Virpi Roto on the Minimap web browser [1]. Last year in a doctoral colloquium in Finland we first discussed some of her work and I was excited to read it in more detail for the PhD exam.

Dissertations in practical areas of computer science that are done in company research labs are at the same time limited and exciting. What can be done is often limited by the company needs but on the other hand it offers the great opportunity to get things out large scale and collect experiences from many users (e.g. you may want to check the ImageExchange project, where the studies were also part of Elina's dissertation).

I like the finnish system of having a long public defense. We discussed about 3 hours with Elina and I enjoyed it :-)

There are two general but important issues I think I take away from our discussion:

1. do question the research process including the steps (e.g. hardware first or applications first), the approach (e.g. human need centred vs. design driven) and the setup of the teams (who is needed to get a successful product? Business, law, design, hardware?).
2. innovation for web services on a global scale comes not from a single company or small set of highly skilled developers. Creating opportunities for a larger number developers (with skills limited skills, e.g. like web development) will be the key to create all the applications people need all over the world. Having a single instance controlling what can be developed does scale.

Guess what was the first web browser on a mobile device I used on a mobile device? It was an Apple Messagepad - and the browser was PocketWeb developed at TecO in Karlsruhe (where I worked from 1998-2001), see [2] and http://www.teco.edu/pocketweb/

[1] Roto, V., Popescu, A., Koivisto, A., and Vartiainen, E. 2006. Minimap: a web page visualization method for mobile phones. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Montréal, Québec, Canada, April 22 - 27, 2006). R. Grinter, T. Rodden, P. Aoki, E. Cutrell, R. Jeffries, and G. Olson, Eds. CHI '06. ACM, New York, NY, 35-44. DOI= http://doi.acm.org/10.1145/1124772.1124779

[2] Stefan Gessler and Andreas Kotulla. PDAs as mobile WWW browsers. Proceedings of the Second World Wide Web Conference '94: Mosaic and the Web. Chicago, Illinois, USA, 1994.

Best Paper at AmI 2009, Visions

Florian Alt presented our work on pervasive advertising, in particular on creating dynamic user profiles in the real world [1]. This research was carried out together with colleagues in marketing and software systems and implemented in one of our course. We are proud that it got named best paper at AmI 2009! Another paper to look at in this context was presented by Jörg Müller; it looked at pervasive advertisement utilizing screens on public phone boxes. The study is amazing in size (20 displays all over Münster, 17 participating shops) and duration (1 year, 24/7) [2]. Even though the results are not completely conclusive it is very interesting to read about the experience of such a large deployment real world in a research context.

This year at AmI included a vision panel with Juan Carlos Augusto, Florian Michahelles, Jörg Müller, Donald Patterson, which I chaired with Norbert Streiz. The main questions for us were: Is there a need for a vision? What are drivers for a new Vision? And what is the value of having a technology vision? The discussion was very diverse touching on various topics. One interesting observation is that many people - including me - see that a mobile personal device (what now is the mobile phone) will stay at the center of a new vision. A further very insightful comment was that we as a community should try to develop more specific visions (e.g. how will education be in 2020, or how will public transport be in 2030) rather than have a Version 2.0 of the overall vision. I think such more specific visions could be valuable to guide research in Ambient Intelligence in the next years.

[1] Florian Alt, Moritz Balz, Stefanie Kristes, Alireza Sahami Shirazi, Julian Mennenöh, Albrecht Schmidt, Hendrik Schröder and Michael Goedicke: Adaptive User Profiles in Pervasive Advertising Environments. Proceedings of the 3rd European Conference on Ambient Intelligence (AmI '09). Springer Berlin / Heidelberg. Salzburg, Austria 2009.

[2] Jörg Müller, Antonio Krüger: MobiDiC: Context Adaptive Digital Signage with Coupons. Proceedings of the 3rd European Conference on Ambient Intelligence (AmI '09). Springer Berlin / Heidelberg. Salzburg, Austria 2009.

Keynote by Frits Grotenhuis at AMI-2009

In the opening keynote of AMI 2009 Frits Grotenhuis (who stepped in for Emil Aarts) looked back at the last 10 years ambient intelligence. In his talk he showed a number of examples of devices that Philips created in this time, including iCat, the Entertaible, Ambilight, and medical devices. He discussed briefly the forces in such developments between market-pull and technology-push and it became evident that many products in this domain are more technology push than market-pull.

I liked the reference back to the Electronic Poem at the 1958 Brussels World’s Fair, which must have been at that time an amazing large scale installation (creating "surround sound" with more than 200 speakers).

The update of the vision (title of the keynote was Ambient Intelligence 2.0: Towards Synergetic Prosperity) suggests a model where the human is in the center and the surrounded by the Mind (well being), Community (participation), Body (health), and Environment (responsibility). I found this did not offer many new insights as by now the human centered approach is widely accepted. Looking at the examples it seems that the vision is very much centering on the "rich" world's problem… perhaps it is really hard to update a vision.

From the Internet of Things to the Web of Things

The central role of ICT becomes very visible when it does not work. Sometimes for the good as I was late arriving at Düsseldorf airport but the Airberlin check-in system was down for a few minutes - just enough that I was still in time :-)

In the evening I met Prof. Lorenz Hilty, who gave a talk in the afternoon at ETH Zurich. I missed the talk but after the interesting and though provoking dinner conversation I decided I should finally really read his book [1] - perhaps over Christmas. Meeting with Friedemann Mattern and Hans Gellersen was very inspiring and I hope we get a change to have future joint projects.

Looking out over Zürich we talked about the transformation from the internet of things to the world wide web of things. The use of prototcol seems a little technical detail, but in my eye it may have a major impact. The WWW of things is creating a world of networked artefacts (much like the internet of things) but is completely based on Web protocols (e.g. http, RESTful web services). By working with web protocols the objects can easily become part of the web and interact with web-platforms and applications on the www (e.g. facebook, twitter, etc.). I expect by having a WWW of things we enable many more developers to create new and exciting applications on top of the internet of things. There are many challenging research questions. I am particularly interested in how will a good platform look like that empowers web programmers to create and distribute applications on the Web of things. I think we should run a workshop on this in the near future!

[1] Information Technology and Sustainability: Essays on the Relationship between Information Technology and Sustainable Development. Lorenz M. Hilty. 2008.

Large pixels along the underpass

In the refurbished railway station (not yet finished) there is an interesting new pixel display in one main underpass. One wall is covered with a display. It is about 10 pixel (probably about 4 meters) high and several hundred pixels long (have not counted/measured them). It changes colors and shows writing (so far not really exciting).

How cool would it be if there is a freely accessible programmable web-service to control these pixel? I would guess people could create all sorts of interesting content… Perhaps people would start to bargain to get their 5 minutes of virtual graffiti shown…

Tangible, Embedded, and Reality-Based Interaction

Together with Antonio's group we looked at new forms of interaction beyond the desktop. The journal paper Tangible, Embedded, and Reality-Based Interaction [1] gives overview and examples of recent trends in human computer interaction and is a good starting point to learn about these topics.

Abstract: Tangible, embedded, and reality-based interaction are among novel concepts of interaction design that will change our usage of computers and be part of our daily life in coming years. In this article, we present an overview of the research area of tangible, embedded, and reality-based interaction as an area of media informatics. Potentials and challenges are demonstrated with four selected case studies from our research work.

[1] Tanja Döring, Antonio Krüger, Albrecht Schmidt, Johannes Schöning: Tangible, Embedded, and Reality-Based Interaction. it - Information Technology 51 (2009) 6 , S. 319-324. (pdf)
http://www.it-information-technology.de/

Our PERCI Article in IEEE Internet Computing

Base on work we did together with DoCoMo Eurolabs in Munich we have published the article "Perci: Pervasive Service Interaction with the Internet of Things" in the IEEE Internet Computing special issue on the Internet of Things edited by Frédéric Thiesse and Florian Michahelles.

The paper discusses the linking of digital resources to the real world. We investigated how to augment everyday objects with RFID and Near Field Communication (NFC) tags to enable simpler ways for users to interact with service. We aim at creating a digital identities of real world objects and by this integrating them into the Internet of Things and associating them with digital information and services. In our experiments we explore how these objects can facilitate access to digital resources and support interaction with them-for example, through mobile devices that feature technologies for discovering, capturing, and using information from tagged objects. See [1] for the full article.

[1] Gregor Broll, Massimo Paolucci, Matthias Wagner, Enrico Rukzio, Albrecht Schmidt, and Heinrich Hußmann. Perci: Pervasive Service Interaction with the Internet of Things. IEEE Internet Computing. November/December 2009 (vol. 13 no. 6). pp. 74-81
http://doi.ieeecomputersociety.org/10.1109/MIC.2009.120

The computer mouse - next generation?

In my lecture on user interface engineering I start out with a short history of human computer interaction. I like to discuss ideas and inventions in the context of the people who did it, besides others I take about Vannevar Bush and his vision of information processing [1], Ivan Sutherland's sketchpad [2], Doug Engelbart's CSCW demo (including the mouse) [3], and Alan Kay's vision of the Dynabook [4].

One aspect of looking at the history is to better understand the future of interaction with computers. One typical question I ask in class is "what is the ultimate user interface" and typical answers are "direct interface to my brain - the computer will do what I think" and "mouse and keyboard" - both answers showing some insight…

As the mouse is still a very import input device (and probably for some time to come) there is a recent paper that I find really interesting. It looks at how the mouse could be enhanced - Nicolas Villar and his colleagues put really a lot of ideas together [5]. The paper is worthwhile to read - but if you don't have time at least watch it on youtube.

[1] Vannevar Bush, As we may think, Atlantic monthly, July 1945.
[2] Ivan Sutherland, "Sketchpad: A Man-Machine Graphical Communication System" Technical Report No. 296, Lincoln Laboratory, Massachusetts Institute of Technology via Defense Technical Information Center January 1963. (PDF, youtube).
[3] Douglas Engelbart, the demo 1968. (Overview, youtube)
[4] John Lees. The World In Your Own Notebook (Alan Kay's Dynabook project at Xerox PARC). The Best of Creative Computing. Volume 3 (1980)
[5] Villar, N., Izadi, S., Rosenfeld, D., Benko, H., Helmes, J., Westhues, J., Hodges, S., Ofek, E., Butler, A., Cao, X., and Chen, B. 2009. Mouse 2.0: multi-touch meets the mouse. In Proceedings of the 22nd Annual ACM Symposium on User interface Software and Technology (Victoria, BC, Canada, October 04 - 07, 2009). UIST '09. ACM, New York, NY, 33-42. DOI= http://doi.acm.org/10.1145/1622176.1622184

Shared editing is still hard - why?

Having coordinated the editing of a shared document with about 100 pages I still wonder why I have not come across a really good solutions that work in a real life context. We were 10 people working on the document which also contained about 100 references various tables and graphs, which originated in spreadsheets. Our solution using (different version) of Microsoft Word and Excel on different platforms (Win and Mac) was at best sup-optimal. Track changes works great if I write something and someone else corrects it - but with a larger number of people creating and reworking the document just seems unmanageable.

We tried google-docs before, which is nice for joint editing but lack essential functions and is to my experience unreliable. We lost most of the document we created at some point. The same happened to one of our students writing up his project…

The purists argue that Latex and SVN is the solution - however if you have ever worked with real people outside the geek world you will know that it is not :-) and it would question if there was any progress in text processing in the last 20 years at all.

Is it only me who does not see the solution? Here are the requirements:

• Shared editing of a document of considerable size (100+ pages)
• Functionality required for larger scientific documents such as styles, (cross)-reference, creation of tables, etc.
• Comfort functions in editing, such as spelling and grammar checking, auto completion, tracking of changes
• Works in a heterogeneous environment including Macs and Windows and across administrative domains (e.g. people can be behind different firewalls)
• Automatically creating a backup of the document every few minutes
• Integration of other media (e.g. images) and data sources (e.g. spreadsheet tables)

What is your solution? I think mine (email and copy and paste) is not really the optimal one….

In comparison to some years ago awareness, video and audio conferencing with skype works very well - but again for application sharing I have not seen a perfect solution that works in real live - any suggestions?

PS: our final and printed document missed 115 spaces (a known error from exchanging docx between Windows and Mac)

What portion of research time is spent writing proposals?

The next European deadline is close and hence everyone is writing proposals…

I wonder if someone has assessed how much work goes into proposal writing on a European scale. On one hand I see the value of forcing researchers to write proposals and to articulate their ideas but on the other hand it seems a great lot of research could be conducted if senior people would use this time for doing actual research. In proposals formulating the actual core of the research idea is exciting (often even more exciting than carrying out research) but this is only one part of proposal writing. But what would be an alternative for deciding what research to fund?

Having spent a 30 hours in Lancaster improved our idea and we got a good step forward…

PS: Birthday is a perfect day for finding out which companies have you on your mailing list

Finishing my term as External Examiner at Trinity College Dublin

Over the last three years I have been regularly to Dublin to act as external examiner for the Ubicomp MSc course. For me this was a good experience to see how serious The School of Computing at Trinity College takes external quality control and how well processes are managed. And besides the administrative part I saw a great many interesting MSc dissertations over the years. Even though the term has come to an end I hope to travel to Dublin in the future too - perhaps on Holiday to see more of the city (which I did not really manage …)

PS: recession seems to have hit Ireland - I have never seen such short queue in Dublin airport - and it is definatly not the selfservice machines that reduced the queue ...

What did you do last Weekend: Soldering a radio kit and trying out a Sony Walkman

What did I do with Vivien the last weekends? We soldered a radio receiver kit (retro style) and it worked - there are still plenty of stations on the air all over Europe. Nowadays you have to make quite some effort to find interesting electronic kits - besides the radio we got a candle light simulator (it is an LED controlled by a PIC microcontroller that imitates a realistic flickering candle in the form factor a small candle).

Do you remember the Sony walkman? It was at the time quite a revolution - looking at it now it looks a bit bulky. The BBC4 program "electric dreams" featuring a fast-forward through technologies from the time I was born till now was very entertaining and it brought back a lot of memories … ups getting old :-(

Land of ideas - Award for the innovation factory at the University of Duisburg-Essen

Every day per year one place or institution is Germany is awarded "a place in the land of ideas". This initiative is quite completive (each year well over 2000 institutions apply and the year has only 365 days…).

The innovation factory at the University of Duisburg-Essen has a really interesting vision and received the award for its concept of converting ideas into innovations and products. It promotes thinking in product rather and by this challenges researchers to look how their research results could enable new products or enhance existing products. I find this a most useful exercise - especially when having basic research results that are not directly linked to an application. To facilitate thinking in products they have a number of industrial designers that act as innovation scouts.

Such events always offer the official speeches (not my favorite part but I see one needs them) but also the chance to meet interesting people. I enjoyed the discussions - in particular one question I found interesting - whether or not the ebook reader based on eInk (now just entering the marked on massive scale) is already dead… and replace by the next generation of multimedia extra-thin wireless tablets.

Workshop on Pervasive Advertising at Informatik 2009 in Lübeck

Following our first workshop on this topic in Nara during Pervasive 2009 earlier this year we had on Friday the 2nd Pervasive Advertising Workshop in Lübeck as part of the German computer science conference Informatik 2009.

The program was interesting and very diverse. Daniel Michelis discussed in his talk how we move from an attention economy towards an engagement economy. He argued that marketing has to move beyond the AIDA(S) model and to consider engagement as central issue. In this context he introduced the notion of Calm Advertising and interesting analogy to Calm Computing [1]. Peter van Waart talked about meaningful adverting and introduced the concept of meaningful experience. To stay with the economy term consider advertising in an experience economy. For more detail see the workshop webpage - proceedings will be soon online.

Jörg Müller talked about contextual advertising and he had a nice picture of the steaming manhole coffee ad - apparently from NY - but it is not clear if it is deployed.

If you are interested in getting sensor data on the web - and having them also geo-referenced - you should have a look at http://www.52north.org. This is an interesting open source software system that appears quite powerful.

Florian Alt presented our work interactive and context-aware advertising insight a taxi [2].

[1] Weiser, M., Brown, J.S.: The coming age of calm technology. (1996)

[2] Florian Alt, Alireza Sahami Shirazi, Max Pfeiffer, Paul Holleis, Albrecht Schmidt. TaxiMedia: An Interactive Context-Aware Entertainment and Advertising System (Workshop Paper). 2nd Pervasive Advertising Workshop @ Informatik 2009. Lübeck, Germany 2009.

Tag clouds as a title page

For some of the recent proceedings (Automotive User Interfaces Conference and Pervasive Advertising Workshop) we used a tag cloud generator (http://www.wordle.net/) to generate a picture for the title page. The tag cloud below of the pervasive advertising workshop is based on the whole proceedings and the one above based on my professional CV.
Bastian just told me there are other interesting ways to characterize a person ... see http://personas.media.mit.edu/

Auto-UI 2010 announced - 2009 Proceedings available in the ACM DL

The Auto-UI 2009 conference in Essen is over - and for us it was very enjoyable to have this many visitors at the University of Duisburg-Essen - see the photos. The conference facilitated good discussions and had a very constructive atmosphere. We should continue this exchange of ideas and there is always room for improvement... and that is why there is a Auto-UI conference 2010 in Pittsburgh, US - and there is interest beyond this in hosting the conference.

You can register to get information about the next conference on the Auto-UI webpage.

The proceedings are now online in the ACM DL and the linked on the program website.

Social event at Zeche Zollverein, Accidents are avoidable

In the evening we went to Zeche Zollverein - a world cultural heritage site called "the most beautiful coal mine in the world". We got a guided tour and had dinner in the Kokerei.
It was interesting to see and learn about working conditions - which were really hard. 100 years ago it was common that live expectancy of the workers was less that 60, that there was typically one serious accident per day and that about 30 people died every year in the coal mine.

We find that nowadays inhumane and it would be in Germany (and many other countries) completely unacceptable. Coming back to cars … we accept that in order to have personal transportation it sees unavoidable to have accidents and that 4477 people were killed 2008 in traffic accidents in Germany (which was lower than all the years before). Perhaps in 100 years people will look back at us similar to how we look back at the working conditions in coal mines 100 years ago. And I think research in Automotive User Interfaces can help working towards safer individual traffic.