New project on ambient visualization - kick-off meeting in Munich

We met in Munich at Docomo Euro Labs to start a new project that is related to context and ambient visualizations. And everyone already got bunnies ;-)

Related to this there is a large and very interesting project: IYOUIT. Besides other things it can record and share your context – if you have a Nokia series 60 phone you should try it out. As far as I remember it was voted best mobile experience at mobile HCI 2008. 

Random Links, toys and free location data

Over the last day I have learned about some (more) interesting things out there – here are some to share with you:

• OpenGeoDB: http://opengeodb.hoppe-media.com is a database with locations in Germany and an open API
  
• A community effort to create an open map: http://www.openstreetmap.org/
• publishing you travels to others: http://www.dopplr.com 
  
• Ubfunckeys: USB connected strange toy, www.ubfunkeys.com
  
• A communicative rabbit: http://www.nabaztag.com/ 
  

Tactile interfaces, Visit from Gordon Bolduan

This afternoon Gordon Bolduan from Technology Review was visiting the lab. We talked about haptic and tactile interfaces and showed some demos (e.g. navigation with tactile cues). 

When preparing for the visit I looked for some good examples of tactile interaction - and interestingly there is more and more work out there that has the potential to change future interfaces and means of communication. 

Recent work on connecting people [1] and [2] at the boundary between computing and design shows new options for emotional communication. 

We used in our work multiple vibration motors and explored the potential for mobile devices [3]. What to use for tactile interaction beyond vibration is one obvious question, and I find the paper by Kevin Li [4] a good starting point to get some more ideas.

When talking about human computer interaction that includes stroking, tapping and rubbing an association to erotic and sexual interactions seem obvious; and there is more to that if you are curious just search for teledildonics and you will find interesting commercial products as well as a lot of DIY ideas.

[1] Eichhorn, E., Wettach, R., and Hornecker, E. 2008. A stroking device for spatially separated couples. In Proceedings of the 10th international Conference on Human Computer interaction with Mobile Devices and Services (Amsterdam, The Netherlands, September 02 - 05, 2008). MobileHCI '08. ACM, New York, NY, 303-306. DOI= http://doi.acm.org/10.1145/1409240.1409274 

[2] Werner, J., Wettach, R., and Hornecker, E. 2008. United-pulse: feeling your partner's pulse. In Proceedings of the 10th international Conference on Human Computer interaction with Mobile Devices and Services (Amsterdam, The Netherlands, September 02 - 05, 2008). MobileHCI '08. ACM, New York, NY, 535-538. DOI= http://doi.acm.org/10.1145/1409240.1409338 

[3] Alireza Sahami, Paul Holleis, Albrecht Schmidt, Jonna Häkkilä: Rich Tactile Output on Mobile Devices. European Conference on Ambient Intelligence (Ami'08). Springer LNCS Nürnberg 2008, S. 210-221. DOI= http://dx.doi.org/10.1007/978-3-540-89617-3_14

[4] Li, K. A., Baudisch, P., Griswold, W. G., and Hollan, J. D. 2008. Tapping and rubbing: exploring new dimensions of tactile feedback with voice coil motors. In Proceedings of the 21st Annual ACM Symposium on User interface Software and Technology (Monterey, CA, USA, October 19 - 22, 2008). UIST '08. ACM, New York, NY, 181-190. DOI= http://doi.acm.org/10.1145/1449715.1449744

One Ez430-F2013 for each student in DSD

This term we teach digital system design and besides the essential (gates, flip-flops, 2-complements, alu, data path, etc) we decided to include some practical parts. In the first part we introduced Verilog. We did the exercises with Icarus Verilog (free, text-based verilog) and there are more powerful tools available, e.g. ISE WebPACK.

In the second part we have practical exercises in assembly language using the Ez430-F2013 development kit. We borrowed one kit to each student (just in case someone is bored over Christmas) and the current task is to complete and assembly program (incomplete sample) that the LED shows repeatedly "Hello" in Morse code. This is just the start - perhaps we do some more interesting stuff in January. The development kit is really interesting - especially given the fact that it is only 20€ (including hardware, compiler, and IDE). The MCU is small - but still good enought to generate a video signal (eagle CAD files, some assembly code). 

Enrico Rukzio visits our Lab

Enrico Rukzio (my first PhD from Munich, now lecturer in Lancaster) visited our Lab. He was make a small tour of Germany (Münster, Essen, Oldenburg). In the user interface engineering class Enrico showed some on his current work on mobile interaction, in particular mobile projectors and NFC tags. After the presentation we wondered how long it will take till kids on the train will play with mobile projections ;-)

We showed Enrico a demo of eye-tracking for active customization of browser adverts. In our setup we use the Tobii X120. For tracking of people in the room we still have not decided on a system - and Enrico told me about the Optitrack system they have. That looked quite interesting... 

As we all do studies in our work - the design of studies is critical and there is an interesting book to help with this: How to Design and Report Experiments by Andy Field  and Graham J. Hole.

Biometrics will come, who will care about privacy

Arriving at the new Terminal 5 at Heathrow airport I saw some extra installations (and extra lines) for iris scan immigration. Arriving at 4 am in the morning they were closed and there were not queues - but I could see that it is very attractive at other times of day when queues are long. On the official website they claim that border control will be down to 20 seconds. There is a more detailed document on the schema - I saved the document to have it in 10 years when we will have a very different view on privacy.

Will we have face-2-face PC meetings in the future?

On Thursday morning I flew to Boston for the CHI 2009 PC meeting. The review and selection process was organized very professional and efficient. We discussed all papers in one and a half days - and I think an interesting program came out and I learned a lot about what values my colleagues see or see not in papers. On Friday afternoon I flew back to the UK for the Pervasive 2009 PC meeting in Cambridge (with the same crew on the plane).

Nevertheless the question remains how sustainable is it that 100 people fly to a face-2-face meeting. In what way could we do such a meeting remotely? Video conferencing still does not really work well for larger group discussions (just collecting experience here in Cambridge during the Pervasive PC meeting)… Can it be so difficult to make a reasonable video link between two meeting room? How could we recreate the social aspects (like a joined dinner or walking back through the city with Gregory) as well as side conversations in the meetings? We probably should try harder - It cannot be that difficult - there have been massive amounts of work in CSCW research? Perhaps we should try linking two rooms at different universities as a group project next term? 

Christmas market in Essen

Going to Christmas market is in Germany a tradition - and obviously our group went, too ;-)
It is interesting that most of us had time for this "appointment" with only two days notice - usually it takes us weeks to find a date for a meeting and so far we did not find a date for a strategy meeting in the near future. Perhaps offering Glühwein (that is what you drink at Christmas markets) would help…
The quality of photos taken with a mobile phone is in difficult context (e.g. night, lights around) still not satisfactory (even with 5MP, downscaling, and image enhancing).

Known route - driving your car in mental auto-pilot?

Mandy Marder, a doctoral student at university hospital in Essen has done an interesting study, looking at the activity of the brain at different driving situations. It seems that if you are driving a well know route you are less alert than when you drive an unknown route (see press release, we have yet to find the appropriate reference). This is an interesting finding that may help to inform some of our work on automotive user interfaces. Together with trends that move more responsibility from the driver to assitive functions this nay be an indication that driving could be a valid domain for serious games.

Exporting your cars information to the mobile phone

In our user interface engineering class one of the tasks in the exercise is to create a concept design for providing information from the car on the mobile phone (e.g. millage, amount a fuel in the car, next service date, alram status, etc). The first part is to assess what information could be made accessible and what value it would create for the user. 

Today I came across a device (Tyredog TD-1000A) that is concerned with a one sub-part of this scenario: checking your pressure in the tires of the car. It is a simple sensor system, screwed on to each of the tires, connected to a wireless receiver. There is also a version that includes features for the car alarm (Tyredog TD-3000A).

Another group is looking yet again into the domain of  restaurant finders or more general night life. Apropos restaurant finders, Saturday night we got out of the subway onto union square and discussed where to go for dinner (an we probably looked disoriented). A local lady stoped and recommended the Union Square Café - and it was just great… sometimes just talking to someone in the street may provide you with an excellent alternative to technologies ;-) Perhaps the students find a solution that can reflect personal recommendations well...

Information vs. Mobility, Percom PC meeting in New York

The PC meeting for Percom 2009 took place at IBM in Hawthorne, NY. Percom had about 200 submissions and many good ones - so we could compile an exciting program across the whole field of pervasive computing and communication. As one of three program vice chairs I have looked in detail in about 1/3 of the submissions that were application related. It is interesting to observe that research as a whole in the field becomes more major and at the same time more incremental. 

To me this puts up the big question in which domains will the new big innovations happen, what is the next trend after we have pervasive computing? There are luckily plenty of options, but at the moment it seems that there develops an interesting relationship between information, communication, mobility and energy. It seems that we can compensate mobility by information and communication and similarly we can reduce energy required by information available. One example is: if I know where things are (=information) I can reduce the effort required to find them (=mobility). Is there more to it?

Each time in the US - even in New York were public transport works quite well - one is surprise how alien it appears to many that it could be an option to take public transport on a business trip (e.g. there are no first class coaches on regional trains). Flying from Düsseldorf into Newark it was convenient to take the train to Penn Station in NY City and then an express train to White Plaines. If we would not have gone for a walk in the city we probably would have been equally fast as by car. With the again low gas prices in the US (less than 2U$ per gallon, down from 4 just a few month ago) I would expect public transport and small cars will not gain too much popularity - before the next rise in gas prices.

PS: it is amazing how many possiblities there are to serve coffee (and this is probably not one of the most environment friendly)

Male (88%), writing like Oscar Wilde (35%)

Looking into Paul Rayson's blog and discovered an interesting link: http://www.genderanalyzer.com. It is a web form where you can put in an URL and you get an estimate whether the author of this text is male or female. For me it worked great ;-) It says that the text I wrote in my blog is with 88% written by a male. I tried it with a few more of my pages and it worked. Then I looked at some pages of some of my female colleagues and to my surprise it seems they do not write their web pages by themselves (as the program indicated 95% male writer) - they probably all have a hidden male assistant ;-)

While I was in Lancaster I shared for most of the time an office with Paul. During this time I learned a lot of interesting things about corpus linguistics and phenomena in language in general - just by sharing the office. One fact at that at the time was surprising to me is that if you take 6 words from an arbitrary text in the exact order as they appear in the text and you search on the web for the exact phrase it is likely that you will only find this text. How many hits do you get for phrase "I was at Trinity College reading" in google? Try it out ;-) [to students: that is why not getting caught when you plagiarize is really hard]

From http://www.genderanalyzer.com I came to http://www.ofaust.com and to my great surprise I write like Oscar Wilde (35%) and Friedrich Nietzsche (30%). Thinking of social networks (and in particular the use of languages within closed groups) such technologies could become an interesting enabling technology for novel applications. Perhaps I should visit Paul again in Lancaster…

PS: and I nearly forgot I am a thinker / INTJ - The Scientists (according to http://www.typealyzer.com/)

PPS (2008-11-17): a further URL contrinuted from my collegues on the gender topic: http://www.mikeonads.com/2008/07/13/using-your-browser-url-history-estimate-gender/

Exoskeletons soon in the real world

Exoskeletons have received some attention over the last years (some with a clear application area in the army), e.g. Robot suit HAL and the BLEEX Project.  

Honda showed now an interesting exoskeletons as walking aid. It is called  "computerized leg device" or "walking assist device" and there is an short movie on youtube and it is designed to help people walk. The application areas are probably broad - I expected mainly in care and rehabilitation, but looking around there seem to be more application domains…

Illusions 2.0, Talk at the Museum Ludwig in Köln

In Cologne in the Museum Ludwig I gave in the afternoon a talk on "Illusions 2.0 - embedded interactive media" at the Forum Mediendesign. The talk focused on the new qualities of magical experiences we will be able to create with pervasive computing technologies in the future and linked this to Alan Kay's notion of user illusion and metaphors. I looked at trends that are ingredience for creating Illusions 2.0 - in particular ubiquitous communication and display, constant tracking and logging, and the decrease of value of traditional content (text, audio, video, software, tv, statistical data). I highlighted one development that have already happened and has impacted our lives with the following statement. The question "If I only would know when the others come and where they are now…" was common to people born before 1970 but is completely alien to people born after 2000. Mobile communication has changed this and tracking will add more change over the next years.

Based on some examples from recent popular fiction (Harry Potter) I showed that things that we have considered magical are becoming rapidly community products (e.g. marauders map). Spinning this idea forward I asked how far are we with regard to other human dreams such as looking into the future or never forgetting anything we have seen or heard. And the answer in short is: we are close ;-) for more see the slides of my talk on Illusion 2.0. There is an upcoming paper we wrote for IEEE Multimedia Magazine on this topic - will tell as soon as it is published ;-)

In the morning I had some time for sightseeing and Vivien and I went to the chocolate museum. The museum is brilliant and I learned how the hollow chocolate santas are made :-) In the top floor they have a table top projection for a quiz - it is very well done (as the whole museum) but the technology did not work on the table close to the window. As we know from our experience if you have sun light your camera-projector systems may have trouble ;-)

PS: it was interesting that the whole organization was done by students as a course in project management - and they did it really well.

My first hotel fire alarm, debugging smart environments

We arrived in the evening in Köln and went to our hotel and around 10:30 pm the fire alarm sounded (really loud - you want to leave) and a voice over the speaker system asked us to leave the hotel immediately. When we checked in an hour earlier we overheard that they called for the elevator repair man…

Better safe than sorry I packed up my laptop and rucksack and we went downstairs. At the reception they were pretty busy - but it seemed everyone clear that this a false alarm but it seemed they had no way of really understanding why the system behaved in this way [gap of evaluation ;-) teaching user interface engineering this term]. The error search reminded me on one error search strategy in C (if you do not have a debugger). Comment out part of the code (here: disable fire sensing for certain areas in the hotel) till you can tell which parts cases the error. If you have found this part and it is not essential just leave it as a comment (you can do the same with fire sensor - hope they did not…)

A fire alarm system has compared to smart environments we envision a very low complexity. I think providing appropriate means for debugging smart environments by end-users could be a topic worthwhile to look at.

PS: the elivator had the best display for showing the level you are in I have seen so far. From a UI perspective it is really a boring recreation of the non-digital version... 

USA votes, election party in our lab

There many good reason to have an international team - especilly when doing research in pervasive computing and user interface engineering. This morning I learned another one: you can have election parties (=drinks and food in the lab ;-). 

Predicting elections results goes new ways and it is interesting that the Xbox Live Polls (http://blog.wired.com/games/2008/09/xbox-live-poll.html) were quite close to the result - already quite some time ago. Perhaps this large poll produced just by accident good results - but it may be a first step towards internet elections. Having internet voting on a game console brings new models for voting to one's mind (e.g. only if you have reached a certain level in the game you can vote ;-)

Geocaching, Travel bug

On Saturday we went for a walk and to make it more exciting we decided to look for a geocache (GCXP87, N 49° 08.775 E 010° 06.672). To our surprise it took us quite some time (10 minutes) to find it as the GPS reception was not perfect. Vivien, my daughter was excited by the idea of geo caching (I think I can anticipate some of our activities on the next weekends ;-), especially as we even found a travel bug. The travel bug is alreay on its second continent and we still discuss where we place it ;-) 

Technologies of Globalization 2008 in Darmstadt

I have been chairing the Stream "Aging as a Global Issue" at the conference Technologies of Globalization 2008 in Darmstadt. It is always very suprising who different research is across different diciplines...

On-Kwok Lai from Kwansei Gakuin University gave a really interesting overview on the current situation in Asia and in particular in Japan with regards the aging society. Learning more about ageing I find myself more often thinking the current “aging research” is more like treating a symptom and not looking at the real problem. And it seems the real problem: reduced reproduction in industrial states – basically we do not have enough children anymore. This leads to the obvious question: would researching into solutions and technologies that make it easier to raise children while working or studying not be the more important challenge?

In another talk Birgit Kasper reported from a study of multi-modal travel in Köln ("Patenticket"). In the trail they got people who have a yearly ticket to introduce other older people to public transport by providing them a 3 month flat-rate ticket for public transport in the region. The benefits seem to come from two sides: (1) people do not worry if they have the right ticket and (2) having a person that acts as a patron learning the public transport system is supported. If we look at the results a radical suggestion would be to introduce a car-city-tax (e.g. like London) and give in return free public transport to everyone – would this simple solution not solve many of our problems (economic, ecological, …) or would it create a two-tier society?

The social event was at castle Frankenstein – but surprisingly everyone came back in the morning unharmed ;-)

History and Future of Computing and Interaction

Today I was teaching my class on user interface engineering and we covered a selected history of HCI and looked at the same time at a potential future. We discussed how user interface evolved and where UI revolutions have happed. To my question "What is the ultimate user interface?" I got three very interesting answers (1) a keyboard, (2) mind reading, and (3) a system that anticipates what I want. 

With regard to history in HCI one of my favorite texts is the PhD dissertation of Ivan Sutherland [1]. The work described was done in 1960-1963 when the idea of personal computing was very far from main stream. Even just browsing some of the pages gives an impression of the impact the work had…

For future user interfaces we talked about brain computer interfaces (BCI) and how they very much differ from the idea of mind reading. I came across a game controller - Mindlink - developed by Atari (1984) and that was never released [2]. It was drawing on the notion of linking to the mind but in fact it only measured muscle activity above the eye brows and apparently did not perform very well. However there is a new round coming up for such devices, see [3] for a critical article on consumer BCI.

On the fun side I found a number of older videos that look at future technology predictions- see the videos for yourself:

• Home Shopping in 1999 (from 1967): http://www.youtube.com/watch?v=uO58SGiYwwo
  
• 1999 AD Kitchen of the Future (from 1967): http://www.youtube.com/watch?v=yFgVTUle_EM 
  
• AT&T 1993 "You Will" Ads: http://www.youtube.com/watch?v=TZb0avfQme8
  
• Vision of year 2000 from year 1957: http://www.youtube.com/watch?v=I7XYXRKIqeY
  

http://www.paleofuture.com one is a site that has an amazing (and largely funny) selection of predictions. There is a more serious - but nevertheless - very entertaining article on predictions for computing and ICT by Friedemann Mattern: Hundert Jahre Zukunft - Visionen zum Computer- und Informationszeitalter (hundred years future - predictions of the computing and information age) [4].

[1] Sutherland's Ph.D. Thesis, Sketchpad, A Man-Machine Graphical Communication System. 1963 http://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-574.pdf

[2] The Atari Mindlink. http://www.atarimuseum.com/videogames/consoles/2600/mindlink.html

[3] Emmet Cole. Direct Brain-to-Game Interface Worries Scientists. Wired. 09.05.07. http://www.wired.com/medtech/health/news/2007/09/bci_games

[4] Friedemann Mattern.Hundert Jahre Zukunft - Visionen zum Computer- und Informationszeitalter. Die Informatisierung des Alltags - Leben in smarten Umgebungen, Springer Verlag 2007. http://www.vs.inf.ethz.ch/publ/papers/mattern2007-zukunft.pdf

Ideas in Advertisment, Privacy, German Law

In our master course we offer a project on pervasive advertisement (it is an interdisciplinary course project from computer science and marketing) where we look at future forms of advertisement that become possible by new technologies.

The students presented a set of really exciting ideas - an I would expect (if they get some of their ideas implemented) that advertising will be more entertaining and fun in the future. For some of the ideas we discussed potential privacy issues and I promised to provide later the reference to the German privacy law that restricts the use of optical/camera devices in public spaces. The German law is at a first glance very restrictive with regard to using cameras in public spaces. In short it can be summarized that data can only obtained for a legitimate, concrete and defined purpose and that the privacy interest of the people are not higher to value as the purpose. Additionally it has to be clear to the person observed that he or she is observed. (We probably need a lawyer to figure out what is allowed ;-) In [1] the text of the law (in German) can be found.

[1] Bundesdatenschutzgesetz. § 6b Beobachtung öffentlich zugänglicher Räume mit optisch-elektronischen Einrichtungen 

Lucia and Thomas from Vodafone R&D visiting

Lucia Terrenghi and Thomas Lang from Vodafone R&D in Munich visited our lab. We talked at lot about the future role of mobile devices and in particular how they may change personal computing in the near future. 

After lunch they gave a talk for our students describing Vodafone research and their particular research interests. Using a nice visualization of train lines they showed the research themes and introduced some of their research foci. One area of interest is electronic paper, resulting future devices and potential applications and services. In the discussion I briefly mentioned that I had a look at some of the displays with my microscope - have a look in the previous blog post if you are interested.

Privacy - will our understanding change radically?

As one issue this morning we came across issues related to privacy. In particular it seems that social network analysis based on behavior in the real world (e.g. the reality mining project [1]) is creating serious interest beyond the technology people. Beyond measuring the frequency of encounters qualifying the way people interact (dominance, emotion, …) will reveal even more about social networks… 

In our discussion I made a reference to a book: "The Transparent Society" by David Brin. Even Though it is now nearly 10 years since it was first published I still think it is an interesting starting point for a privacy discussion.

[1] Eagle, N. and (Sandy) Pentland, A. 2006. Reality mining: sensing complex social systems. Personal Ubiquitous Comput. 10, 4 (Mar. 2006), 255-268. DOI= http://dx.doi.org/10.1007/s00779-005-0046-3 

[2] The Transparent Society: Will Technology Force Us to Choose Between Privacy and Freedom? David Brin, Basic Books (June 1, 1999). At Amazon

2 day faculty meeting in Essen

The last 2 day we had a meeting - with all faculty of the ICB - discussing the future challenges of work at universities in the context of our current situation. For me the this time was very well spent as I got to know many of my colleagues better and it was incredible to see the potential (from a scientific perspective as well as looking at the people) we have in our organization.

To me it is always amazing how much of a difference it makes to have a professional external moderator running such workshops (and we were very lucky with Klaus Schneider Ott from focus-team). Even though many of the psychological games are well known to many of us - they still work well and move discussion forward and help in creating common ground and even increase trust. He lectured us (after much of the discussion has happened) on basics of communication (e.g. transactional analysis) and again not novel they lead to serious reflection and progress.

PS: Do you know how to continue the row in the image below? What is the next sign? …

If you think of M-Omega-8 you are wrong - but still it is really easy ;-)

Trip to Dublin, Aaron's Display Project

Visiting Dublin is always a pleasure - even if the weather is rainy. Most of the day I was at Trinity College reading master theses (which is the second best part of being external examiner, best part is to have lunch at the 1592 ;-)

In the evening I met with Aaron Quigley and we talked about some ongoing display and advertsing projects in our groups. He told me about one of their recent workshop papers [1] on public displays where they investigated what people take in and what people remember of the content on displays in an academic environment. It is online available in the workshop proceedings of AIS08 [2]. I found it worthwhile to browse the whole workshop proceedings.

[1] Rashid U. and Quigley A., "Ambient Displays in Academic Settings: Avoiding their Underutilization", Ambient Information Systems Workshop at UbiComp 2008, September 21, Seoul, South Korea (download [2], see page 26 ff)

[2] Proceedings of the Ambient Information Systems Workshop at UbiComp 2008
http://eniac.hopto.org/~whazlewo/downloads/AIS08_Full_Proceedings.pdf (~8MB)

Richard Atterer defended his PhD-thesis

At LMU in Munich Richard Atterer defended his PhD thesis on Usability Tool Support for Model-Based Web Development. The external examiner was Prof. Martin Gaedke- it was great meeting Martin again, we shared an office some years back at TecO, University of Karlsruhe. While I was with LMU I worked with Richard on a number of interesting topics and I learned to value his technical skills and insightful reflection - hope he stays in academia ;-) 

Our experiments with remote usability assessment and effectively fine-grain user tracking on web pages [1] created a number of ideas for follow-up projects. The overall concept is really simple yet powerful: people use a proxy server (willingly or transparent) and by these means Javascript code is included in the html-source of arbitrary web pages to add new functionality [2]. Florian Alt extended this concept to an annotation system and currently we look into more general implication of this approach.

While writing his theses Richard countered our routine question "how is progress?"  with a dynamically generated graph on his webpage. Each time he checked his document in SVN the graph was updated with the current number of words and pages ... but we still kept asking ;-) 

 [1] Atterer, R. and Schmidt, A. 2007. Tracking the interaction of users with AJAX applications for usability testing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (San Jose, California, USA, April 28 ? May 03, 2007). CHI '07. ACM, New York, NY, 1347?1350. DOI= http://doi.acm.org/10.1145/1240624.1240828

[2] Atterer, R., Wnuk, M., and Schmidt, A. 2006. Knowing the user's every move: user activity tracking for website usability evaluation and implicit interaction. In Proceedings of the 15th international Conference on World Wide Web (Edinburgh, Scotland, May 23 ? 26, 2006). WWW '06. ACM, New York, NY, 203?212. DOI= http://doi.acm.org/10.1145/1135777.1135811 

Article on Tangible UIs in the c't-Magazine

The current issue of the ct-Magazine (one of the largest computing magazines in Germany) features an introductory article on tangible user interfaces (in German).

The article is based on three interviews (Eva Hornecker, Reto Wettach, Albrecht Schmidt) and provides a good overview on this topic for a general audience. The article is in c’t no 21, 2007, p86-88 (no online version yet).

Mobile images / video as proof

While waiting for my conneting flight a saw a women posting a letter and filming this as she did it. She created some sort of proof. Thinking a little more and having further information in the background (e.g. a clock, the schedule display, people waiting) this this has some potential to replace registered mail for certain domains? This gives me an idea for a small weekend project...

Closing Panel at Ubicomp 2008

The closing panel at Ubicomp dicussed the last 10 years of ubicomp and potential future direction (with regard to community and technology). On the panel were Gregory Abowd, Hide Tokuda, Lars Eric Holmquist, Eric Paulos, and Albrecht Schmidt. In the following I will just describe some of the points I raised in my short statement. 

For me the first observation is that many ideas that were discussed at the first HUC99 (the first ubicomp conference started by Hans Gellsersen) - and were considered very speculative ideas have become common products and services by now (e.g. pocket bargain finder, predictive text input, mobile collaboration tool, mobile photo-sharing, location aware technologies). Here it is apparent that with regard to envisioning applications the conference has made impact. But there is a curious phenomenon: at the moment a device or service is available in the shop we do not recognize it as ubicomp anymore.

In some areas the complexity of the problems (when moving from the lab to the real world) has been underestimated - here context and context-awareness is a good example. If you realize the full vision it is basically solving AI. But nevertheless we progressed - there are applications for commercial mobile devices that do context and activity recognition - and it is just 10 years that we discussed this in a HUC99 in our paper on advanced interaction in context [1] - which was at that time really innovative! Context-awareness will happen - be patient :-) but its has to take into account: humans are adaptive, too. 

The papers which had a large impact (based on citations e.g. check [1] on google scholar) seem more the papers that score high on novelty, even if the may lack scientific rigor.

 

For future directions I hinted some general directions (not necessarily my research directions): 

• Implanted activity recognition and interaction (put the sensing and actuation into the body solves a lot of the problems … obviously it creates many new ones, too) 
  
• Implantable persuasion and amplifying bodily experiences. Here I gave the example that we would be able to create a device to motivate you do sports by making your back hurt. I used this to emphasise that ethics will play a large role in the future…
  
• Prediction technologies (e.g. the weather forecast as an inspiration, forecasting traffic conditions, parking situation, restaurant business, costs, …) we will create systems that allo us to look up predictions (cost, quality of the experience, stress, time needed, etc.) for future activities (e.g. when choosing a restaurants, booking a travel, deciding on dating a person, making a business deal, accepting a position, ...)
  
• And finally I suggested that we will have fun with papers on privacy published now when reading them in 20 years :-) because our perception of this topic will change massively.

With regard to the community I made the statement that Ubicomp became the Starbucks of ubiquitous computing research - premium but based on the US idea of quality. Have you ever been in a Vienna coffee house, in an Italian espresso bar, or had tea in the middle east - it is very different. We lost some of the international spirit and we stopped arguing what good research in ubicomp is - this discurse should be started again!. Looking at the countries where ubicomp technologies come from (e.g. a lot from Aisia and Europe) we should again make a effort to more value the international diversity and the different styles and approaches in ubicomp research - scientific rigor is not the only axis to consider. 

[1] Schmidt, A., Aidoo, K. A., Takaluoma, A., Tuomela, U., Laerhoven, K. V., and Velde, W. V. 1999. Advanced Interaction in Context. In Proceedings of the 1st international Symposium on Handheld and Ubiquitous Computing (Karlsruhe, Germany, September 27 - 29, 1999). H. Gellersen, Ed. Lecture Notes In Computer Science, vol. 1707. Springer-Verlag, London, 89-101. DOI= http://dx.doi.org/10.1007/3-540-48157-5_10

My Random Papers Selection from Ubicomp 2008

Over the last days there were a number of interesting papers presented and so it is not easy to pick a selection... Here is my random paper selection from Ubicomp 2008 that link to our work (the conference papers link into the ubicomp 2008 proceedings in the ACM DL, our references are below):

Don Patterson presented a survey on using IM. One of the finding surprised me: people seem to ignore "busy" settings. In some work we did in 2000 on mobile availability and sharing context users indicated that they would respect this or at least explain when interrupt someone who is busy [1,2] - perhaps it is a cultural difference or people have changed. It may be interesting to run a similar study in Germany.

Woodman and Harle from Cambridge presented a pedestrian localization system for large indoor environments. Using a XSens device they combine dead reckoning with knowledge gained from a 2.5D map. In the experiment they seem to get similar results as with a active bat system - by only putting the device on the user (which is for large buildings much cheaper than putting up infrastructure).

Andreas Bulling presented work where he explored the use EOG goggles for context awareness and interaction. The EOG approach is complementary to video based systems. The use of gesturest for context-awarenes follows a similar idea as our work on eye gestures [3]. We had an interesting discussion about further ideas and perhaps there is chance in the future to directly compare the approaches and work together.

In one paper "on using existing time-use study data for ubiquitous computing applications" links to interesting public data sets were given (e.g the US time-use survey). The time-use surevey data covers the US and gives detailed data on how people use their data.

University of Salzburg presented initial work on an augmented shopping system that builds on the idea of implicit interaction [4]. In the note they report a study where they used 2 cameras to observe a shopping area and they calculated the "busy spots" in the area. Additional they used sales data to get best selling products. Everything was displayed on a public screen; and an interesting result was that it seems people where not really interesting in other shoppers behavior… (in contrast to what we observe in e-commerce systems).

Researchers from Hitachi presented a new idea for browsing and navigating content based on the metaphor of using a book. In is based on the concept to have a bendable surface. In complements interestingly previous work in this domain called Gummi presented in CHI 2004 by Schwesig et al.

[1] Schmidt, A., Takaluoma, A., and Mäntyjärvi, J. 2000. Context-Aware Telephony Over WAP. Personal Ubiquitous Comput. 4, 4 (Jan. 2000), 225-229. DOI= http://dx.doi.org/10.1007/s007790070008
[2] Albrecht Schmidt, Tanjev Stuhr, Hans Gellersen. Context-Phonebook - Extending Mobile Phone Applications with Context. Proceedings of Third Mobile HCI Workshop, September 2001, Lille, France.
[3] Heiko Drewes, Albrecht Schmidt. Interacting with the Computer using Gaze Gestures. Proceedings of INTERACT 2007.
[4] Albrecht Schmidt. Implicit Human Computer Interaction Through Context. Personal Technologies, Vol 4(2), June 2000

Our Posters at Ubicomp 2008

In the poster session we showed 3 ideas of ongoing work from our lab and got some interesting comments and had good discussions. 

Ali presented initial findings from our experiments with multi-tactile output on a mobile phone [1]. He created a prototype with 6 vibration motors that can be individually controlled. In the studies we looked at what locations of the vibration elements and what patters can be spotted by the user. In short it seems that putting the vibration motors in the corners works best.

Florian showed ideas on counting page impressions on public advertising screens we sketched out with Antonio's Group in Münster [2]. The basic idea is to use sensors to get an idea of the number of people passing by. To calibrate such a system (as the sensor observations are only incomplete) we propose to use existing ways of counting people (e.g. access gates to public transport) and extrapolate based on this information.

I presented Dagmar's poster on initial work on providing information on the fuel economy in an engaging way to the user [3]. In a focus group study we observed that people are more aware of the price of a journey when using public transport than when using a car. For the car they know well the price per liter but have to calculate the price for a typical trip (e.g. to work). We suggest ideas where one can compete with others (e.g. from the social network) in saving energy on a specific route.

[1] Alireza Sahami Shirazi, Paul Holleis, Albrecht Schmidt. Rich Tactile Output for Notification on Mobile Phones (2-page paper, poster). Adjunct proceedings of Ubicomp 2008, Seoul, Korea, p26-27

[2] Albrecht Schmidt, Florian Alt, Paul Holleis, Jörg Müller, Antonio Krüger. Creating Log Files and Click Streams for Advertisements in Physical Space (2-page paper, poster). Adjunct proceedings of Ubicomp 2008, Seoul, Korea, p28-29

[3] Dagmar Kern, Paul Holleis, Albrecht Schmidt. Reducing Fuel Consumption by Providing In-situ Feedback on the Impact of Current Driving Actions (2-page paper, poster), Adjunct proceedings of Ubicomp 2008, Seoul, Korea, p18-19

How long before traditional TV will be marginalized?

TV and media consumption changes and one gets aware of this especially here in Seoul. People watch mobile TV on the subway and watching youtube videos in the hotel is fun as the available bandwidth seems massive. At the same time there is a convergences in technologies (TV hardware and UI still looks much the same but on the insight they are some sort of PC) is apparent and it takes little imagination to picture a TV set that integrates traditional services (e.g. TV over cable, terrestric, satellite) with new services (e.g. youtube, basically all flash-based video portals) in a transparent way. I would guess such a UI could be created in a way that the user does not really see the difference between a video from youtube or from BBC (only that he cannot fast-forward the BBC one). 

Given this technical prediction we discussed over dinner when traditional TV will be marginalized (in Europe). We could not really agree how we could tell that the traditional TV has been marginalized; One indicators we discussed is there will be no commercial TV stations (as we know them now) that provide a full program with a schedule broadcast. 

Based on this we made our predictions (if I got you wrong please correct it in the comments):

Jakob Bardram: never (just the carrier will change to IP); Alireza Sahami: 8 years; Florian Alt, 14 years, Jani Mantyjarvi, 7 years; Steinar Kristoffersen, 12 years; Nick Villar: 10 years; Chris Kray: 15 years; Albrecht Schmidt: 12 years

For most people live broadcast was one of the issues that they though may keep the traditional stations living longer. But I would argue we will have with the next generation of mobile devices means for broadcasting live, too… The final question is if people really go for professional high quality content over home-made content - I am not sure…

Perhaps we explore an implementation of an integrated UI in our course on user interface engineering in the coming winter term or if good student looks for a project topic.

PS: Steinar added that paper business cards will disappear befor the TV...

Keynote at Ubicomp 2008, Dr. Shim Yoon

The opening keynote at Ubicomp 2008 was presented by Dr. Shim Yoon (Vice President Samsung SDS) on Realizing Ubiquitous Cities. She stated out with a few  scenarios of life in the future. Information is embedded into the environment - just in place and just in time - just what you need. Environments become pleasant and technologies look nice, so you can mistake a power distribution box for a sculpture. Cities will be really save as monitoring will be ubiquitous and continuous. Much of the vision is convincing and convenient (given that you have given up on privacy, too). 

There has been an interesting slide in the scenarios that  made me smile (in short "22:47, pick up the book reserved in advance for lending after taking a walk"). It basically says we will still have pretty long days and books will be around, too. 

I think in the age of ebooks I will not pick up books late in the evening. Looking at the analogies of previous technical revolutions one can argue that these reduced the number of hours people had to work to have a good life - it seems this technical revolution creates the opposite (u-work everywhere and anytime ;-) - is fine with me because I like my work :-)She described how technologies changes the urban culture, e.g. provision of water, later roads and harbors, and very recent fiber optics and mobile communications. In her view ubiquitous technologies will add a further paradigm shift. 

The overall concept of u-city can be condensed to her statement:  "u-city can solve complicated urban problems" and this is attempted by combining political, physical and techno solutions. It is envisioned having IT included in all city elements. The approach follows a layered design with a base layer of the communication infrastructure (e.g. fixed and wireless infrastructure). In the middle there are the u-facility arrangements (e.g. sensor network, CCTV, power distribution), and on top u-services (u-home, u-office, u-traffic, …) are provided. 

It is envisioned that u-services are partly free (provided by the government) and partly

 commercial. Some of the examples for services see straight forward and very close (e.g. u-urban facility management that allows to monitor, check, maintain the infrastructure) and some are somehow scary (e.g. u-safety and security allows complete monitoring over the while city, ranging from fire recognition to behavioral analysis of its inhabitants). She pointed out that u-service modeling needs to include technical aspects as well as a business analysis.

The importance Korea sees in this topic is amazing and is summarized by the following quote "U-city is a key aspect in the next generation industry and economy in Korea". The Korean government is behind the developments and urban planning embraces the concept and puts it at the center of the planning. Looking at the u-urban information concept it becomes apparent that this concept is valuable along the entire lifecycle, including planning, simulation, management and operation. One catch is that it has to be integrated while building, otherwise it will be expensive and it is not possible to gain the full potential. This is the bad message for most developed countries as we usually change our cities and do not build them from scratch…

Explaining more on the realization of u-city she argued that a new process is required and that construction and IT processes have  to go together. During city planning a u-city consulting process is performed (creating an IT master plan for the new city including the envisioning of the services). In the building process the IT-infrastructure is included (when it is easiest to realize). Running and managing the city includes operating U-city (basically running the IT services by a specific UbiCenter - a control center for the city).

In the final part of her keynote she talked a little about the Samsung U-City strategy. Here she saw a necessity to merge the city construction with the provision of IT capabilities. This ranges from  developing convergence Services (e.g. ubiquitous communication service, city control an & security services) to the creation of comprehensive services for complex housing environments. An example of a infrastructure device she mentioned is a complex lighting pole (which could include information display, LED lighting, CCTV camera, wireless access points, etc.). In summary the strategy is the development (and potentially the export) of world class u-city technologies. 

In the question session I raised the issue if this may lead to even more people moving from the rural areas to cities because of the new qualities such u-cities will provide. And it seems there will be even a greater difference between rural areas and cities in the future and this may strengthen the trend towards mega-cities. Perhaps we should think a bit more about u-rural…

Doctoral colloquium at Ubicomp 2008

In the doctoral colloquium at Ubicomp 2008 we saw an interesting mix of topics including work on context-awareness, interaction in smart space, home infrastructures and urban environments. Overall there is again the observation that in ubicomp topics are very broad (at least in the beginning) and that it is not easy to narrow it down.

As Ali works on tactile feedback it was very interesting to see the presentation of Kevin Li on eyes-free interaction. He has an upcoming paper at UIST 2008 which is worthwhile to check out [1]. It was interesting some of the questions that relate to "easily learnable" or "intuitive" related to the discussion we had 2 weeks ago at the automotive UI workshop - what are tactile stimuli that are natural and we associate meaning with them without explanations or learning?
There are many more papers to read if you are interested in tactile communication and output, here are two suggestions [2] and [3].

[1]. Li, K. A., Baudisch, P., Griswold, W.G., Hollan, J.D. Tapping and rubbing: exploring new dimensions of tactile feedback with voice coil motors. To appear in Proc. UIST'08.

[2] Chang, A. and O'Modhrain, S., Jacob, R., Gunther, E., and Ishii, H. ComTouch: design of a vibrotactile communication device. Proc. Of DIS'02, pp. 312-320.

[3] Malcolm Hall, Eve Hoggan, Stephen A. Brewster: T-Bars: towards tactile user interfaces for mobile touchscreens. Mobile HCI 2008: 411-414

PS: Just one remark on the term "framework" (a favorite word to use in dissertation and paper titles) - it is not a clear term and expectations are very different, hence it make sense to think twice before using it ;-)

Which way did you fly to Korea?

We got a new USB GPS tracker(from Mobile Action, GT100) and had to try it out on the trip to Korea. It worked very well compared to the other devices we had so far. It got the bus trip in Düsseldorf airport right and the entire flight from Amsterdam to Seoul. Tracking worked well in the taxi from the Airport to the hotel. While walking in downtown Seoul it still performed OK (given the urban canyons) with some outliers.

It did not get any signal while we were on the Fokker-50 from Düsseldorf to Amsterdam :-( I slept a few hours on the flight to Seoul but I think someone took a photo (probably of me) over Mongolia… If you wonder if it is allowed to used your GPS in the plane or not – it is – at least with KLM (according to a random website http://gpsinformation.net/airgps/airgps.htm :-)

Back in Korea, Adverts, Driving and Entertainment

On the way into town we got a really good price for the taxi (just make a mental note never to negotiate something with Florian and Alireza at the same time ;-) It seems taxi driving is sort of boring – he too watched television while driving (like the taxi driver some weeks ago in Amsterdam). I think we should seriously think more about entertainment for micro breaks because I still think it is for a good reason not allowed to watch TV while driving.

Seoul is an amazing place. There are many digital signs and electronic adverts. Walking back to the hotel I saw a large digital display on a rooftop (would guess about 10 meters by 6 meters). If working it is probably nice. But now it is mal functioning and the experience walking down the road is worsened as one inevitably looks at it. I wonder if in 10 years we will be used to broken large screen displays…   

What can you alarm clock do? Platform for the bedside table

I have learned about Chumby an interesting platform that is designed to replace devices on your bedside table. Looking forward to get one or some when I fly next time to the US.

For a design competion at the appliance design conference I did a design concept for a networked alarm clock [1] assuming that networked device will be soon cheaply available. Maybe we should look at the paper again and think about how to push such ideas forward as the devices are on the market…

[1] Schmidt, A. 2006. Network alarm clock (The 3AD International Design Competition). Personal Ubiquitous Computing Journal. 10, 2-3 (Jan. 2006), 191-192. DOI= http://dx.doi.org/10.1007/s00779-005-0022-y

Buying Music Online – how easy is it?

Imagine there is a song – you know band and title – and you want to buy it. Should not be really something worthwhile reporting in a blog…

How long does it take to buy a song and how many steps does it need? I tried myself and was pretty much amazed that it is still more difficult than other ways to get music. The idea was to put the song into my shopping cart, press check-out, pay by credit card, and download. On the stores I encountered you have to register before to buy… I finally got the song and here are the steps at a major German music store: go to shop page, search for song, put in shopping cart, go to checkout, told to register, fill in registration form, told to confirm email, opened email client, waited 3 minutes for email, confirmed email, logged in on webpage, realized shopping card is empty :-(, search for song, put in shopping cart, go to checkout, entered credit card information, pay about 1.69€, got download link, got music.

I really wonder how many people will become first time buyers in this shop. Sometimes I think the things we teach in User Interface Engineering are obvious – but real life tells me they are not! If you run a music download portal or if you are in the music business and you wonder why no-one buys – we can tell you :-) it may be about utility and usability of your online offers… if you need more details we are happy to help you :-)

PS: there was a store with a .ru address with better usability that offered the song with no registration at 0.20€ - but I did not want to give my credit card details… 

Workshops at Informatik 2008 in Munich, e-ink prediction

Yesterday there was a workshop on Mobile and Embedded Interaction as part of Informatik2008 in Munich. The talks and discussions were very interesting. Lucia and Thomas raised interesting issues on a new notion of personal computing, where the mobile device becomes the center of a personal computing infrastructure. This idea has been around for some time (e.g. Roy Wants Personal Server [1]) but the new ideas and the feasibility with current hardware makes it really an exciting topic. On the general topic there are many open questions, as visible on the slide.

After the workshop, when swapping business cards, we started the discussion when in the future we will have business cards (in larger quantities, to give away) that have active display elements (e.g. eInk) included. Everyone gave a predictions in how many years we will have it (Lucia Terrenghi:never; Raimund Dachselt:7; Thomas Lang: business card will disappear; Albrecht Schmidt:9; Heiko Drewes:10; Florian Echtler:5; Michael Rohs:5; Paul Holleis:5). Lets get back in 5 years and see… In September 2008 the Esquire Magazine featured an e-ink cover page - have not seen it myself:-( but there is a video: http://www.esquire.com/the-side/video/e-ink-cover-video

Today we organized a workshop on Software, Services and Platforms for new infrastructures in telecommunication. We had a set of really interesting talks. As I did my PhD on context-awareness I was quite impressed by work on context oriented programming and the advances over the last years in this domain (good starting point on the topic with some publications [2]).

At the end of the workshop I gave the following scenario as an impulse for discussion: image there are 10 million facebook users that contniouly stream the video of what they see into the net, e.g. using eagle-i. The discussion raise many technical as well as social challenges!

[1] Want, R., Pering, T., Danneels, G., Kumar, M., Sundar, M., and Light, J. 2002. The Personal Server: Changing the Way We Think about Ubiquitous Computing. In Proceedings of the 4th international Conference on Ubiquitous Computing (Göteborg, Sweden, September 29 - October 01, 2002). G. Borriello and L. E. Holmquist, Eds. Lecture Notes In Computer Science, vol. 2498. Springer-Verlag, London, 194-209.

[2] http://www.swa.hpi.uni-potsdam.de/cop/

PS: there are few photos as someone in the workshop today objected to be on the net…

Workshop on Automobile User Interfaces

For the second time we ran this year a workshop on automobile user interfaces and interactive applications in the car at the German HCI conference: http://automotive.ubisys.org/

In the first session we discussed the use of tactile output and haptics in automotive user interfaces. It appears that there is significant interest in this area at the moment. In particular using haptics as an additional modality creates a lot of opportunities for new interfaces. We had a short discussion about two directions in haptic output: naturalistic haptic output (e.g. line assist that feels like going over the side of the road) vs. generic haptic output (e.g. giving a vibration cue when to turn).

 I think the first domain could make an interesting project – how does it naturally feel to drive too fast, to turn the wrong way, to be too close to the car in front of you, etc…

In a further session we discussed framework and concepts for in-car user interfaces. The discussion on the use of context with the interface was very diverse. Some people argued it should be only used in non-critical/optional parts of the UI (e.g. entertainment) as one is not 100% sure if the recognized context is right. Others argue that context may provide a central advantage, especially in safety critical systems, as it gives the opportunity to react faster. 

In the end it comes always down to the question: to what extent do we want to have the human in the loop… But looking at Wolfgang's overview slide it is impressive how much functionality depends already now on context...

In the third session we discussed tools and methods for developing and evaluating user interfaces in the car context. Dagmar presented our first version of CARS (a simple driving simulator for evaluation of UIs) and discussed findings from initial studies [1]. The simulator is based on the JMonkey Game engine and available open source on our website [2].

There were several interesting ideas on what topics are really hot in automotive UIs, ranging from interfaces for information gather in Car-2-Car / Car-2-Envrionment communication to micro-entertainment while driving.

[1] Dagmar Kern, Marco Müller, Stefan Schneegaß, Lukasz Wolejko-Wolejszo, Albrecht Schmidt. CARS – Configurable Automotive Research Simulator. Automotive User Interfaces and Interactive Applications – AUIIA 08. Workshop at Mensch und Computer 2008 Lübeck 2008

[2] https://www.pcuie.uni-due.de/projectwiki/index.php/CARS

PS: In a taxi in Amsterdam the driver had a DVD running while driving – and I am sure this is not a form of entertainment that works well (it is neither fun to watch, nor is it save or legal).