Albrecht Schmidt - User Interface Engineering

Facebook – a platform to spot when companies go bankrupt? Real world example.

2012-01-21T00:34:00.000+01:00

In the Germany the drug store chain Schlecker announced to be insolvent, see the Reuter news post. If you look at the company’s Facebook page and scan the comments from the last 4 weeks it is apparent that some people in the crowd and employees expected it already last year.
Schlecker is a large drug store chain with probably over 10.000 outlets in Europe and more than 30.000 employees.

The following screen shots show some selected examples I took from the following page: http://www.facebook.com/schlecker.drogerie
The posts are in German – the minimal summary should give you some idea…

This one the company wishes a happy Christmas and reminds people of a chance to win a car. The first replies echo the holiday greetings but then one complains that they let their shops bleed out (run empty) and that the order good do not arrive (probably posted by an employee). One further speculates that the company is close to bankruptcy. (over 3 weeks before the official note of insolvency)

The company announces a 2 euro discount on a product. Then employees post that they would like to sell the goods to the customers but that they do not get the goods for their shops. Additionally they complain that the goods they get from other closed down shows are not what they need. One says we want to work but we cat (as they are running out of stock). (over 2 weeks before the official note of insolvency)

The company announces price reductions in some goods. Some says that is great – but would be much better if these goods would be in the shops to buy them. (9 days before the official note of insolvency)

Overall I think this is an instructive real world example of the information that can be found in social networks about the health/value of companies. In particular the mix of customers and employees posting makes it a good example to study. I would expect that companies will learn lessons from this with regard to guidelines for the employees… and about transparency / openness…to understand how reliable such posts are we probably need to do some more research? let us know if you are interested in working this with us.

Congratulations to Frau Doktor Dagmar Kern for a great PhD defense (No. 5)

2012-01-20T23:29:00.000+01:00

Dagmar Kern has successfully defended her PhD on “Supporting the Development Process of Multimodal and Natural Automotive User Interfaces” in Essen. External examiner was Antonio Krüger from University of Saarbrücken. Her dissertation will be available online soon. The core contribution of the thesis is the investigation of how to improve a user centered design process for automotive user interfaces. In order to systematically assess user interface designs in cars she developed a design space (inspired by Card et al [5]). In various cases studies she create novel in-car user interfaces and explored experimentally the implications on driver distraction.

Dagmar started working with me as a student of Media Informatics at the LMU Munich in 2005, then jointed my group at Fraunhofer IAIS/BIT in Bonn and move in 2007 with the group to Essen. She was for a short research stay in Saarbrücken and Milton Keynes and was extremely productive over the last years – 18 publications she co-authored are listed in DBLP and here are some highlights of here research:

exploration of how to present navigation information (e.g. vibra tactile steering wheel) [1]
gazemarks - an approach to aid attention switching between the road and an in car display using eye gaze date [2]
a multi-touch steering wheel, that reduced driver distraction [3]
a design space for automotive user interfaces [4]

Additionally to the publications one of the side products of here thesis is the CARS open source driving simulator. It is a configurable low cost simulator that can be used to measure driver distraction, e.g. as an alternative to LCT.

Dagmar’s defense brought us back to Essen and it was great to meet many colleagues again. We finally managed to have a group photo taken with nearly all the team (Elba is missing in the Photo).

The doctoral hat may look strange to non-Germans but it has some funny tradition. It is hand crafted by the colleagues and each of the items on the hat tells a story – usually known to the group but in the best case hard to guess for outsiders. Besides others Dagmar’s hat included a scrap heap of cars, a giraffe, a personal vibration device, a yoyo, a railway station building side, and a steering wheel cover.

[1] Dagmar Kern, Paul Marshall, Eva Hornecker, Yvonne Rogers, and Albrecht Schmidt. 2009. Enhancing Navigation Information with Tactile Output Embedded into the Steering Wheel. InProceedings of the 7th International Conference on Pervasive Computing (Pervasive '09). Springer-Verlag, Berlin, Heidelberg, 42-58. DOI=10.1007/978-3-642-01516-8_5 (free PDF)

[2] Dagmar Kern, Paul Marshall, and Albrecht Schmidt. 2010. Gazemarks: gaze-based visual placeholders to ease attention switching. In Proceedings of the 28th international conference on Human factors in computing systems (CHI '10). ACM, New York, NY, USA, 2093-2102. DOI=10.1145/1753326.1753646 (free PDF)

[3] Tanja Döring, Dagmar Kern, Paul Marshall, Max Pfeiffer, Johannes Schöning, Volker Gruhn, and Albrecht Schmidt. 2011. Gestural interaction on the steering wheel: reducing the visual demand. In Proceedings of the 2011 annual conference on Human factors in computing systems (CHI '11). ACM, New York, NY, USA, 483-492. DOI=10.1145/1978942.1979010 (free PDF)

[4] Dagmar Kern and Albrecht Schmidt. 2009. Design space for driver-based automotive user interfaces. In Proceedings of the 1st International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '09). ACM, New York, NY, USA, 3-10. DOI=10.1145/1620509.1620511 (free PDF)

[5] Stuart K. Card, Jock D. Mackinlay, and George G. Robertson. 1991. A morphological analysis of the design space of input devices. ACM Trans. Inf. Syst. 9, 2 (April 1991), 99-122. DOI=10.1145/123078.128726

Auto-UI 2012 in the US, looking for hosts for 2013

2011-12-02T20:12:00.001+01:00

The next and 4rd international conference on Automotive User Interfaces and Vehicular Applications (AutoUI 2012) will be in Portsmouth, New Hampshire in the USA. The dates for the conference are 17-19 of October 2012. The first day for workshops and tutorials and 2 days for the main conference. Portsmouth is about an 1 hour drive from Boston and the timing is great (fall foliage – the photos of the colorful forests looked good ;-)

The steering committee (sc@auto-ui.org) is inviting proposals for Auto-UI 2013 from the community of researchers in the field. The conference was 2009 in Essen (Germany), 2010 in Pittsburgh (USA), 2011 in Salzburg (Austria), and it will be in 2012 in Portsmouth (USA). Keeping this cycle between Europe and North America 2013 should be in Europe.

Paper and demo in Salzburg at Auto-UI-2011

2011-12-02T13:34:00.001+01:00

At the automotive user interface conference in Salzburg we presented some of our research. Salzburg is a really nice place and Manfred and his team did a great job organizing the conference!

Based on the Bachelor Thesis of Stefan Schneegaß and some follow-up work we published a full paper [1] that describes a KLM-Model for the car and a prototyping tools that makes use of the model. In the model we look at the specific needs in the car, model rotary controllers, and cater for the limited attention while driving. The prototyping tool provides means to quickly estimate interaction times. It supports visual prototyping using images of the UI and tangible prototyping using Nic Villar´s VoodooIO. Looking forward to having Stefan on our team full-time :-)

We additionally had a demo on a recently completed thesis by Michael Kienast. Here we looked at how speech and gestures can be combined for controlling functions, such as mirror adjustments or windscreen wipers, in the car. This multimodal approach combines the strength of gestural interaction and speech interaction [2].

The evening event of the conference was at Festung Hohensalzburg - with a magnificent view over the twon!

[1] Stefan Schneegaß, Bastian Pfleging, Dagmar Kern, Albrecht Schmidt. Support for modeling interaction with in-vehicle interfaces. (PDF) Proceedings of 3rd international conference on Automotive User Interfaces and Vehicular Applications 2011 (http://auto-ui.org). Salzburg. 30.11-2.12.2011

[2] Bastian Pfleging, Michael Kienast, Albrecht Schmidt. DEMO: A Multimodal Interaction Style Combining Speech and Touch Interaction in Automotive Environments. Adjunct proceedings of 3rd international conference on Automotive User Interfaces and Vehicular Applications 2011 (http://auto-ui.org). Salzburg. 30.11-2.12.2011

Bryan Reimer: Opening keynote at Auto-UI 2011 in Salzburg

2011-12-01T11:00:00.004+01:00

Bryan started his keynote talk the automotive user interface conference (auto-ui.org) in Salzburg with reminding us that having controversial discussions about the HMI in the car is not new. Quoting a newspaper article from the 1930s on the introduction of the radio in the car and its impact on the driver he picked an interesting example, that can be seen as the root of many issues we have now with infotainment systems in the car.

The central question he raised is: how to create user interface that fit human users? He made an important point: humans are not “designed” to drive at high speed in complex environments; perception has evolved for walking and running in natural environment. Additionally to the basic limitations of human cognition, there is a great variety of capabilities of drivers, their skills and cognitive ability (e.g. influence of age). A implication of the global change is demographics is that the average capabilities of a drivers will be reduced – basically as many older people will be drivers…

Over the last 100 years cars have changes significantly! Looking more closely Bryan argues that much of the chance happened in the last 10 years. There has been little change from the 1950s to the 1990s with regard to the car user interface.

It is apparent that secondary tasks are becoming more important to the user. Users will interact more while driving because the can. It is however not obvious that they are capable of it.

Even given these developments it is apparent that driving has become safer. Passive safety has been improved massively and this made driving much safer. There seems to be a drawback to this as well, as people may take greater risks as they feel safer. The next step is really to avoid accidence in the first place. Bryan argues that the interaction between driver, environment, and vehicles is very important in that. He suggests that we should make more of an effort to create systems that fit the drivers.

The Yerkes-Dodson Law helps to understand how to design systems that keep peoples attention in the optimal performance. He made an important point: there are certain issues that cannot be solved, e.g. if someone is tired we can do only very little – the driver will need to rest. We should make sure that we take these things into account when designing systems.

Visual distraction is an obvious factor and much discussed in the papers at the conference – but Bryan argued that “eyes on the road” is not equal to “mind on the road”. I think this is really a very important point. Ensuring that people keep their eyes on the road, seeing things is not enough. The big resulting question is how to keep or get people focused on the street and environment. It seems there is some more research to do…

The variety of interfaces and interaction metaphors build into cars opens more choices but at the same time creates problems, as people need to learn and understand them. A simple question such as: How do you switch the car off? may be hard to answer (Bryan had the example of a car with a push button starter, where you cannot remove the key). I think there are simple questions that can be learned from industry and production machines… add an emergency stop button and make it mandatory ;-)

If you are interested more about Bryan’s work look at his webpage or his page at the MIT agelab or one of his recent publications [1] in the IEEE Pervasive Computing Magazine’s special issue on automotive computing, see [2] for an introduction to the special issue.

Sorry for the poor quality photos ... back row and an iPhone...

[1] Joseph F. Coughlin, Bryan Reimer, and Bruce Mehler. 2011. Monitoring, Managing, and Motivating Driver Safety and Well-Being. IEEE Pervasive Computing 10, 3 (July 2011), 14-21. DOI=10.1109/MPRV.2011.54 http://dx.doi.org/10.1109/MPRV.2011.54

[2] Albrecht Schmidt, Joseph Paradiso, and Brian Noble. 2011. Automotive Pervasive Computing. IEEE Pervasive Computing 10, 3 (July 2011), 12-13. DOI=10.1109/MPRV.2011.45 http://dx.doi.org/10.1109/MPRV.2011.45

Automotive UI in Salzburg

2011-12-01T09:42:00.001+01:00

Manfred Tscheligi opend the Automotive UI conference in Salzburg. The conference is now in its 3rd year after 2009 in Essen and 2010 in Pittsburgh. The conference is growing – there were well over 130 people registered :-)

The proceedings of the conference series are online available at http://auto-ui.org

3rd Auto-UI Proceedings 2011 (soon in the ACM DL)
2nd Auto-UI Proceedings 2010 (ACM DL)
1st Auto-UI Proceedings 2009 (ACM DL)

Guests in my multimodal interaction class

2011-11-24T20:42:00.001+01:00

Today I had brought 3 more professors with me to teach the class on multimodal interaction (I learned from Hans). As we have the pd-net project meeting Nigel Davies, Marc Langheirich, and Rui Jose were in Stuttgart and ‘volunteered’ to give a talk.

Nigel talked about the work in Lancaster on the use of mobile computing technology to support sustainable travel. He explained the experiments they conducted for collecting and sharing travel related information. In the 6th Sense Transport project they look beyond looking at understanding the current context into predictions and eventually ‘time travel’ ;-)

Marc presented a one hour version of his tutorial on privacy introducing the terminology and explaining the many facets this topic has. We discussed the ‘NTHNTF’ argument (Nothing To Hide Nothing To Fear) and Marc used an example of AOLstalker.com to show the weaknesses of this argument. Marc suggested some reading if you want to dive into the topic, see [1,2,3,4].

Rui focused in his lecture on pervasive public displays. He gave an overview of typical architectures for digital signage systems and the resulting limitation. The pd-net approach aims at creating an open platform that allows many different applications and use cased. He showed once concept of using virtual pin-badges to trigger content and to express interest in a certain topic.

There is more information on the pd-net project on http://pd-net.org

[1] David Brin. The Transparent Society. Perseus Publishing, 1999.
[2] Simson Garfinkel: Database Nation – The Death of Privacy in the 21st Century. O’Reilly, 2001.
[3] Lawrence Lessig: Code and Other Laws of Cyberspace. Basic Books, 2006. http://codev2.cc/
[4] Waldo, Lin, Millett (eds.): Engaging Privacy and Information Technologygy in a Digital Age. National Academies Press, 2007.

Call for Papers: Symposium on Pervasive Display Networks

2011-11-24T10:45:00.000+01:00

Rui José and Elaine Huang are chairing an international symposium on pervasive displays in Portugal. The conference will be held June 4-5 2012 in Porto. The submission deadline for full papers is January 16th, 2012.

With our research in the PD-net project we encounter many interesting research questions and met with many other researchers interested in the topic. It seems that the many real deployments of electronic displays is fueling ideas and makes it obvious that research is required to understand the properties of this new upcoming media. The call states: “As digital displays become pervasive, they become increasingly relevant in many areas, including advertising, art, sociology, engineering, computer science, interaction design, and entertainment.”

We hope with this symposium we will bring together researchers and practitioners as well as users to share research results and generate new ideas.

Submissions that report on cutting-edge research in the broad spectrum of pervasive digital displays are invited, ranging from large interactive walls to personal projection, from tablets and mobile phone screens to 3-D displays and tabletops. Topics include:

Novel technologies
Architecture
Applications
Domains and formative studies studies
Evaluations and deployments
Interfaces and interaction techniques
Content design

Have a look at the webpage and the call for paper at http://pervasivedisplays.org/cfp.php

Hippy, purple hair, piercing, ..., facebook? How to rebell

2011-11-21T09:05:00.000+01:00

Inspired by some discussion on the implication of information sharing at AMI2011 I wanted to put this up for discussion here...

If you read newspapers you find a lot of people have major concerns about how young people are using social networks, and especially facebook. I believe many of them are well meaning when the speculate about the lasting damage young people do to themselves if the post too much too openly. Some of the concerns I share but I think there is another dimension to it, too.

Being young (e.g. teenager) is not about being sensible, reasonable, rational - it is about exploring the world and rebelling. Probably most of us looked for way to provoke reactions from parents and society in this phase of live. Some examples to remember... Hippies and sex in 70’s, green, blue, red and orange hair in the 80’s, in then in 90’s it was piercing. Now what can the young people do today? Granny had a skirt that was really short; parents had piercings in places where you don't want to think about…. It is easy - sharing a picture on facebook where you wear to little or nothing - and you get all the reactions. Especially you will get the same reaction that has been around for many decades (and your parent and grandparents got, too): if you do this no one will ever give you a job ;-)

I don't want to deny the risks of sharing information online, but I think we should analyze things a bit more deeply ...

Closing Keynote at AMI2011, Beyond Ubicomp – Computing is Changing the Way we Live

2011-11-18T22:33:00.001+01:00

On Friday afternoon I had the privilege to present the closing keynote at AMI2011 in Amsterdam with the title ‘Beyond Ubicomp – Computing is Changing the Way we Live’. The conference featured research in Ambient Intelligence ranging from networking and system architecture to interfaces and ethnography. It brought an interesting set of people together and it was good to see many students and young researchers presenting their work.

In my closing keynote at talked about my experience of the last 13 years in this field and about a vision of the future. My vision is based on a basic technology assessment - basically looking what technologies will (in my view) definitely come over the next 20 years and looking at the implications of this. I stared out with a short reference to Mark Weiser's now 20 year old article [1]. The upcoming issue of IEEE Pervasive Magazine will have a in-depth analysis of the last 20 years since Weiser' article - we have also an article in there on how interaction evolved.

The vision part of the talk looked "Perception beyond there here and now" [2] from 3 different angles:

Paradigm Shift in Communication
Here I argue that the default communication in the future will be public communication and only if something is secret we will try to use non public channel. First indicators of this are a switch from email to twitter and facebook. I used a cake baking example to highlight the positive points of this shift.
Steep Increase in media capture
The second angle is just observing and extrapolating the increase in capture of media information. If you go already now on youtube you will information about many things (backing a cake, repairing a bike, etc.). The implication of this increase in media capture will be virtually unlimited access to experience other people share
Transformation of experienced perception
The final angle is that this creates a new way of perceiving the world. We will extent perception beyond the here and now and this is bringing a completely new way of creating and accessing information. I used the example of enquiring about buying an international train ticket at the station in Amsterdam. If you can look there through other people’s eyes the question becomes trivial.

My overall argument is that we are in for a major transformation of our knowledge and information culture. I would expect that this shift is as radical as the shift from an oral tradition to the written societies - but the transition will be much quicker and in the context of a globalized and competitive world.

The main conclusion from this is: Ethics and values are the central design material of this century.

Looking at twitter it seems it got across to some in the audience ;-) If your are interested, too have a look at the slides from the keynote.

[1] Mark Weiser. The computer for the 21st century. Scientific American, Vol. 265, No. 3. (1991)
[2] Albrecht Schmidt, Marc Langheinrich, and Kritian Kersting. 2011. Perception beyond the Here and Now. Computer 44, 2 (February 2011), 86-88. DOI=10.1109/MC.2011.54 http://dx.doi.org/10.1109/MC.2011.54

Opening Keynote at AMI 2011: Margaret Morris

2011-11-18T01:54:00.001+01:00

Margaret (Margi) Morris presented the opening keynote at the 2011 conference on ambient intelligence in Amsterdam (AMI2011) with the title “Left to our own devices”.

Margaret brought up an interesting point on motivation: Showing people what they lose is a stronger motivator than the prospective of gain. She made the point in order to implement this the depicted loss has to be very specific. She showed a facebook application “With a little help from my friends”, where this basic concept is applied. I had recently seen a bill board adverting campaign for safe driving on motorways in Germany using this approach (basically showing the risk of loss of family).

In the talk several examples of devices and applications were presented. To learn more about her work I recommend the following two papers: at tool to improve emotional self-awareness [1] and an investing in social networks and their utility to promote health [2].

Another point that made me think was the question of how we design interventions. One conceptual example was about an obesity campaign. The official UK campaign starts out with the statement that obesity is a problem for 9 million kids. Her alternative is to provide instead of the information a specific hint about an opportunity for action for an individual (e.g. telling the kid when it leaves school in the afternoon: now is probably a good time to play soccer with your friends, as 16 of them like to play soccer). An open research question that relates to this seems to me to investigate the impact of information about the norm, e.g. how will it affect my behavior if I know that 70% of my friends think driving too fast is OK vs. if I know that only 20% find it acceptable. I think this could be further explored in the context of social networks to create interesting persuasive technologies.

There has been an interesting discussion after the talk. Norbert Streitz questioned if it is a good idea to ask people to engage more with digital devices (e.g. self monitoring one’s mood). The question is hinting that the engagement with the digital device keeps us from interaction in the “real” world. I think this separation is disappearing fast – making a phone call, listening to MP3, chatting with friends on facebook is for many of us real, we live in a world that is augmented by technology and the boundaries are bluring...

[1] Morris ME, Kathawala Q, Leen TK, Gorenstein EE, Guilak F, Labhard M, Deleeuw W. Mobile Therapy: Case Study Evaluations of a Cell Phone Application for Emotional Self-Awareness. Journal of Medical Internet Research 2010;12(2):e10. URL: http://www.jmir.org/2010/2/e10/

[2] Margaret E. Morris. 2005. Social Networks as Health Feedback Displays. IEEE Internet Computing 9, 5 (September 2005), 29-37. DOI=10.1109/MIC.2005.109 http://dx.doi.org/10.1109/MIC.2005.109

Opening at AMI2011: Emile Aarts

2011-11-18T00:12:00.002+01:00

Today the conference on ambient intelligence 2011 (AMI2011) has started in Amsterdam. Emile Aarts reflected on what Ambient Intelligence is and was.

He looked back of some of the concepts that were well received in the research community but did not make it to the market (e.g. iCat). He made a very important point on how to involve users in the research: Don’t ask the customer –look at what he is doing. (he attributed it to someone from booking.com but it has been around for a while, e.g. in Jakob Nielsen's Alertbox 2001). Even if this is well known I think it was great to remind us again! It is still the most important points in user centered research that is often completely missed!

He suggested that the most important topic in ambient intelligence is lighting. It seems very simple but he had some very interesting example (e.g. a hospital or an airport) where light has a measureable impact on people’s experience.

For some more impressions of his talk, see the random slide selection below.

ESSPRITS 2011 workshop, keynote

2011-10-20T21:57:00.000+01:00

I was invited to present a keynote at the joint German-Canadian workshop on embedded systems, signal processing, and IT security (ESSPRITS 2011). The workshop is interdisciplinary and technically focused on embedded systems.

My talk was entitled “Challenges in a World of Ubiquitous Computing” and it looked at how the world changed over the last 20 years as ubiquitous computing has become reality. Computing technologies have become an integral part of our life and they shape more and more how we perceive the world and how we interact with each other. The talk highlights some technology trends that enable a new generation of computing systems. It presents a new vision suggesting new forms of perception and interaction without temporal and spatial boundaries.

During the lectures it was great to get a feel of what research is happening on a signal processing level. With regard to security I got the feeling that we need in the future a closer cooperation between people doing cryptography and those doing user interfaces. It seems it is currently still at the level that HCI people understand little about the algorithms, methods and concerns in IT-security and that people in security optimize for security that can be proved, but may not fit humans (and may not really increase security in the real world).

Complex circles, decision-making, expectations, plausible deniability

2011-08-05T00:49:00.000+01:00

Google+ circles are on a conceptual basis well argued (e.g. the much talked about real world analogy) but it seems they do not to well for many of us. I though I share my limited observations in a blog post (if I would have done a real study I would publish it in a top conference ;-)

To me deciding what circles I need and where to put people in these circles is pretty hard – ok I am in academia and this is not a typical environment (separation of work, hobby, friends...). Which of my co-workers are friends; do I differentiate between students in a course and the ones who do a thesis with me. Who belongs to “family” or do I need 5 or more categories to describe my family? It seems the number of circles is growing equally fast than the number of friends. Its probably just me who can not discriminate between different parts of live.

The implications of the many circles is that I have to make many more decisions than on facebook. If I accept an invitation it is a yes/no/not now decision in facebook (about 300-500ms plus the time to click ;-) … much longer with circles. When I post it is again time for making decisions – whom to include and who not to include.

The main issue with circles is for me the responsibility in sharing. In theory this is the great advantage – but in real live I think it is not (it is just a way of keeping old way of communication alive for some more time - if I want to address specific people I can use email ;-). As the others know that I have the choice to limit sharing to circles the expectation is that I manage this well. With whom should I share my unhappiness about a too long faculty meeting – thinking in circles – probably no one (or only the people waiting for me). Who should know that I have read an interesting article about planting bamboo – again in circles – probably only my wife because she asked me about it.

In summary this privilege (or the responsibility) to be able to specify whom we share information with make the posts much more predictable. I share with the HCI community the calls for papers, links to surveys we need participants, and the great papers we published, I share with the family the nice photo from our weekend hike, and I share with my students a link to a great article in the pervasive magazine they should read. Given my option to share to groups, sharing a photo of my daughter and me building a pneumatic lift with my students and colleagues would be inappropriate. However I argue that to share beyond circles – sharing things we would usually not share with this group – is what makes my facebook stream so much more exciting that the google+ stream. The comments of the people who I would not have included in a circle based addressing are the once which are often most interesting. From an information theoretical point of view the facebook stream has more entropy and carries massively more information as it is less predictable.

… and in facebook we (still) have an excuse (sort of plausible deniability) as there is no real responsibility for the sender to limit the receivers – it just a binary responsibility of is it OK to share or not.

cfp: IEEE Special Issue on Interaction Beyond the Keyboard

2011-07-11T21:30:00.000+01:00

IEEE Computer will have a special issue on "Interaction beyond the Keyboard" ... and till Nov 1st 2011 you still have a chance to submit :-)

--- from the call (http://www.computer.org/portal/web/computingnow/cocfp4) ---
Final submissions due: 1 November 2011
Publication date: April 2012

IEEE Computer seeks submissions for an April 2012 special issue on interaction beyond the keyboard.

Interaction with computers has become an integral part of daily life for most people. When making a phone call, listening to music, taking a photo, getting money from an ATM, or driving a car, we operate computer systems with complex functionalities. As technologies progress, the proliferation of computing technologies increases, and simple user interfaces and ease of use are becoming key success factors for a wide range of products.

Although the keyboard and mouse are still the dominant user interfaces in home and office environments, with the massive increase in mobile device usage and the many new interaction technologies available, the way we interact with computers is becoming richer and more diverse. Touch-enabled surfaces, natural gestures, implicit interaction, and tangible user interfaces mark some of these trends.

The overall goal of interaction beyond the keyboard is to create natural and intuitive forms of human-computer interaction that make it easier for people to achieve their goals while using computers as tools.

For this special issue, we seek original research that describes groundbreaking new devices, methods, and approaches to human-computer interaction in a world of ubiquitous computer use. In particular, we're looking for exciting work that is concerned with the following topics:

interactive surfaces and tabletop computing;
mobile computing user interfaces and interaction while on the go;
tangible interaction and graspable user interfaces;
embedded user interfaces and embodied interaction;
natural interaction and gestures; and
user interfaces based on physiological sensors and actuators.

Articles should be understandable to a broad audience of computing science and engineering professionals. The writing should be practical and original, avoiding a focus on theory, mathematics, jargon, and abstract concepts. All manuscripts are subject to peer-review on both technical merit and relevance to Computer's readership. Accepted papers will be professionally edited for content and style.

-----

please see: http://www.computer.org/portal/web/computingnow/cocfp4

Percom 2012 - call for papers

2011-07-05T02:28:00.002+01:00

Percom2012 - Call for papers as PDF or as text-file.

PerCom 2012
IEEE International Conference on Pervasive Computing and Communications
March 19 - 23, 2012, Lugano, Switzerland

CALL FOR PAPERS

IEEE PerCom, now in its 10th edition, has established itself as the premier annual scholarly venue in the areas of pervasive computing and communications. Pervasive computing and communications has evolved into an active area of research and development, due to the tremendous advances in a broad spectrum of technologies and topics including wireless networking, mobile and distributed computing, sensor systems, RFID technology, and the ubiquitous mobile phone.

PerCom 2012 will be held in Lugano, an international city and the crossroads and melting pot of European culture. PerCom 2012 will provide a leading edge, scholarly forum for researchers, engineers, and students alike to share their state-of-the art research and developmental work in the broad areas of pervasive computing and communications. The conference will feature a diverse mixture of interactive forums: core technical sessions of high quality cutting-edge research articles; targeted workshops on exciting topics; live demonstrations of pervasive computing in action; insightful keynote speeches; panel discussions from domain experts; and posters of budding ideas. Research contributions are solicited in all areas pertinent to pervasive computing and communications, including:

- Innovative pervasive computing applications
- Context modeling and reasoning
- Programming paradigms for pervasive systems
- Software evolution and maintenance in pervasive systems
- Middleware services and agent technologies
- Adaptive, autonomic and context-aware computing
- Mobile/Wireless computing systems and services in pervasive computing
- Energy-efficient and green pervasive computing
- Communication architectures for pervasive computing
- Ad hoc networks for pervasive communications
- Pervasive opportunistic communications and applications
- Enabling technologies for pervasive systems (e.g., wireless BAN, PAN)
- Positioning and tracking technologies
- Sensors and RFIDs in pervasive systems
- Multimodal sensing and context for pervasive applications
- Pervasive sensing, perception and semantic interpretation
- Smart devices and intelligent environments
- Trust, security and privacy issues in pervasive systems
- User interface, interaction, and persuasion
- Pervasive computing aspect of social network software
- Virtual immersive communications
- Wearable computers
- Standards and interfaces for pervasive computing environments
- Social and economic models for pervasive systems

Workshops and affiliated events:

Many workshops will be held in conjunction with the main conference. Workshop papers will be included and indexed in the IEEE digital libraries (Xplore), showing their affiliation with IEEE PerCom. As in the past, PerCom 2012 will also feature a PhD Forum, Demonstrations and a Work-in-Progress Session. Please see the website www.percom.org for details on current and past PerCom conferences.

Important Dates
Paper Registration: Sep 23, 2011
Paper Submission: Sep 26, 2011
Author Notification: Dec 20, 2011
Camera-ready Due: Jan 27, 2012

Submission Guidelines
Submitted papers must be unpublished and not considered elsewhere for publication. They must show significant relevance to pervasive computing and networking. Only electronic submissions in PDF format will be considered. Papers must be 9 pages or less, including references, figures and tables (at least 10pt font, 2-column format). The IEEE LaTeX and Microsoft Word templates, as well as formatting instructions, can be found at the conference web site. Submissions will undergo a rigorous review process handled by the Technical Program Committee. The best paper will receive the prestigious Mark Weiser Best Paper Award. Top selected papers will be considered for a special issue of the Elsevier journal of Pervasive and Mobile Computing (PMC)

For additional information, see www.percom.org for details on current and past PerCom conferences, or contact the PerCom 2012 organizing committee at percom2012@supsi.ch

Organizing Committee

General Co-Chairs
Silvia Giordano, SUPSI, CH
Marc Langheinrich, Univ. of Lugano, CH

Program Chair
Albrecht Schmidt, Univ. of Stuttgart, DE

Vice Program Co-Chairs
Jie Liu, Microsoft Research, USA
Georges Roussos, Univ. of London, UK
Alexander Varshavsky, AT&T Labs, USA

Workshops Co-Chairs
Pedro Marron, Univ. Duisburg-Essen, DE
Marius Portmann, Univ. of Queensland, AU

Steering Committee Chair
Marco Conti, IIT-CNR, IT

Our Article one Phones as Components of Future Appliances is published in IEEE Pervasive Magazine

2011-07-01T01:01:00.001+01:00

In this paper we reflect the opportunities that arise from using consumer devices, such as phones and mp3 players, as components for future devices. With this article also a new department on Innovations in Ubicomp Products has been started. The article “Phones and MP3 Players as the Core Component in Future Appliances” [1] is also available openly in at ComputingNow.

The rational is

developing a custom embedded computer is expensive
specific devices are not economic for small quantities
phones are becoming cheap (in small quantities a phone may be cheaper than buying a touch screen component for an embedded device)
development on phones has become easy and many developers are around
IO capabilities can be added to these devices (e.g. Project HiJack)

The main question is: why not use the consumer device as a part (potentially partly hidden) as computing platforms in new devices? There are examples but also some difficulties… read the article to get a more in-depth discussion.

[1] Albrecht Schmidt and Dominik Bial. 2011. Phones and MP3 Players as the Core Component in Future Appliances. IEEE Pervasive Computing 10, 2 (April 2011), 8-11. DOI=10.1109/MPRV.2011.31 http://dx.doi.org/10.1109/MPRV.2011.31 (also available in ComputingNow, download PDF)

Summer school in St Andrews, Teaching Context-Awareness

2011-06-29T02:10:00.000+01:00

I had the privilege to teach a course on context-awareness [1] as part of the SICSA Summer School on Multimodal Systems for Digital Tourism. The summer school was directed by Aaron Quigley (University of St Andrews), Eva Hornecker (University of Strathclyde), Jon Oberlander (University of Edinburgh) and Stephen Brewster (University of Glasgow).

It was very exciting to discuss with the students ideas for novel digital devices to support tourists and come up with new concepts in this domain. Ideas ranged from interactive umbrellas (taking the concept described in [2] further) to digital souvenirs that ensure a lasting memory.

On Monday night Chris Speed gave an inspiring talk on ghosts, memories, and things reflecting on history, the Internet of things and how we perceive the world around us in a very though provoking way. Hi inspired us to think about the stories and memories that surround us and that are inherently linked to all things humans us. … it was in a long time a story about ghosts that made a lot of sense :-)
When going back we saw a great example of a security system that is based on physical constraints... you can open it from the inside but not from the outside:

Aaron asked me to talk on context-awareness. I did the talk along the lines of a soon to appear chapter on www.interaction-design.org. To me one of the – still remaining – fundamental challenges in HCI with context-aware systems is that the system as well as the human is adaptive. And as people learn often incredibly fast the adaptation may be contra-productive, hence it is essential to take this into account. Have a look at my slides if you like to learn more about context-awareness and HCI.

When we were there, we learned that St. Andrews is the place to play golf – the old course is where you need to go. Looking more closely it became clear that this is for others ;-) but there is a option for the rest of us. It is called the The Ladies putting Club St.Andrews “Himalayas” – just walk in and play (2 pound per person, and no need to book a year ahead). And if your friends don’t play golf you get away with the photos you take there as it is only 5 meters from the old course.

[1] http://dl.dropbox.com/u/5633502/talk/context-aware-systems-004-print-small.pdf
[2] Sho Hashimoto, Takashi Matsumoto. The Internet Umbrella. http://www.pileus.net/

Somnometer – A Social Alarm Clock – Users Wanted!

2011-06-22T19:18:00.000+01:00

We have continued our work on the social alarm clock for Android phones. The Somnometer App can be used as a regular alarm clock but offers functions to:
(1) rate your sleep
(2) monitor your sleep duration (manually based on wake-up time)
(3) have graphical representations of the sleep quality and duration
(4) optionally share some of this information with your friends on facebook

Are you interested in trying this alarm clock application? Please have a look at the app home page (http://somnometer.hcilab.org) or download it from the android market.

We are looking for volunteers to participate in a study with this alarm clock application. If you are interested in the new functions and if you are an active facebook user, please contact us. There will also be a chance to take part in a comparative study using a different sleep monitoring device and the alarm application. Our email address for the project is: somnometer@hcilab.org.

Self-expression, Belonging, and Respect – Is Taking Risks Part of it?

2011-06-20T22:41:00.001+01:00

Seeing someone walking up the leaning tower in Pisa with shoes that were clearly not designed for this situation I wondered about the risks people take in live. We recently had a discussion (with other parents) on the risks kids take today in the digital world – put up regrettable pictures flickr, liking a politically incorrect site on facebook, or posting silly things on twitter.

I sometimes feel in these discussions that I want to put things into perspective… We do a lot of things that are not reasonable in order to express ourselves and to present an image to our peer group (e.g. tattoos and piercings are common and there are risks associated). We want to belong to a group and hence we do things that are expected by our peers or even to impress them (e.g. doing a skateboard trick without protection or skiing where it is not allowed). If think hard there are probably many things you remember where you took major risks (when you were young)… On TV I saw a yesterday night a documentary on the Hippie movement in the 1960/1970. In comparison to the risks young people took in order to change the world (or to just be different and accepted in their peer group) the risks you take on the Internet seem very tame…

There is a further point we can learn from this: eventually society (and the law) will catch up and some of the innovations will stay and change society. But some will no be accepted… People need to explore boundaries – otherwise progress is unlikely.

For many people who have explored boundaries in 1970ies (ranging from drugs to violence – in a way we have agreed today is completely unacceptable) this has not hindered their careers. People generally see actions in context… Hence having the “wrong” photo on facebook is probably not harming someone’s career (but probably the time they spend on facebook rather than revising for exams may).

Gestural Input on a Touch Screen Steering Wheel in the Media

2011-06-17T20:22:00.000+01:00

At CHI 2011 we presented initial work on how to use gestural input on a multi-touch steering wheel [1]; a 20 second video is also available [2]. The paper described a prototype - a steering wheel where the entire surface is a display and can recognize touch input. The study had two parts. In the first part we identified a natural gesture set for interaction and in the second part we looked at how such interaction impacts the visual demand for the diver. The results in short: using gestural input on the steering wheel reduces the visual demand for the driver.

Shortly after the conference a journalist from discovery news picked the topic up and did some interviews. This resulted in an article: "Touch-Screen Steering Wheel Keeps Eyes on Road" (Discovery News, 6.6.2011)

ACM Tech News mentioned the Discovery News article News (ACM Tech News June 8 2011).
After this it found its way around and appeared more widely than expected :-) examples include

"Touch-screen steering wheel keeps drivers focused on the road" (Physorg, 6.6.2011)
"German Researchers Working On Multi-touch Gestures Steering Wheel" (Softsailor, 15.6.2011)
"New Touchscreen Steering Wheel May Reduce Driver Distractions" (Extreme Tech, 14. 6. 2011)
"Researchers invent touchscreen steering wheel" (Sympatico.ca, 13.6.2011)
"Touch Screen Steering Wheel Aims To Improve Driving Safety" (PSFK, 13.6.2011)
"Researchers working on Touch Schreen Steering Wheel" (Pistonheads, 13.6.2011)

There were also a German article "Touchscreen-Lenkrad Wischen wechselt Radiosender" (sp-x, 14.6.2011), e.g. found in:

[1] Tanja Döring, Dagmar Kern, Paul Marshall, Max Pfeiffer, Johannes Schöning, Volker Gruhn, and Albrecht Schmidt. 2011. Gestural interaction on the steering wheel: reducing the visual demand. In Proceedings of the 2011 annual conference on Human factors in computing systems (CHI '11). ACM, New York, NY, USA, 483-492. DOI=10.1145/1978942.1979010 http://doi.acm.org/10.1145/1978942.1979010

[2] http://www.youtube.com/watch?v=R_32jOlQY7E Gestural Interaction on the Steering Wheel - Reducing the Visual Demand. chi2011madness Video.

Keynote at EICS 2011

2011-06-17T15:49:00.000+01:00

I was invited to present a keynote at EICS 2011 in Pisa. In the talk "Engineering Interactive Ubiquitous Computing Systems" I motivated why user interface engineering approaches are well suited for creating user interfaces in the context of embedded and ubiquitous computing systems. Looking at desktop applications and mobile devices I think the quality and ease of use is high - compared to 20 years back or compared to embedded and ubiquitous computing systems. I think a lot of user interface research, and in particular engineering approaches for interactive systems, could have a great impact on real world systems beyond the desktop or phone.

As one example of an engineering process for embedded user interfaces I shared our experience with developing Gazemarks [1]. Gazemarks is a technology based on eye-gaze tracking that reduced the time required for attention switching. It eases tasks that require the user to move attention repeatedly between 2 or more displays or between the real world and a set of digital displays. Application domains could be looking at the street and at the satnav while driving or switching attention between a screen in an operating theatre and the patient.

When investigating development from an embedded user interface to interactive ubiquitous computing systems further issues come up. As we investigate the PDnet project [2] with public displays we see that the concerns of the stakeholders play a much bigger role than in traditional systems and that finding an appropriate business model is very close to the user interface development process.

In the final part of the talk I shared a future vision of how technology may change the way we live. In the not so distant future we could imagine that the traditional boundaries of perception (mainly temporal and spatial) will fall [3]. This would create an entirely new experience where "Perception beyond the here a now" change fundamentally the way we see and experience the world. The slides of the keynote are available in as PDF.

From the research we did over the last 15 years I picked some lessons learned:

Novelty may be about the values/ethics
Implement it and try it out!
20% who like the UI/system are a large market
Humans are smart and adaptive
Design for creative users

[1] Dagmar Kern, Paul Marshall, and Albrecht Schmidt. 2010. Gazemarks: gaze-based visual placeholders to ease attention switching. In Proceedings of the 28th international conference on Human factors in computing systems (CHI '10). ACM, New York, NY, USA, 2093-2102. DOI=10.1145/1753326.1753646 http://doi.acm.org/10.1145/1753326.1753646

[2] Pdnet project homepage: http://pd-net.org/

[3] Albrecht Schmidt, Marc Langheinrich, and Kritian Kersting. 2011. Perception beyond the Here and Now. Computer 44, 2 (February 2011), 86-88. DOI=10.1109/MC.2011.54 http://dx.doi.org/10.1109/MC.2011.54

CHI 2011 in Vancouver, Keynote and Papers

2011-05-17T13:57:00.026+01:00

In the opening keynote Howard Rheingold proclaimed that we are in a time for learners and he outlined the possibilities that arise from the interactive media that is available to us. In particular he highlighted the fact that people share and link content and to him this is at the heart of learning. Learning as a joined process where contributions by students - in different forms of media - become major a resource was one example.

I best liked his analogy on how little innovation there is in teaching. "If you take a warrior from 1000 years ago on a battlefield today - they will die - quickly. If you take a surgeon from a 1000 years ago and put them in a modern hospital - they will be lost. If you take a professor from 1000 years ago and put them in a University today he will exactly know what to do. " I am not sure about the 1000 years but it by 100 years the story works just as well. In essence he argued that there is a lot of potential for new approaches for teaching and learning.

After initially agreeing I gave it some more thoughts and perhaps the little change in learning and teaching shows that learning is very fundamental and technology is overrated in this domain? What is more effective than a teachers discussing in an exciting topic face to face with a small set of students - perhaps even while on a walk? Reminds me about things I read about the Greek teachers and there practices several thousand years ago … and it makes me looking forward to our summer school in the Italian Alps (http://www.ferienakademie.de/).

I found the SIGCHI Lifetime Achievement Award lectures very exciting and educational. Especially the talk by Larry Tesler provided deep insight into how innovation works in user interfaces - beyond the academic environment. He talked about the "invention" of cut and paste - very enjoyable!

This year we had a number of papers describing our research in CHI:

Elba reported on the field study in Panama using mobile phones to enhance teaching and learning [1]
Ali presented work on how to increase the connectedness between people by simple means of iconic communication in the context of a sports game [2]
Tanja showed how touch and gestural input on a steering wheel can reduce the visual distraction for a driver [3], and
Gilbert (from LMU Munich) presented work on interaction with cylindrical screens [4].

The most inspiring and at the same time the most controversial paper for me was the possessed hand by Jun Rekimoto et al. [5]. He reported their results in using electro stimulation in order to move fingers of a hand.

Bill Buxton showed throughout the conference his collection of input and output devices (Buxton Collection). Seeing the collection physically is really exciting, but for all who did not have the chance there is a comprehensive online version with photos and details available at micosoft research: http://research.microsoft.com/en-us/um/people/bibuxton/buxtoncollection/

[1] Elba del Carmen Valderrama Bahamondez, Christian Winkler, and Albrecht Schmidt. 2011. Utilizing multimedia capabilities of mobile phones to support teaching in schools in rural panama. In Proceedings of the 2011 annual conference on Human factors in computing systems (CHI '11). ACM, New York, NY, USA, 935-944. DOI=10.1145/1978942.1979081 http://doi.acm.org/10.1145/1978942.1979081

[2] Alireza Sahami Shirazi, Michael Rohs, Robert Schleicher, Sven Kratz, Alexander Müller, and Albrecht Schmidt. 2011. Real-time nonverbal opinion sharing through mobile phones during sports events. In Proceedings of the 2011 annual conference on Human factors in computing systems (CHI '11). ACM, New York, NY, USA, 307-310. DOI=10.1145/1978942.1978985 http://doi.acm.org/10.1145/1978942.1978985

[3] Tanja Döring, Dagmar Kern, Paul Marshall, Max Pfeiffer, Johannes Schöning, Volker Gruhn, and Albrecht Schmidt. 2011. Gestural interaction on the steering wheel: reducing the visual demand. In Proceedings of the 2011 annual conference on Human factors in computing systems (CHI '11). ACM, New York, NY, USA, 483-492. DOI=10.1145/1978942.1979010 http://doi.acm.org/10.1145/1978942.1979010

[4] Gilbert Beyer, Florian Alt, Jörg Müller, Albrecht Schmidt, Karsten Isakovic, Stefan Klose, Manuel Schiewe, and Ivo Haulsen. 2011. Audience behavior around large interactive cylindrical screens. In Proceedings of the 2011 annual conference on Human factors in computing systems (CHI '11). ACM, New York, NY, USA, 1021-1030. DOI=10.1145/1978942.1979095 http://doi.acm.org/10.1145/1978942.1979095

[5] Emi Tamaki, Takashi Miyaki, and Jun Rekimoto. 2011. PossessedHand: techniques for controlling human hands using electrical muscles stimuli. In Proceedings of the 2011 annual conference on Human factors in computing systems (CHI '11). ACM, New York, NY, USA, 543-552. DOI=10.1145/1978942.1979018 http://doi.acm.org/10.1145/1978942.1979018

Floor for activity recognition

2011-04-17T09:12:00.000+01:00

Patrick Baudisch showed in Paris at the Microsoft Software summit interesting photos of their DIY activity in creating a glass floor for tracking and activity recognition. With a fairly large glass pane in the floor they have created an interesting environment… I am sure there will be interesting things coming out of this installation.

Some years back in 2002 (and looking at the photos and the amount of hair I still had this seems long ago) in Lancaster we also looked in what to do with floors (and we were into DIY as well). We also considered arrangements with a floor tables and furniture on top. As you can see from Kristof, Hans and me on the photo it was a fun project.

The positive point in using load sensing is that you can track unobtrusive and potentially large scales with little instrumentation. We even considered the possibility to put a house on 4 load cells and do activity recognition based on this. We never got around to building the house ;-) The problem with load sensing is that you can only track one moving object/subject at the time.

Looking at the signature of the load measured and doing some signal processing we could detect events – unobtrusive and cheap – but only for single events.

Interested in more details? Have a look at the publications on load sensing [1], on the interaction [2], and at a patent [3] describing the basic technology.

[1] Schmidt, A., Strohbach, M., Laerhoven, K. v., Friday, A., and Gellersen, H. 2002. Context Acquisition Based on Load Sensing. In Proceedings of the 4th international Conference on Ubiquitous Computing (Göteborg, Sweden, September 29 - October 01, 2002). Springer-LNCS, London, 333-350.

[2] Schmidt, A.; Strohbach, M.; van Laerhoven, K. & Hans-W., G. 2003. Ubiquitous Interaction - Using Surfaces in Everyday Environments as Pointing Devices, Universal Access Theoretical Perspectives, Practice, and Experience (UI4ALL 2003), Springer LNCS, 263-279.

[3] Schmidt, A., Strohbach, M., Van Laerhoven, K., Friday, A., Gellersen, H-W., Kubach, U.; Context acquisition based on load sensing. US Patent 7434459. US Patent Issued on October 14, SAP AG (DE), 2008

WP7 Tutorial - part 5: Orientation and Acceleration - X,Y,Z

2011-04-11T21:21:00.000+01:00

Detecting gestures, orientations, and movement can be realized with the accelerometer. The accelerometer is a sensor that measures the acceleration in 3 dimensions (X, Y, and Z). If the device is not moved the accelerations measure are the gravity forces in each direction. If the device is accelerated the measured results are a combination of the acceleration and the gravity.

The accelerometer data can be accessed via the AccelerometerReadingEventArgs class. The class has values for the X, Y, and Z axis. The values are of type double and between -2 and 2 which related to acceleration "for each axis in gravitational units" - 1 is the gravitational force of the earth. See: http://msdn.microsoft.com/en-us/library/ff431744(v=vs.92).aspx and http://msdn.microsoft.com/en-us/library/ff431810(v=vs.92).aspx or on page 80ff, C. Petzold, Programming Windows Phone 7.

A typical exercise for understanding the accelerometer is to create a bubble level (a tool to measure if something is horizontal or vertical - e.g. for hanging pictures on the wall). You probably want to freshen up on arctan2 - at least I needed ;-)

See below the c# example reading out the accelerometer on a windows phone 7. You can also download the accelerometer project directory in a single ZIP-file.

using System;
using System.Windows;
using Microsoft.Phone.Controls;
using Microsoft.Devices.Sensors;

// A simple example to read the accelerometer and display the values
// In order to make it work you have to add the refercerence to 
// Microsoft.Devices.Sensors to your project. To do this right-click
// in the Solution Explorer on References and than choose add Reference
// in the dialog then select Microsoft.Devices.Sensors
// Albrecht Schmidt, University of Stuttgart

// for a more comprehensive example see: 
// http://msdn.microsoft.com/en-us/library/ff431810(v=vs.92).aspx
// http://msdn.microsoft.com/en-us/library/ff431744(v=vs.92).aspx
// and page 80ff, C. Petzold, Programming Windows Phone 7

namespace Accl_X_Y_Z
{
    public partial class MainPage : PhoneApplicationPage
    {
        Accelerometer accelerometer;
    
        public MainPage()
        {
            InitializeComponent();
            // create a new instance
            accelerometer = new Accelerometer();
            // register a callback function for when values change
            accelerometer.ReadingChanged += new EventHandler<AccelerometerReadingEventArgs>(accelerometer_ReadingChanged);
            // start the accelerometer
            accelerometer.Start();
        }

        void accelerometer_ReadingChanged(object sender, AccelerometerReadingEventArgs e)
        {
            // required as from here the textBlocks cannot be accessed
            Deployment.Current.Dispatcher.BeginInvoke(() => ChangeUI(e));
        }
        
        void ChangeUI(AccelerometerReadingEventArgs e)
        {
            // show the values on the screen
            textBlock1.Text = "X: " + e.X.ToString("0.000");
            textBlock2.Text = "Y: " + e.Y.ToString("0.000");
            textBlock3.Text = "Z: " + e.Z.ToString("0.000");
        }
    }
}

WP7 Tutorial - part 4: Storing Data on the Phone

2011-04-11T20:15:00.002+01:00

If you want to save high scores or preferences you need persistent memory on the phone. On a traditional computer you would create a file and store your information in the file; another option on a Windows PC would be store such information in the registry. For security reasons there is no API to access the file system and there is no global persistent memory across applications on a WP7.

In general there are two ways to store data: (1) in a application specific storage (isolated storage) on the phone or (2) remotely on the internet (or to use another buzzword "in the cloud").
In this example the use of the phone isolated storage API is demonstrated. It is shown how to store and retrieve name-value pairs on the phone (to people who programmed Java ME this is conceptually similar to the record store).

For more details see: http://msdn.microsoft.com/en-us/library/cc221360(v=VS.95).aspx and page 126ff, C. Petzold, Programming Windows Phone 7. It is also possible to created an isolated storage for files see: http://msdn.microsoft.com/en-us/library/system.io.isolatedstorage.isolatedstoragefile(v=VS.95).aspx

See below the c# example using the local storage on a windows phone 7. You can also download the IsolatedStorageSettings project directory in a single ZIP-file.

using System;
using System.Windows;
using Microsoft.Phone.Controls;
using System.IO.IsolatedStorage;

// example of how to save to and load from the isolated application storage 
// this helps to create persistence storage within a single application
// in this example it is shown how to do it for a string 
// Albrecht Schmidt, University of Stuttgart  


// For more details see:
// http://msdn.microsoft.com/en-us/library/cc221360(v=VS.95).aspx
// page 126ff, C. Petzold, Programming Windows Phone 7

// storing files/directory structures see:
// http://msdn.microsoft.com/en-us/library/system.io.isolatedstorage.isolatedstoragefile(v=VS.95).aspx

namespace PhoneStorage
{
    public partial class MainPage : PhoneApplicationPage
    {
        // Constructor
        public MainPage()
        {
            InitializeComponent();
        }

        #region Save and Load Parameters from the Application Storage
        void saveToAppStorage(String ParameterName, String ParameterValue)
        {
            // use mySettings to access the Apps Storage
            IsolatedStorageSettings mySettings = IsolatedStorageSettings.ApplicationSettings;

            // check if the paramter is already stored
            if (mySettings.Contains(ParameterName))
            {
                // if parameter exists write the new value
                mySettings[ParameterName] = ParameterValue;
            }
            else
            {
                // if parameter does not exist create it
                mySettings.Add(ParameterName, ParameterValue);
            }
        }

        String loadFromAppStorage(String ParameterName)
        {
            String returnValue = "_notSet_";
            // use mySettings to access the Apps Storage
            IsolatedStorageSettings mySettings = IsolatedStorageSettings.ApplicationSettings;

            // check if the paramter exists
            if (mySettings.Contains(ParameterName))
            {
                // if parameter exists write the new value
                mySettings.TryGetValue<String>(ParameterName, out returnValue);
                // alternatively the following statement can be used:
                // returnValue = (String)mySettings[ParameterName];
            }

            return returnValue;
        }
        #endregion

        private void button1_Click(object sender, RoutedEventArgs e)
        {
            saveToAppStorage("myIdentifer1", "Last used @ " + System.DateTime.Now.ToString("HH:mm:ss"));
            textBox1.Text = "saved...";
        }

        private void button2_Click(object sender, RoutedEventArgs e)
        {
            textBox1.Text = loadFromAppStorage("myIdentifer1");
        }

    }
}

Online German Language Corpus, UCREL Summer School

2011-03-27T21:42:00.005+01:00

At the University of Leipzig a German Language corpus is available (Projekt Deutscher Wortschatz). The database can be queried from different programming languages and access is also possible via a web service. Requests can ask for co-occurrences of words, base forms, about words that often occur to the right and to the left of the word, word frequency, synonyms and much more. If you develop text input systems this may be a very useful resource, see the web services overview page (with links to downloads), the list of web-service-requests offered or have a look at some php-examples.

You can try the service interactively at http://wortschatz.uni-leipzig.de/abfrage/. See the pictures for an example query on the term Internet. They also feature a German-English dictionary.

Since I shared and office at Lancaster University with Paul Rayson from UCREL (University Centre for Computer Corpus Research on Language) I find corpus linguistics an interesting topic. By the way UCREL runs a Summer School in Corpus Linguistics from 13 to 15 July 2011 - would love to go there...

WP7 Tutorial - part 3: Using Location

2011-03-25T15:22:00.006+01:00

In this example the use of the location API is demonstrated. The API is a high level interface to geo location. How the location is determined (e.g. GPS, GSM cell information) is of no concern to the developer.

The basic approach is to create an instance of GeoCoordinateWatcher and register two callback functions: one for when the status changes and one for when the location changes. The program demonstrates how these call backs are set up and how from within those function the user interface is updated with the received information. If the status is changes, the program checks what the current status is, and shows this in the status line (textBlock8.Text). If the position is changed then the new position information (Position.Location.Longitude, Position.Location.Latitude) - and additional information such as Speed, Altitude, Course, Accuracy are shown.

As an exercise you can build an application that shows you how close you are to a given target. In two input fields you enter the longitude and latitude of the destination (e.g. a geo cache location). And then you can calculate the difference from the current position to the target location and visualize or sonify the distance.

There is another example (Geo coordinate watcher) how to use this API on the Microsoft msdn website. In C. Petzold's book there is also a good example, see page 91ff.

See below the c# example using geo location on a windows phone 7. You can also download the geolocation project directory in a single ZIP-file.

using System;
using System.Collections.Generic;
using System.Windows;
using Microsoft.Phone.Controls;
using System.Device;
using System.Device.Location;

// the example shows the basic functionality of the location device
// you need to add in the solution explorer a reference to System.Device
// right click on References in the solution explorer, click Add Reference, and then
// System.Device
// Albrecht Schmidt, University of Stuttgart

// for a more comprehensive example see: 
// http://msdn.microsoft.com/en-us/library/system.device.location.geocoordinatewatcher.aspx
// http://msdn.microsoft.com/en-us/library/ff431744(v=vs.92).aspx
// and page 91ff, C. Petzold, Programming Windows Phone 7

namespace Geo_Location
{
    public partial class MainPage : PhoneApplicationPage
    {
        GeoCoordinateWatcher watcher;

        // Constructor
        public MainPage()
        {
            InitializeComponent();
        }

        // the initialize and start button is pressed
        private void button1_Click(object sender, RoutedEventArgs e)
        {
            // initialize the geo watcher with defaul accuracy (battery saving)
            // user GeoPositionAccuracy.High for higher accuracy
            watcher = new GeoCoordinateWatcher(GeoPositionAccuracy.Default);
            // set movement threhold - as distance in meters - default is 0
            watcher.MovementThreshold = 10;

            // add a handler that is called when position is changed more than MovementThreshold
            watcher.PositionChanged += new EventHandler<GeoPositionChangedEventArgs<GeoCoordinate>>(watcher_PositionChanged);
            // a handler for status change
            watcher.StatusChanged += new EventHandler<GeoPositionStatusChangedEventArgs>(watcher_StatusChanged);
            
            // Start reading location data
            watcher.Start();
        }

        void watcher_StatusChanged(object sender, GeoPositionStatusChangedEventArgs e)
        {
            // you cannot change the UI in this function -> you have to call the UI Thread
            Deployment.Current.Dispatcher.BeginInvoke(() => ChangeStatusUI(e));
        }

        void ChangeStatusUI(GeoPositionStatusChangedEventArgs e)
        {
            String statusType="";
            if ((e.Status) == GeoPositionStatus.Disabled)
            {
                statusType = "GeoPositionStatus.Disabled";
            }
            if ((e.Status) == GeoPositionStatus.Initializing)
            {
                statusType = "GeoPositionStatus.Initializing";
            }
            if ((e.Status) == GeoPositionStatus.NoData)
            {
                statusType = "GeoPositionStatus.NoData";
            }
            if ((e.Status) == GeoPositionStatus.Ready)
            {
                statusType = "GeoPositionStatus.Ready";
            }
            textBlock8.Text = statusType;
        }

        void watcher_PositionChanged(object sender, GeoPositionChangedEventArgs<GeoCoordinate> e)
        {
            // you cannot change the UI in this function -> you have to call the UI Thread
            Deployment.Current.Dispatcher.BeginInvoke(() => ChangeUI(e));
        }

        void ChangeUI(GeoPositionChangedEventArgs<GeoCoordinate> e)
        {
            textBlock1.Text = "Longitute: " + e.Position.Location.Longitude;
            textBlock2.Text = "Latitute:  " + e.Position.Location.Latitude;
            textBlock3.Text = "Speed:     " + e.Position.Location.Speed;
            textBlock4.Text = "Altitude:  " + e.Position.Location.Altitude;
            textBlock5.Text = "Course:  " + e.Position.Location.Course;
            textBlock6.Text = "Vertical Accuracy:  " + e.Position.Location.VerticalAccuracy;
            textBlock7.Text = "Horizontal Accuracy:  " + e.Position.Location.HorizontalAccuracy;
            textBlock8.Text = "location updated at " + System.DateTime.Now.ToString("HH:mm:ss");
        }

        // the stop button clicked ... stop the watcher
        private void button2_Click(object sender, RoutedEventArgs e)
        {
            if (watcher != null) { watcher.Stop(); }
            textBlock8.Text = "location reading stopped";
        }
    }
}

Percom 2011 in Seattle, keynote

2011-03-25T15:08:00.004+01:00

This year's Percom conference was held in Seattle and offered an exciting and diverse program. Have a look the program to see it for yourself. The two keynotes were both looking at the implications of pervasive computing and communication - especially when thinking about the data is collected and how the data may be used.

Alex Pentland from MIT talked about their work on reality mining. The work looks at how one can capture interactions between people and between people and their environment and how such information can be exploited. One example he gave was on looking at the effect of face to face communication on the performance on workers. The basics insights of this work are thrilling and thinking it through it becomes obvious that we are at the start of new era of mankind. The arguments he made that we can contain and control such information I did not find convincing and I think it may be dangerous to tell decision makers in politics that we can provide solutions. I see no way (that is not restricting people's freedom massively or which reduces productivity massively) that would allow to control the information that will become available through pervasive computing… and all the solutions I have heard either will plainly not work or would require a global agreement over data protection laws…

The keynote on the second day was by Derek McAuley from Nottingham University. One of his topics was on product history and how the availability of product history has the potential to increase the value of products. I think this is a very powerful concept and we will in the near future see this commercially exploited.
Furthermore Dereck discussed interesting issues that come up with crowd sourcing and participatory sensing. One central point is where the data is hold and who controls the data collected. Especially in the context of cloud services this becomes transparent and important at the same time. With regard to the implementation is does not matter; however from a legal perspective it may make a serious difference whether you cloud service runs in German, the US, or on a ship somewhere in the Atlantic. An example he gave are navigation systems in cars which have a back channel. The cars sent back information about their speed and whereabouts and the information is used to predict the state of the road, which is then used to improve the navigation. He raised the questions what happens if this information is held somewhere were legislation has no control? I think this is going to happen and there is no real approach against it…
He made a case that end-users (individuals) should be able to bring together information about them and make use of it. On principle I like this idea to put the individual into control and allow them to exploit this data. For me this is however not a solution for data protection, as a certain part of individuals will sell their data - and in a free country there is probably very little society can do against it.

In summary - we are heading towards an exciting future!

PS: Percom 2012 will be in Lugano with Silvia and Marc chairing the conference. And I have the honor to serve as program chair. See the web page for more information (will be available soon) or the photo of the call for papers here.

WP7 Tutorial - part 2: Vibration

2011-03-19T09:28:00.005+01:00

This examples shows how to activate the vibration motor / vibration actuator in the phone. The calls Microsoft.Devices.VibrateController.Default.Start and Microsoft.Devices.VibrateController.Default.Stop are used to switch the actuator on and off.
When switching the vibration on the parameter sets the duration for which it should be on. The duration is between 0 and 5 seconds. With the function TimeSpan.FromMilliseconds(duration), where duration is a number, the parameter can be set easily.

The standard API only supports to switch on and off the vibration. We experimenting with the code you can explore how to have vibrations of different intensity. To do this you have to switch on and off the vibration (e.g. 100 ms on then 50 ms off) - basically doing pulse-width modulation.

There is more information on the vibration controller on the Microsoft site.

See below the c# example for controling the vibration a windows phone 7.
You can also download the vibration project directory in a single ZIP-file.

using System;
using System.Windows;
using Microsoft.Phone.Controls;

// example of how switch on the vibration motor for a given time
// another call to switch it off
// Albrecht Schmidt, University of Stuttgart

// see:
// http://msdn.microsoft.com/en-us/library/microsoft.devices.vibratecontroller.default(v=VS.92).aspx


namespace Vibration
{
 public partial class MainPage : PhoneApplicationPage
 {
     // Constructor
     public MainPage()
     {
         InitializeComponent();
     }

     private void button1_Click(object sender, RoutedEventArgs e)
     {
         double duration;
         duration = Convert.ToDouble(textBox1.Text);
         if (duration > 5000)
         {
             duration = 5000;
         }
         // starts the vibrations (valid duration are between 0 and 5 seconds)
         Microsoft.Devices.VibrateController.Default.Start(TimeSpan.FromMilliseconds(duration));
     }

     private void button2_Click(object sender, RoutedEventArgs e)
     {
         Microsoft.Devices.VibrateController.Default.Stop();
     }
 }
}

WP7 Tutorial - part 1: Phone calls and SMS

2011-03-18T21:13:00.007+01:00

Over the next weeks I like to share some examples I have created while learning to program the Windows Phone 7 platform. The tutorial is mainly explaining some of the APIs and components I found interesting (in particular related to context-awareness and human computer interaction).

If you are really new to programming on this platform the App-Hub is probably a good place to start. There are also plenty of helpful examples on the Microsoft web page. As prerequisite for this tutorial I assume that you have successfully installed Visual Studio (Express), the windows phone development tools, and that you have managed to get your first "Hello World" written, compiled, and deployed. There is also plenty of material on youtube that helps to get started. The Programming Windows Phone 7 book by Charles Petzold (free available as PDF) is also a good starting point.

Several examples that follow in the tutorial (e.g. vibration, accelerometer) having a real phone to test the programs is highly recommended.

The Windows Phone 7 is a phone and hence I start with a program that makes use of the basic phone functionality: making a phone call and sending an SMS.

This example also highlights the approach taken in several of the APIs. Your program (a third party application) is restricted to transfers control to the phones basic functions when making a call or sending an SMS. Basically the API call opens the phone/SMS applications with the parameters you hand over. From a security point of view this is nice as an application can not send SMS or do phone calls without the user recognizing (and agreeing to) it. From a programming perspective this has disadvantages as automation of functionality (e.g. always sent an SMS when I am coming closer than 10 km to home) is not possible.

The example demonstrates how to access interactively the phone book (using the PhoneNumberChooserTask), how to initiate a phone call task (using Microsoft.Phone.Tasks.PhoneCallTask), and how to initiate a SMS task (using Microsoft.Phone.Tasks.SmsComposeTask).
The PhoneNumberChooserTask is an example of a callback - used a lot when programming for WP7. The basic concept is to register a function that is called when an event happens. In this case the event is that the number is chosen and then the function myPhoneNumberChooser_Completed is called. The nice thing with Visual Studio is that you do not have to type this (or remember the syntax) - just use the TAB-key after you typed +=

See below the c# example for sending SMS and making a phone call. You can also download the Phone/SMS project directory in a single ZIP-file.

using System;
using System.Collections.Generic;
using System.Windows;
using System.Windows.Controls;
using Microsoft.Phone.Controls;
using Microsoft.Phone.Tasks;

// example how to invoke a phone call and an SMS from a program
// the basic approach is that you can set the phone number (and
// for SMS the message body) and then you call the phone/sms
// application
// Albrecht Schmidt, University of Stuttgart
// more on how to invoke phone tasks (photo, email, ...)
// http://msdn.microsoft.com/en-us/library/ff769543(v=vs.92).aspx

namespace Phone_Call_and_SMS
{
public partial class MainPage : PhoneApplicationPage
{
  // Constructor
  public MainPage()
  {
      InitializeComponent();
  }

  // choose a phone number
  private void button1_Click(object sender, RoutedEventArgs e)
  {
      PhoneNumberChooserTask myPhoneNumberChooser;
      myPhoneNumberChooser = new PhoneNumberChooserTask();
      myPhoneNumberChooser.Completed += new EventHandler&lt;PhoneNumberResult&gt;(myPhoneNumberChooser_Completed);
      myPhoneNumberChooser.Show();
  }

  // call back after the phone number is choosen
  void myPhoneNumberChooser_Completed(object sender, PhoneNumberResult e)
  {
      if (e.TaskResult == TaskResult.OK)
      {
          textBox1.Text = e.PhoneNumber.ToString();
      }
  }

  // making a phone call (opening the sms application with presets)
  private void button2_Click(object sender, RoutedEventArgs e)
  {
      Microsoft.Phone.Tasks.PhoneCallTask phonecall = new Microsoft.Phone.Tasks.PhoneCallTask();
      phonecall.PhoneNumber = textBox1.Text; // set phone number
      phonecall.Show();
  }

  // sending an sms (opening the sms application with presets)
  private void button3_Click(object sender, RoutedEventArgs e)
  {
      Microsoft.Phone.Tasks.SmsComposeTask sms = new Microsoft.Phone.Tasks.SmsComposeTask();
      sms.To = textBox1.Text; // set phone number
      sms.Body = textBox2.Text; // set body
      sms.Show();
  }
}
}

Source code formating in blogspot

2011-03-18T12:57:00.009+01:00

I am putting together a Windows Phone 7 tutorial ... and then I came across the issue of how to include code in the blog posts. Here is the solution.

First get the code nicely formatted - the following tools may help
http://jsbeautifier.org/
http://www.prettyprinter.de/

Then include the prettyprint function into your Blogspot template. The following Link show how:
http://lukabloga.blogspot.com/2008/10/to-test-new-highlighting.html

You have alter the template you can use the following command to include code in the HTML code with the pre-tag and the class=prettyprint.

Alternatively you can convert source code into HTML:
http://www.tohtml.com:
http://www.neathighlighter.com/

more links at
http://techbrij.com/3/code-syntax-highlighting-for-a-free-hosted-blogger-blog

Visualization - Tokyo Underground Map

2011-03-09T16:07:00.002+01:00

I spotted this visualization of an underground line in Tokyo and thought that I can use this as an interesting introductory example in information visualization. The very simple question is: what does this visualization communicate to the user? The bi-lingual labeled picture is quite rich in the way it communicates the information. Take as one example the visualization of "Where am I?" and "Where is the train going to?". This is visualized in multiple and redundant ways: (1) the station where the user is highlighted with a red background color, (2) the visualization of the underground line has a break at the current station, (3) the stations the train has already passed are lighter colored (pink) as the stations to come, and (4) there is an arrow showing the direction.

Overall I think this is a good example of how to efficiently communicate such information. But be careful if you travel … there are also much more confusing signs around here.

How will computing change the world? Our view in Computing Now.

2011-03-08T17:07:00.003+01:00

Together with Marc Langheinrich and Kristian Kersting we wrote an article on how computing is going to change our world [1] and featured in Computing Now. We discuss how upcoming technologies will change the our perception. Besides others we make the bold statement "By the middle of this century, the boundaries between direct and remote perception will become blurred".

We discuss how our perception is extended and augmented by technical means and how this will eventually lead to a new augmented sense of ubiquitous perception. We expect this will radically change the way we live and hence ethical considerations are central. We make the argument that ethics become a major design factor. We are looking forward to feedback on this vision - even if you disagree.

[1] Albrecht Schmidt, Marc Langheinrich, Kritian Kersting, "Perception beyond the Here and Now," Computer, vol. 44, no. 2, pp. 86-88, Feb. 2011, doi:10.1109/MC.2011.54 (PDF)

Lukowicz Consumer Confidence Index

2011-03-08T03:26:00.003+01:00

Paul Lukowicz taked in the symposium on Interaction with Smart Artifacts in Tokyo about "Large scale, context aware socio-technical systems". As a number of other talks in our symposium he also pointed out that value of the massive amounts of data that become available by mobile sensing and crowd interaction.

Paul gave one example how such sensing data could be used. I think this is really interesting that is why I share here the Paul "Lukowicz Consumer Confidence Index" (CCIL) which is based on detecting the whereabouts of people. It assumes that we can estimate the number of people shopping at various places based on the use of their mobile devices.

Interesting touch technology: PhotoelasticTouch

2011-03-07T07:05:00.003+01:00

Hideki Koike talked at the Symposium on Interaction with Smart Artifacts in Tokyo. One of the examples he showed is the PhotoelasticTouch technology. The idea is to use a transparent gel and the effect of photoelasticity to detect interaction on a surface, see [1] for details. [2] give a brief overview (the picture below is taken from this publication).
There is a nice youtube videp the shows the concept very well: http://www.youtube.com/watch?v=RJaEg0xB98E

[1] Toshiki Sato, Haruko Mamiya, Hideki Koike, and Kentaro Fukuchi. 2009. PhotoelasticTouch: transparent rubbery tangible interface using an LCD and photoelasticity. In Proceedings of the 22nd annual ACM symposium on User interface software and technology (UIST '09). ACM, New York, NY, USA, 43-50. DOI=10.1145/1622176.1622185 http://doi.acm.org/10.1145/1622176.1622185

[2] One page abstract: http://www.vogue.is.uec.ac.jp/PET/photoelasticTouch1Page.pdf

Lab Tour on arrival in Tokyo

2011-03-05T23:07:00.001+01:00

After a long day/night of travelling I arrived in Tokyo. Arriving at the hotel we were met by our Japanese Colleague Yoshito Tobe and guided us to a Lab tour in the afternoon. We went by train to the Morikawa Labs in the Komaba Research Campus at the University of Tokyo.

At the lab students from different group showed us their work and discussed their ideas with us. To mention only a few things I got to try out a tutoring system for Japanese calligraphy, we saw prototypes for phone based urban sensing, and saw an implementation of a system that communicates between two devices using accelerometers and vibration motors [1].

After the tour we went up a tower building to look at the sunset above Tokyo and we even had a view on Mount Fuji. And as the observation floor is a touristy place there are all sorts of interesting things - and I operated a nice machine to get a puri-CUBE.

Some more photos are available at: http://tinyurl.com/LabTok11 (facebook account required) or public on flickr.

[1] vib-connect: A Device Selecting Interface Using Vibration by Hiroshi Nakahara et al. Demo at IOT 2010.

Keyboard I/O hack

2011-03-03T22:43:00.009+01:00

During my research visit at Microsoft Research in Cambridge with Steve Hodges group I spend a lot of time building things and programming (Windows Phone 7, C#, microcontroller...) - which I really enjoyed!

In this post I want to share a mini project, I did together with Nic Villar between two coffees … it reminded me somehow of the Friday afternoon projects (=2h projects you do because they are fun and sometimes you learn something) while I did my PhD.

Perhaps some context first: working on Microsoft .NET Gadegteer we discussed many ideas of how to ease code/system creation for physical and embedded computing. Looking into it I came across a number of interesting approaches for software development that runs in the browser. Examples include: http://ideone.com/ (a online compiler for a large number of languages), http://wonderfl.net/ is a online development environment for Flash that allows to build on other people's code by forking their project, and http://mbed.org/ is a online tool for rapid prototyping of a microcontroller system.

Now to the mini project: a keyboard hack that supports input as well as output. You find many example of keyboard hacks on the web - basically you take out the PCB of the keyboard an replace the keys with something more interesting… here is ours that supports input (which is common) as well as output - from the computer (or web browser) to the environment. Using AQW210EH allows more flexibility in what you drive and what you create the inputs with. The attractive thing in using a keyboard is that it will work without a driver - you just plug it in and it works.

To control it I wrote a java applet and some JavaScript code. Basically the java-applet is controlling the Scroll-Lock LED and from the website this is called via JavaScript (try here if it works on your computer - if it works your CAPS, NUM, and SCROLL LED will blink once the web page is open). I tried it on some Windows machines and it worked well - the java applet seems not to work on a Mac (so you probably have to find another way to set you Scroll Lock LED programmatically). You can extend the output to CAPSLOCK and NUMLOCK - but these are sometimes used - in contrast to the SCROLL LOCK. Input is simple - by connecting one of the rows to one of the colloms you generate a letter comes in as if you would type it. The nice thing with USB is that you can have multiple keyboards connected at the same time (however the LEDs are synchronized between them).

In C# you can use the SendKeys.Send("{SCROLLLOCK}") command (for details see: http://msdn.microsoft.com/en-us/library/system.windows.forms.sendkeys.send.aspx,
http://support.microsoft.com/Default.aspx?id=177674) The SendKeys.Send may only give an pulse to the LED and hence you may need to call a system function.

Using a JavaApplet it looks like this.

import java.applet.*;
import java.awt.*;
import java.awt.Toolkit;
import java.awt.event.KeyEvent;
public class keyLed extends Applet {
   Font f = new Font("TimesRoman",Font.BOLD,20);
   String strMessage;
   Toolkit tk = Toolkit.getDefaultToolkit();
   public void init() {
       setBackground(Color.white);
       setStrMessage("Applet loaded.");
   }
   public void paint(Graphics g) {
       g.setFont(f);
       g.drawString(this.strMessage, 100 , 25);
   }
   public void ScrollLedOn() {
       tk.setLockingKeyState(KeyEvent.VK_SCROLL_LOCK,true);
       setStrMessage("Scroll on");
   }
   public void ScrollLedOff() {
       tk.setLockingKeyState(KeyEvent.VK_SCROLL_LOCK,false);
       setStrMessage("Scroll off");
   }
}

And in HTML/JavaScript you call it:

<APPLET CODEBASE="." CODE="keyLed.class" WIDTH=300 HEIGHT=50 NAME="keyLedApplet">
</APPLET>
<script language="JavaScript" type="text/javascript"> 
function LEDOn()
{
document.keyLedApplet.ScrollLedOn();
}
 
function LEDOff()
{
document.keyLedApplet.ScrollLedOff();
}
 </script>

Workshop: Context-aware QoS

2011-03-03T11:49:00.000+01:00

The deadline for the workshop on "International Workshop on Context-aware QoS Provisioning and Management for Emerging Networks, Applications and Services" is extended to March 18. Please have a look at the call for papers: http://www.contextqos.org

The workshop is co-located with 20th International Conference on Computer Communications and Networks (ICCCN) 2011 from July 31 - August 4, 2011 at the Sheraton Hotel and Resort, Maui, Hawaii.

Storytelling for advertising

2011-02-27T01:19:00.005+01:00

I just watched one of my favorite adverts on UK TV (http://meerkat.comparethemarket.com/home).

It is amazing that they have implemented a complete website for comparing meerkats (I especially like the meerkat in Berlin wearing flippers). Not sure however how well it works as I have never visited the website it is advertising for… (but I do remember the name, hence it worked somehow).

There are a lot of new ways for story telling on the web. I like the article by Jim Miller that was 2005 in the interaction magazine [1] that discusses a case study of web based storytelling and how it may evolve. It talks about how to create and unfold a story using multiple web sites. This is pre-facebook and I think the possibilities now are even greater. I wonder if based on this one could create a class project on web user interfaces.

[1] Jim Miller. 2005. Storytelling evolves on the web: case study: EXOCOG and the future of storytelling. interactions 12, 1 (January 2005), 30-47. DOI=10.1145/1041280.1041281 http://doi.acm.org/10.1145/1041280.1041281

Tangible User Interface - Understandable, Ergonomic, Aesthetic

2011-02-05T00:11:00.004+01:00

As I am always on the lookout for user interfaces it was delightful to see the bridge and the engine room of the Queen Mary with truly tangible man-machine-interfaces. Large wheels and all sorts of knobs, cranks, and levers are there to operate the many functions. At a first glance they look nice, intuitive, and easy to use - but are they? Looking closer it becomes clear that many user interface elements were built primarily to enable mechanical control and not because of ergonomic considerations. The size of physical controls (e.g. the length of a lever) is often due to the fact that certain forces must be created to control the machine. The size and visibility of the mechanical properties are helpful for the user to understand the operation and to some extent foresee the impact of the operation. At the same time many of these user interface elements make the operation very visible to co-workers - you do not have to tell that you change the speed as everyone around you can clearly see it from your actions. It seems that ergonomic properties are of lesser importance in many of these controls, e.g. some controls require strong forces or large physical movements. All in all I would conclude that these tangible controls (that are designed due to physical constraints) are helping with the understandability of a user interface but are not necessarily a good model for creating ergonomic controls.

In some recent examples of tangible user interfaces I feel that people took the worst from both worlds - they argue for the physical controls - and all they get are less ergonomic UIs (e.g. needs more forces, movements, etc.) which are less understandable as the physical constraints do not map to the constraints of the digital system… Hence I think it is important to keep this in mind: the physicality of the controls should be used to make interaction understandable and the design should not compromise ergonomics (as we do not live in the mechanic era anymore).

One further thing that can be learned for these mechanical interfaces is the beauty of the design and implementation. Some are in shinny and polished metal, others are nicely decorated, and on others it is just pleasant to touch the wood. Beauty and esthetical qualities play a major role - and we know this for screen based UIs from Noam Tractinsky's work [1,2] as well as from the market success of devices like the iPhone.

[1] Noam Tractinsky. 1997. Aesthetics and apparent usability: empirically assessing cultural and methodological issues. In Proceedings of the SIGCHI conference on Human factors in computing systems (CHI '97), Steven Pemberton (Ed.). ACM, New York, NY, USA, 115-122. DOI=10.1145/258549.258626 http://doi.acm.org/10.1145/258549.258626

[2] Tractinsky, N., Shoval-Katz A. and Ikar, D. (2000) What is Beautiful is Usable. Interacting with Computers, 13(2): 127-145.

IEEE Computer Editorial Board Meeting in Long Beach

2011-02-02T19:48:00.001+01:00

This week I went for my first IEEE Computer editorial board meeting to Long Beach. The meeting was held in the meeting facilities on the Queen Mary. I found this location very interesting and inspiring and at the same time a vivid reminder of changing times. The Queen Mary was an ocean liner - not a cruise ship - and its purpose was to get people quickly and comfortable between the US and Europe. With this regard I found the location very fitting when discussing the future directions of the IEEE Computer Magazine, the IEEE Computer Society's flagship publication, a magazine still printed on paper and distributed via traditional mail to most members. We all know that it will become digital - and many read it already in digital form. But the exciting and challenging questions are how a digital magazine will look like and it is hard to tell the timeframe in which the change is going to happen.

The discussions during the meeting and the expertise of the people on the board was for me reassuring that these massive transitions that are ahead will be mastered and that the publication will get even more exciting over the next years. I would expect within the next five years the way research results are presented, delivered, and discussed will change to a great extent. For me the assent the IEEE Computer Society is its base of experts and that it facilitates exchanges between these experts and the quality control it can provide in its top publications. Furthermore I think there is a great value in the editing of articles for the magazine. Articles become much more enjoyable to read after they are edited - this is especially valuable for authors who are non-native speakers like me, but I think it is beneficial to all articles!

Seeing companies moving into the space, such as Facebook (e.g. facilitating group communication and dissemination), Google and Microsoft (e.g. offering search in the academic space and analyzing relationships and citations) I would expect there is no reason to compete in these areas, but it should be figured out how to cooperate.

We had a long discussion focusing on the editorial direction for the next year and the number of ideas that can up for what type of content we should attract for the magazine and what topics to cover was amazing and motivating - and it showed me again how multi-faceted and exciting computer science is.

For 2011 the following topics will be included: cloud computing, Entrepreneurship, mobile computing, software, hardware, smart planet, Computational Archaeology and computing & arts (could be interesting for the TEI community!). If you have ideas what topic IEEE Computer should cover please tell me (e.g. by commenting or emailing me) or submit an article to the magazine! I would also hope to see a little more on user interfaces and interaction technologies.

Meeting the Maker of littleBits due to Canceled Flights

2011-01-26T21:29:00.004+01:00

I got up this morning before 4am… that is really early. Arriving at the airport I found out that the 5:40 flight was canceled. I should have checked the flight status before getting out of bed (or TAP/Lufthansa could have sent me a SMS). It was not a complete surprise as yesterday someone mentioned that due to the strong rain some flights did not arrive in Funchal.

I was rebooked on a later flight and met by chance Ayah Bdeir, who was at TEI and on the same flight to connect vi FRA. At the demo session I saw her product: littleBits (http://littlebits.cc) but did not talk to her.

littleBits is a set of different hardware modules that can be connected by small magnet. On the website it is described as

"littleBits is an opensource library of discrete electronic components pre-assembled in tiny circuit boards. Just as Legos allow you to create complex structures with very little engineering knowledge, littleBits are simple, intuitive, space-sensitive blocks that make prototyping with sophisticated electronics a matter of snapping small magnets together."

The approach is very interesting and shows one trend that is currently ongoing in the TEI community: enabling a wider set of people to create functional interactive products by lowering the barrier and effort for prototyping systems that consist of physical components, electronics, and software. The target audience ranges from designers to artists, but also includes the education domain. A point that makes littleBits interesting is that they will be open source. They are not yet available, but one can subscribe for further information on the website: http://littlebits.cc there are also some cool videos...

At TEI it was great to see so many toolkits and platforms appearing. When we did our first open source hardware/software platform in 2002 with Smart-Its we followed a DIY-approach for building the hardware (having a step by step photo tutorial to building the modules) and having a lot of options for programming it. Additionally each module had wireless included. At the time we consider this a good idea - we thought it will empower the developers. Looking back this limited the number of people that would be able use it - but allowed the one's how made the effort to do very forward looking prototypes. We summarized some of the experience in [1]. The current research and products seen at TEI focus much stronger on making it easy for a large set of people to use it - it will be exciting to see if and how this speeds up the creation of new products in the next years.

[1] Gellersen, H., Kortuem, G., Schmidt, A., and Beigl, M. 2004. Physical Prototyping with Smart-Its. IEEE Pervasive Computing 3, 3 (Jul. 2004), 74-82. DOI= http://dx.doi.org/10.1109/MPRV.2004.1321032

TEI2011 Keynote: Bounce Back by Gilian Crampton Smith

2011-01-25T16:14:00.005+01:00

Gilian Crampton Smith is one of the people that massively influenced the field of interaction design from the very beginning. She is currently working in Venice: www.interaction-venice.com. Nearly 20 years back in London Durrell Bishop was one of her students. She pointed out that the central insight of the early work was "We know how to control objects by physical means". This insight has since become a basic motivation for tangible interfaces and embodied interaction. Closely linked to this is the complexity of interaction. To me one great point with designing tangible user interfaces is that you are forced to come up with a simple solution. Overall the argument was that embodied interaction works because it draws on knowledge we have. An example is that two physical things cannot be in exactly the same place and another one is that things stay where they are if there is no force moving them. There are clear limitations to the interaction with physical objects that give indications how to use it; she referenced a paper on an exploration of physical manipulation [1].

As one example Gilian showed the marble answering machine concept video from 1992. To me this video is a great example that shows the power of a concept video. Using a simple animation in this concept video enables viewers to understand the idea and interactions very quickly. I like the simplicity and clarity of this video - it would be very useful for teaching. Hopefully it will made publicly available; see below the bad-quality copy I took with my mobile. A good quality version ist at http://vimeo.com/19930744

One interesting point was a reflection on the shortcomings of the traditional view of interaction being based on input and output. Gilian argued that output should be separated in feedback and results. Input is categorized in according the interaction required, such as simple (e.g. text, minimal movement), medium (e.g. GUI), maximum (e.g. musical instrument, movement of the whole body). The feedback is related to the sense human have, and includes modalities such as visual, auditory, tangible, kinesthetic, proprioception. Results are related to feedback but are oriented on the outcome. Results may be visual (e.g. symbolic, words, icon, films); auditory (e.g. sounds, words or music) or physical such as touch (e.g. massage machine) or movement.

Gilian went on to the question: "Why aren't tangible, embedded and embodied interfaces out there in our everyday world?" For me the most important answer she gave is that it is hard to do them. It is very difficult to created sensor based interaction in a way that users understand it and that it feels natural to use. Counter examples where people got it wrong are those many lights that go on when you are already half way up the stairs or the automatic doors where people wave their hands in order to open them. As much of the interaction technology in such systems is not visible people have a hard time figuring out how things work. Based on their (incomplete) experience base on a few interactions they will create their mental modal … which is often wrong and potentially leads to frustration. A further point she made is that creating physical interactive objects and things is difficult and involves a lot of different skills. In combination with the function it is difficult to get the aesthetics right. One very good example is how hard it is to the aesthetics of feel is right, as it includes a well balanced design taking into account touch, weight, balance, movement, and sound. Overall this requires a great passion for detail to get the quality of engineering right; Gilians examples for this was the AUDI advert where an Engineer opens and closes the car door and listens carefully. I think I have seen in 8 years back in the UK as part of the ad-campain "Vorsprung durch Technik". If anyone has a link to a copy please post it.

When I was in 2002 (or around this time) visiting her at Ivrea near Torino I was very much impressed by the creativity and inventiveness. In her talk she explained one approach to teaching I really like. The students get a device (e.g. an alarm clock, answering machine) without manual with the task to figure out how it works. While exploring the functionality they film themselves. Based on this they reflect on the design and then go on to do a completely new design for a device with a similar functionality (with the constraint that it cannot have buttons).

I like a reference to something Bruce Steering wrote. In short it basically says research is like crime; you need Means, Motivation and Opportunity.

Some further example she showed:

A communication device based candle: ceramic liaison. It is a bi-directional 1-bit communication device, that is esthetically pleasing. The state of the candle (real wax with fire) on either side is reflected by lighting up some ceramics on the other side.
A full body experience interaction device for controlling games: Collawobble. Using two bouncing balls a packman game can be controlled. One user controls with the bouncing X and the other Y.

update:
Ellen Do posted 2 clips that were shown in the keynote

http://vimeo.com/11802060
L'età del fuoco - The age of fire
www.youtube.com/watch?v=LE9t98Gox60
Jaques Tati - Mon Oncle (1958) - Kitchen Scene - The struggle of a simple man with a "highly modern" kitchen and a bouncing jug

and suggested a further one: http://www.youtube.com/watch?v=oS6pwQqSY70

the marble answering machine video on http://vimeo.com/19930744

[1] Andrew Manches, Claire O'Malley, and Steve Benford. 2009. Physical manipulation: evaluating the potential for tangible designs. In Proceedings of the 3rd International Conference on Tangible and Embedded Interaction (TEI '09). ACM, New York, NY, USA, 77-84. DOI=10.1145/1517664.1517688 http://doi.acm.org/10.1145/1517664.1517688

TEI Studio, making devices with Microsoft Gadgeteer

2011-01-25T15:45:00.005+01:00

Nic Villar and James Scott from Microsoft Research in Cambridge, UK, run a Studio on building interactive devices at TEI 2011 in Madeira. Studio's are mixture of tutorials and hands-on workshops and there were very interesting ones at this year's conference - it was very hard to choose.

Microsoft Gadgeteer is a modular platform for creating devices. It includes a main board, displays, a camera, sensors, interaction elements, motors and servos, memory board, and USB-connectors as well as power supply options. It is programmable in C# using Visual Studio. The development involves the connecting up of the hardware and the writing of the software. A "hello world" example is a simple digital camera. You connect up the main board with the camera, a display, and a button. Writing less than 20 lines of code you implement the functionality of the camera. It follows an asynchronous approach. When the buttons is pressed the camera is instructed to take a picture. An event handle is registered that is called when the camera is ready capturing the image. And in the code in the handler you take the image and show it on the display. 10 years back we worked on a European project Smart-Its [1] and started with basic electronic building blocks and the idea of open source hardware where people could create their devices. Looking now at a platform like Gadgeteer one can see that there has been great progress in Ubicomp research over that time. It is great to see how Nic Villar has been pursuing this idea from the time he did his BSc thesis with Smart-Its in our lab in Lancaster, with Pin-and-Play/Voodoo I/O [2] in his PhD work, to the platform he is currently working on and the vision beyond [3].

It was amazing how quickly all groups got this examples working. For me, with programming experience - but no real experience in C# and with little practice in programming over the last few years - it was surprising how quickly I got into programming again. To me this was mainly due to the integration with visual studio - especially the suggestions made by the IDE were very helpful. James said the design rational for the Visual Studio integration was "you should never see just a blinking cursor" - and I think this was well executed.

All groups made over the day a design. I liked the "robot-face" that showed minimal emotions… basically if you come to close you can see it getting unhappy. My mini-project was a camera that use an ultrasonic range sensor and a larger display. It takes photos when someone comes close and shows the last 8 photos on the display - overwriting the first one when it comes to the end of the screen. Interested in how little code is required to do this? Check out the source code I wrote.

Two more studios I would have loved to attend: Amanda run a Studio on creating novel (and potentially bizarre) game controllers and Daniela offered a Studio to explore what happens if bookbinding meets electronics.

[1] Holmquist, L. E., Gellersen, H., Kortuem, G., Schmidt, A., Strohbach, M., Antifakos, S., Michahelles, F., Schiele, B., Beigl, M., and Mazé, R. 2004. Building Intelligent Environments with Smart-Its. IEEE Comput. Graph. Appl. 24, 1 (Jan. 2004), 56-64. DOI= http://dx.doi.org/10.1109/MCG.2004.1255810

[2] Van Laerhoven, K., Villar, N., Schmidt, A., Gellersen, H., Håkansson, M., Holmquist, L. E. 2003. In-Home Networking - Pin&Play: The Surface as Network Medium, IEEE Communications Magazine, vol. 41, no. 4, April 2003.

[3] Steve Hodges and Nicolas Villar, The Hardware Is Not a Given, in IEEE Computer, IEEE, August 2010

TEI growing up, Dinner with Hiroshi Ishii and Don Norman

2011-01-24T00:59:00.001+01:00

This is now the 5th TEI conference and it has about 300 people attending. It is still a very young community, with many students attending, but also the "big names" are at the conference. When Brygg and I started with a small conference in 2007 (in Baton Rouge) and 2008 (in Bonn) we considered there is a need for tangible and embedded interaction work to find a venue but we did not expect that the community was growing so rapidly. Looking at the presentations and contributions show at this year's conference it is clear that the conference is growing up - without losing its exciting mix of contributions.

It is great to talk to Don Norman and Hiroshi Ishii over dinner and seeing them engaging with this young community. They both have made major contributions to this community and have inspired my personal research some 10 years back. If you have not done so I recommend to read some of their early contributions, such as Hiroshi's CHI1997 paper [1,2] and Don's invisible computing book [3,4]. Both have published and publish many interesting articles and books which are central to the HCI literature, check out their web pages: Hiroshi Ishii and Don Norman.

[1] Ishii, H. and Ullmer, B., "Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms," Proceedings of Conference on Human Factors in Computing Systems (CHI '97), ACM, Atlanta, March 1997, pp. 234-241.

[2] Fitzmaurice, G., Ishii, H., Buxton, W., "Bricks: Laying the Foundations for Graspable User Interfaces," Proceedings of Conference on Human Factors in Computing Systems (CHI '95), ACM, Denver, May 1995, pp. 442-449.

[3] Donald Norman, The Invisible Computer. 1998, Cambridge MA, MIT Press

[4] Donald Norman, The design of Everyday Objects, 2002 Basic Books (Perseus)

pdnet meeting at the University of Minho in Portugal

2011-01-21T20:10:00.003+01:00

Over the last day we got together in Portugal to discuss the progress in the EU-FET-OPEN project pdnet. We are now 9 month into the project and it is great to see the results appearing. Stay tune for upcoming papers. Besides other research we have done interesting observational work and have gained understanding what matters for people when setting up public displays and this informs our system design.

In summer we have the opportunity to try our system by deploying two applications in Oulu, Finland. With two applications proposals we were selected as finalist for UbiChallenge. In June we will go with our students to Oulu and implement our system and study the usage in a public deployment.

Food was amazing…

What alarm clock are you using?

2011-01-21T16:49:00.004+01:00

The answer is probably "my phone" - it seems that for many people the phone has become their primary alarm clock. We have discussed this before in a blog post…

Some years back I took part in a design competition at the appliance design conference and suggested an alarm clock that links you to your friends [1]. One of the ideas was to have dynamic wake up times based on when you friends got up. The paper was accepted in March 2005 - this was before twitter was founded and before facebook was open for general registration. At this time we envisioned this as a stand-alone appliance as micro-blogging was not yet around.

Time has move on and many appliance ideas have since become apps on the phone. In the course of his research Ali is working on ideas for increasing the connectedness between people. One of the case studies is now an alarm clock - called weSleep - that has the basic alarm clock function and has additionally means to log sleep hours and perceived sleep quality. It also allows to post information related to the going to sleep or being woken up to social networking software such as facebook.

Interesting in trying it out? Check out the web page of weSleep and if you are interested in taking part in a study please contact Ali (not sure if he still is interested in more volunteers).

[1] Schmidt, A. 2006. Network alarm clock (The 3AD International Design Competition). Personal Ubiquitous Comput. 10, 2-3 (Jan. 2006), 191-192. DOI=http://dx.doi.org/10.1007/s00779-005-0022-y

Moved to the University of Stuttgart

2011-01-18T15:41:00.001+01:00

No posts for some time... the reason: I have changed jobs and this was eating up more time than expected.

Since December, 17th 2010 I am with the University of Stuttgart in the south of Germany - much closer to home :-) I have taken a position as professor in Human Computer Interaction in the Institute for Visualization and Interactive Systems. The position was created in the context of the SimTech cluster of excellence.

My new office address (in a brand new building) is:
Albrecht Schmidt
Pfaffenwaldring 5a
70569 Stuttgart
Germany

Email and web server are still in the making... but my regular acm email-address will be still valid.

PARC - touching computing history

2010-11-10T22:02:00.000+01:00

At PARC I had the chance to talk to people about some of our current projects. Les Nelson has done interesting work on public displays [1]. This work is highly relevant to ideas we pursue in the pdnet project and it was great to get a first person view from the researchers involved.

Being at PARC history of computing is all around you! Seeing the original Ethernet cable, tapes from Alan Kay or Lucy Suchman, the Alto computer, one of the original laser printer, and different Ubicomp artifacts from Mark Weiser's group really makes you feel that this is a special place for anyone interested in personal computing and ubicomp.

[1] Elizabeth F. Churchill, Les Nelson, and Gary Hsieh. 2006. Cafe life in the digital age: augmenting information flow in a cafe;-work-entertainment space. In CHI '06 extended abstracts on Human factors in computing systems (CHI '06). ACM, New York, NY, USA, 123-128. DOI=10.1145/1125451.1125481 http://doi.acm.org/10.1145/1125451.1125481

Random Links for Scientific Search in CS

2010-10-18T14:02:00.004+01:00

Scientific search sides:
http://scholar.google.de/
http://www.confsearch.org
http://academic.research.microsoft.com/
http://arnetminer.org/

Digital libraries:
http://acm.org/dl
http://ieeexplore.ieee.org
http://www.springerlink.com/

Listings of publications, co-authors, and relationships:
http://www.informatik.uni-trier.de/~ley/db/index.html
http://dblp.mpi-inf.mpg.de/dblp/index.php

Do tangible user interface make sense? Yes they are a great design tool.

2010-10-14T07:39:00.000+01:00

The question "Do tangible user interface make sense?" is a question that probably everyone who seriously works in this field has asked themselves once in a while.

Seeing the iPhone and iPod app of the people doing the reactable made me think about this question again! What is really - in the use case of the reactbale the value of the physical over the touch screen? Or is it just sentimental and old school to believe in the physical? Not sure … needs probably some more thinking and research ;-)

One other points which this example underlines is that tangible interaction is a great design tool (still in the process of writing a paper about this - but here the basic idea for discussion). And I strongly believe that this is a great value for user interface design in general. I suggest the following approach:

Analyze your task
Find data elements that can be made tangible
Find operators/manipulators on the data elements that can be made tangible
Create a tangible user interface to realize all the interaction required
Port it to a touch screen or conventional user interface

The steps 1-4 will ensure simplicity and in step 5 you may lose some of the "ah" and "wow" but it is very likely that you have created a usable and simple interface!

Will social science change completely?

2010-10-13T22:02:00.002+01:00

Seeing the recent post on blog.okcupid.com (Gay Sex vs. Straight Sex) made me think if we are approaching a point where our understanding of society will massively change (hopefully for the good) and where we will get much greater insights in who we are. Is this similar to the era of the invention of the microscope? Things become visible and one does not need to guess anymore?

The amount of data collected on websites is huge - and in many cases the data is probably of very high quality as it matter to people who contributed it (probably higher than what you get with a random questionnaire) . I think this is exciting and looking at some of our project proposals going beyond explicit data collection to implicit data collection may even make this approach stronger (adding another x10 on the new microscopes).

Competitions in computer science for schools

2010-10-08T18:28:00.001+01:00

Spending 3 intensive days at the University of Freiburg as member of the jury at the finals of the German computer science competition (Bundeswettbewerb Informatik) I learned once more how vast our field is … especially at the theoretical end. The tasks on the first day were related to stream processing algorithms and on the second day to games on graphs. But don't be fooled theoreticians have a very different understanding what a good game is ;-)

The 28 people (pupils and high school students - who have not yet started studying) at the finals are the "best" from over 1000 participants and had successfully passed two rounds before. There level of CS knowledge was massively impressive. Many of them would have passed the BSc exams - in Math and theoretical computer science - without much further preparation! The event showed that computer science has a great potential to attract young people.

Here are links to German competions:

Informatik Biber (the general CS completion for students from class 5-13, last year some 80.000 pupils took part)
Bundeswettbewerb Informatik (the more difficult completion, last year bit more that 1000 pupils took part)

Around the event there were some interesting demos (to impress the prospective students), including Toyota Robina and an autonomous mini-airship.

Two automotive deadlines today!

2010-10-01T11:10:00.002+01:00

Today (Friday, October 1st 2010) there are two deadlines related to automotive computing research:

Early registration deadline of the 2nd International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI 2010)
Submission deadline for the Special Issue of the IEEE Pervasive Computing magazine on "Automotive Pervasive Computing"

For the AutomotiveUI 2010 the program looks really exciting, see the conference web page. John Krumm is giving the keynote and the program includes 26 papers (if I counted right) in the areas of: Attention and Distraction, Speech and Sound, Exploring Modes of Interaction, Supporting the Driver, and Connected Cars.

With regard to the special issue I heard that there is a chance to get a few days extension ;-)

Ubicomp 2010

2010-09-27T15:29:00.005+01:00

Today the 12th international conference on Ubiquitous Computing (ubicomp2010) started in Copenhagen. The conference is very competitive showing a wide range of work in the space of computing beyond the desktop. This year 39 of 202 papers and notes were accepted in the main program. In this part of the program there is a focus of work from North America (which seems to go together with conferences becoming ACM conferences).

The opening keynote was by Morton Kyng on "Making dreams come true - or how to avoid a living nightmare". In his talk he outlined his view on palpable computing which basically described user centered development of pervasive systems.

This years Ubicomp has a large number of demos and it was fun to engage with these and with the people presenting them. Christian Winkler from our group had an invited demo on "Sense-sation: An Extensible Platform for Integration of Phones into the Web" showing a combined web and mobile phone platform that eases the development of applications that run across several phones. For example is it very easy to create an application where you have a map interface and you can mark an area on the map and request that each of the devices currently in this area is going to take a photo and sent it back (given that the devices run the platform and that you have the right to use the camera on these phones). There will be a full paper on this in a few weeks published at the Internet of Things Conference in Japan and you can already check out the web page: http://www.test.sense-sation.de/

As Ubicomp is not held at a hotel (which I like) there is also no conference hotel with a default bar. Hence the organziers name a Ubicomp 2010 bar: Nyhavn 17. I think this is a good idea!

Ubicomp 2010 Workshop: Ubiquitous Computing for Sustainable Energy (UCSE2010)

2010-09-26T23:54:00.001+01:00

Together with Adrian I organized a workshop at Ubicomp2010 in Copenhagen on Ubiquitous Computing for Sustainable Energy. The motivation for this were for me the question (1) if ubicomp can help to make energy provision more sustainable and (2) what are the central areas where ubicomp technologies can help. Over the last years we have seen a lot of example of motivational technologies - which I am not convinced of. For me the example of standby power is symptomatic. There was a lot of discussion how to reduce the standby consumption motivating people to actively do it and providing more awareness about energy consumption. This lead to a number of academically interesting investigations and prototype making people more aware of their consumption (e.g. the power aware cord)- however to me they do not make a real difference yet. A "simple law" (as we have recently seen in Europe, COMMISSION REGULATION (EC) No 1275/2008 following Directive 2005/32/EC) saying that you do not get the CE-certification for your device if it exceeds a certain power in standby did the job - at least in Europe. Within a few month all TVs that I have seen being advertised were below 1W standby consumption.

If you are more interested in the topic please have a look at the workshop web page. There are also the online proceedings available as well as some results of the discussion. During the workshop we got some feedback on facebook, a colleague stated: "if we didn't have ubiquitous computing, our energy situation would be more sustainable ... every time, for instance, a customer upgrades their mobile - iphone 5, anyone, the energy waste is huge". I think that is a really important and valid comment, and I made the following reply "it is more complicated than that, e.g. how does this change if you use public transport instead of your Hummer (=personal lorry) because of your iPhone 5 ;-) or as you do your email on the iPhone and hence do not have a PC at home anymore ... to be more serious one of the questions we posed the questions if sustainability is a CS topic and in what sense (or if this is rather a political questions)". Adrian added a further response: "consumerism clearly has a lot to answer for. If we didn't have conference travel, or didn't submit the papers in the first place? ... :-) I'm sure you know: Elaine M. Huang, Khai N. Truong's CHI 2008 paper: Breaking the Disposable Technology Paradigm..." [1]. We continued this discussion over dinner and I think the ultimate answer is to go towards a live style of reduced consumption - but I expected this would crash our current economic system…

Coming out of the restaurant we saw an impressive firework and it seemed people (including me) liked it and we did not really think about wasting resources and polluting the environment for a short display…

[1] Huang, E. M. and Truong, K. N. 2008. Breaking the disposable technology paradigm: opportunities for sustainable interaction design for mobile phones. In Proceeding of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy, April 05 - 10, 2008). CHI '08. ACM, New York, NY, 323-332. DOI= http://doi.acm.org/10.1145/1357054.1357110

PD-NET web page and facebook page

2010-09-22T18:06:00.003+01:00

For our European FET open project PD-NET we have a web page at http://pd-net.org/ online. There we describe the project and there is already the first publication (a paper we will present at ACM Multimedia 2010) listed. The page also includes a description of the project and the partners. If you work on public displays or if you are interested in this topic (e.g. as developer, service provider, device manufacturer, content designer, or user) please feel free to contact us. Perhaps there is chance for collaboration.

The project objectives are:

To create enabling technologies for large-scale pervasive display networks through the design, development and evaluation of a robust, scalable, distributed and open platform for interconnecting displays and their sensors.
To establish Europe as the international centre for work on pervasive display networks.
To address key scientific challenges that may inhibit the widespread adoption of pervasive display network technology: Tensions between privacy and personalization, situated displays, business and legislative requirements, User Interaction.

To share resources and ideas with a community of researchers interested in the topic we have setup a facebook page: http://www.facebook.com/pdnet. There is one album where we hope to collect a larger number of photos of public displays (traditional and digital) from around the world to document the variety of public displays and people’s interaction with them. If you have interesting photos of public displays please share them on the page. We also encourage researchers publishing in this research field to share their publications (or links to the publications) with the community (and it is obvious that everyone will cite you ;-)

Interviewing with Rikke Friis Dam and Mads Søgaard

2010-09-20T23:53:00.002+01:00

Rikke Friis Dam and Mads Søgaard are currently working on a re-launch of the website interaction-design.org. The side has over the last years involved in a useful resource for researchers and practitioners in human computer interaction and interaction design. There is a very comprehensive calendar that includes most relevant events in HCI.

With their current work Rikke and Mads pursue a mission to create a new and free resource for teaching and learning interaction design and HCI. In a first step they work with researchers (like myself) around the world that are experts on a certain topic (in my case context-awareness and implicit interaction) to create new teaching materials. This includes a chapter (about 3000 words) that has tutorial character and interviews in which specific topics are discussed in more details.

It was great fun to work with them and I look forward to seeing the new material online.

CFP - new conference series: Augmented Human

2010-09-16T13:33:00.002+01:00

From the cfp
"The second Augmented Human (AH) International Conference will be held
in Tokyo Water Front on March 12th, 13th and 14th 2011
Full information on: http://www.augmented-human.com

The AH international conference focuses on scientific contributions
towards augmenting humans capabilities through technology for
increased well-being and enjoyable human experience. The topics of interest include, but are not limited to: Augmented and Mixed
Reality, Internet of Things, Augmented Sport, Sensors and
Hardware, Wearable Computing, Augmented Health, Augmented
Well-being, Smart artifacts & Smart Textiles, Augmented Tourism
and Games, Ubiquitous Computing, Bionics and Biomechanics
Training/Rehabilitation Technology, Exoskeletons,
Brain Computer Interface, Augmented Context-Awareness,
Augmented Fashion, Safety, Ethics and Legal Aspects,
Security and Privacy Aspects"

Sounds exciting! The majority of researchers in the PC are from France and Japan.

Important Dates:
December 23rd 2010, paper submission deadline
January 22nd 2011, author notification
March 12th/13th/14th 2011, Conference in Tokyo

Comprehensive and modern German book on HCI by Bernhard Preim and Raimund Dachselt

2010-09-15T16:04:00.000+01:00

After having read an electronics preprint some weeks ago Bernhard and Raimund showed me the first printed version of their book "Interaktive Systeme 1: Grundlagen, Graphical User Interfaces, Informationsvisualisierung, Mobile Interaktion" (amazon, springer). The new edition brings a comprehensive overview of human computer interaction with many illustrations and examples. It is perfectly suited for teaching.

Congratulation to finishing it! I am always amazed how people manage to create these books without locking themselves away from the world for a year…

Ed H. Chi visiting our Lab

2010-09-15T00:39:00.000+01:00

We had the great pleasure to have Ed H. Chi from PARC for the afternoon and evening in Essen. Ed gave a keynote at Mensch und Computer 2010 and we took the chance to discuss some of our work with him. From eye-gaze based security, to social TV and vibration notifications he got a broad overview of the work we currently do.

Ed, thanks again for the feedback and the many ideas you shared with us!

On the way at Essen train station we could show a real world example that usability is not restricted to computers and that common sense is not enough to design usable environments. If you are between track 1/2 and 3/4 and you look towards the main electronic train time table you better be not taller than 10 cm - otherwise the traditional signs will block your view (an so far I have not seen such small people inEssen).

Keynote: Steve Benford talking on "Designing Trajectories Through Entertainment Experiences"

2010-09-14T11:29:00.001+01:00

On Tuesday morning Steve Benford presented the entertainment interfaces keynote. He is interested in how to use computer technology to support performances. Steve works a lot with artist group, where the University is involved in implementing, running and studying the experiences. The studies are typically done by means of ethnography. The goal of this research is to uncover the basic mechanisms that make these performances work and potentially transfer the findings to human computer interaction in more general.

I particularly liked the example of "Day of the figurines". Steve showed the video of experiences they created and discussed the observations and findings in detail. He related this work to the notion of trajectories [1], [2]. He made the point that historic trajectory are especially well suited to support spectators.

Some years back I worked with Steve in the Equator and we even have a jointed publication [3] :-) When looking for these references I came across another interesting paper - related to thrill and excitement, which he discussed in the final part of the talk [4].

PS: we had a great party on Monday night but the attendance was extremly good :-)

[1] Benford, S. and Giannachi, G. 2008. Temporal trajectories in shared interactive narratives. In Proceeding of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy, April 05 - 10, 2008). CHI '08. ACM, New York, NY, 73-82. DOI= http://doi.acm.org/10.1145/1357054.1357067

[2] Benford, S., Giannachi, G., Koleva, B., and Rodden, T. 2009. From interaction to trajectories: designing coherent journeys through user experiences. In Proceedings of the 27th international Conference on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009). CHI '09. ACM, New York, NY, 709-718. DOI= http://doi.acm.org/10.1145/1518701.1518812

[3] Benford, S., Schnädelbach, H., Koleva, B., Anastasi, R., Greenhalgh, C., Rodden, T., Green, J., Ghali, A., Pridmore, T., Gaver, B., Boucher, A., Walker, B., Pennington, S., Schmidt, A., Gellersen, H., and Steed, A. 2005. Expected, sensed, and desired: A framework for designing sensing-based interaction. ACM Trans. Comput.-Hum. Interact. 12, 1 (Mar. 2005), 3-30. DOI= http://doi.acm.org/10.1145/1057237.1057239

[4] Schnädelbach, H., Rennick Egglestone, S., Reeves, S., Benford, S., Walker, B., and Wright, M. 2008. Performing thrill: designing telemetry systems and spectator interfaces for amusement rides. In Proceeding of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy, April 05 - 10, 2008). CHI '08. ACM, New York, NY, 1167-1176. DOI= http://doi.acm.org/10.1145/1357054.1357238

Opening Keynote of Mensch&Computer 2010 by Ed H. Chi

2010-09-13T11:45:00.000+01:00

Ed H. Chi from PARC presented the opening keynote for Mensch&Computer 2010. In the motivation of the talk he showed a document on "Applied Information processing psychology" from 1971 - probably very few had seen this before. It makes an argument for an experimental science that is related to augmented cognition. The basic idea is very similar to Vannevar Bush's Memex - to extend the human cognitive power by machines (and especially computer technology). It is apparent that these ideas became the backdrop of the many innovations that happened at PARC in the early days.

Ed stressed that there is still a lot of potential for the application of psychological phenomena and models to human computer interaction research. As an example he used the idea that speech output in a navigation system could use your name in an important situation making use of the attenuation theory of attention (the cocktail party effect). By hearing your name you are more likely to listen - even if you are yourself in a conversation. The effect may be stronger if the voice is your mother's voice ;-)

The main part of the talk centered on model driven research in HCI. Using the ScentHighlights [1] examples he outlined the process. I liked very much the broad view Ed has on models and the various uses of models he suggested, e.g. generative models that generate ideas; or behavioral models that lead to additional functionalities (as example he used: people are sharing search results in google, hence sharing should be a basic function in a search tool). Taking the example of Wikipedia he showed how models can be used to predict interaction and growth. I found the question on the growth of knowledge very exciting. I think it is defiantly not finite ;-) otherwise research is a bad career choice. Looking at the Wikipedia example it is easy to imagine that the carrying capacity is a linear function and hence one could use a predictive function where a logistic growth curve is overlayed with a linear function.

Random link from the talk: http://mrtaggy.com/

Ed discussed yahoo's social pattern library:
http://developer.yahoo.com/ypatterns/social/people/reputation/
This pattern library is pretty interesting. I found the reputation pattern pretty comprehensive. It seems that this library is now comprehensive enough for using it for real and in teaching.

[1] Chi, E. H., Hong, L., Gumbrecht, M., and Card, S. K. 2005. ScentHighlights: highlighting conceptually-related sentences during reading. In Proceedings of the 10th international Conference on intelligent User interfaces (San Diego, California, USA, January 10 - 13, 2005). IUI '05. ACM, New York, NY, 272-274. DOI= http://doi.acm.org/10.1145/1040830.1040895

Mensch und Computer 2010 at the University of Duisburg-Essen

2010-09-12T23:07:00.000+01:00

The German HCI conference Mensch und Computer 2010 started today. Under a single roof - called interactive culture - three more conferences are co-located: the German UPA track, the German E-learning conference, and a track on entertainment interfaces. The size of the conference is with about 500 people impressive and it shows that interactive computing and user experience has become a major field in Germany - in academia as well as in industry - and I am proud to have chaired the paper program for Mensch&Computer together with Jürgen Ziegler.

On Sunday we ran a number of workshops: Mobile HCI (by Enrico Rukzio), Methods and Tools in HCI (by Nicole Krämer), Web 2.0 and CSCW (by Tom Gross), and on Writing scientific papers (by Geraldine Fitzpatrick). I enjoyed myself attending two of the tutorials and I have to admit I learned interesting things :-) and got ideas for my own teaching.
The paper program starting on Monday was selective as we had 119 submissions (full and short papers) and the committee chose 41 to be presented at the conference (is about 34% acceptance rate).

A restaurant to remember (in a very positive sense): Dreigiebelhaus.

Our Paper on Mobile Product Review Systems at Mobile HCI 2010

2010-09-10T01:51:00.000+01:00

Felix von Reischach investigated in his PhD mobile product review systems. The paper [1] he presented at mobile HCI 2010 in Lisbon compares different modalities for product reviews and recommendations. In particular we looked at the following modalities: discrete scale (stars), text, and video. In a study at the SAP retail lab we compared how easy it was for participants to create reviews in each of the modalities and how much they like creating these. Additionally we also compared which modalities are most liked by people in a buying situation and which type of review the trust. Interestingly a star rating scheme is most liked - for input and output.
Our general recommendation is to allow users to rate products on a scale (e.g. using stars) in different, potentially user defined categories. For a more detailed discussion see the paper [1].

The evening event of was at Palácio da Pena, Sintra - a castle close to Lisbon. The view and the food were magnificent - it felt like a real treat after the one hour walk up the steep hill.

[1] von Reischach, F., Dubach, E., Michahelles, F., and Schmidt, A. 2010. An evaluation of product review modalities for mobile phones. In Proceedings of the 12th international Conference on Human Computer interaction with Mobile Devices and Services (Lisbon, Portugal, September 07 - 10, 2010). MobileHCI '10. ACM, New York, NY, 199-208. DOI= http://doi.acm.org/10.1145/1851600.1851635

Abstract:
Research has shown that product reviews on the Internet not only support consumers when shopping, but also lead to increased sales for retailers. Recent approaches successfully use smart phones to directly relate products (e.g. via barcode or RFID) to corresponding reviews, making these available to consumers on the go. However, it is unknown what modality (star ratings/text/video) users consider useful for creating reviews and using reviews on their mobile phone, and how the preferred modalities are different from those on the Web. To shed light on this we conduct two experiments, one of them in a quasi-realistic shopping environment. The results indicate that, in contrast to the known approaches, stars and pre-structured text blocks should be implemented on mobile phones rather than long texts and videos. Users prefer less and rather well-aggregated product information while on the go. This accounts both for entering and, surprisingly, also for using product reviews.

Tobii Mobile Eye-Tracker

2010-09-09T17:02:00.001+01:00

After quite some time of announcements and paper presentation Tobii showed a beta version of their mobile eye tracker at Mobile HCI 2010. It is small and lightweight but requires a import phase after recording. It seems well suited to do classical studies (e.g. you have people doing some task and you analyse afterwards what they looked at) but of little use for interactive application (e.g. the system reacts in real-time to the users gaze). Technically the glasses record the video and gaze information on a memory card. After the recording this needs than to be "imported" on the PC. After the import the gaze is superimposed over the recorded video. Tobii also provides active markers that can be attached to objects and regions to ease post processing and analysis. The price tag in Europe is just below 20.000€. SMI has a similar product: IVIEW X HED.

Keynote by Josh Ulm at Mobile HCI 2010

2010-09-09T16:49:00.000+01:00

Josh Ulm discussed how branding (and marketing in more general) has evolved and how this is now central to user experience and user interface design. He started out with showing how Nike changed marketing with the "Just Do It" campaign. He suggested that this was a transition from a product focus to a personal usage focus asking "who you become if you use the product".

Moving towards more resent trends he argued that the iPod made the interaction with the product the essential part of the branding. With examples such as eBay and Google he showed that interaction in combination with information presentation becomes that discriminating factor and the way these brands define themselves. Overall this suggests that the user interaction and user experience is the central part for making a brand to stand out.

Standing out is not sufficient however. The experience has to be ownable. Using zappos as an example he showed how such an experience needs to be consistent across all touch points with the user - especially if you are defining your brand by experience - in short a brand has to have a unique user experience that is associated with the brand only. To achieve this there are three ingredients:

Values - "values have to extent into every single touchpoint of the experience"
Differentiation "you need to stand out in the market place" - many companies do not innovate enough, you have to take risks to stand out because you have to stand out a lot to be different - most companies do only innovate a little
Integrity - internal consistency - the brand is about what customers really touch - what reaches the customers - can a customer recognize that this is your experience, detail matter

To be successful an experience needs to be ownable and good. You need both - one is not enough. As a concluding example he showed Vodafone 360 - (which very few people in the audience knew). My short assessment is that the brand failed on "Differentiation" - to me (and obviously I would expect Josh Ulm is not agreeing) it is too similar to other things in this space…

As a further example of a strong Differentiation he showed Jeff Fong's Metro UI for the new Windows mobile phone platform. If you have not seen the design check this out. It is very different from current Android and iPhone UIs. Everyone is really curious how it will do in the market…

Opening Keynote of Mobile HCI by Patrick Baudisch

2010-09-07T23:32:00.000+01:00

Patrick Baudisch presented the opening Keynote. He gave an interesting overview of work on interaction with small screen devices. It is very apparent that currently the form factor of devices is nearly exclusively determined by the I/O components. Throughout the examples it became very explicit, that the usefulness of a device for certain tasks it determined by the options for interaction. E.g. by increasing the input precision or by reducing the fat finger problem new tasks (and eventually new applications) will become possible on mobile devices. Patrick showed an example where he asks his students, to design a UI for a device as small as a one Euro coin (e.g. digital jewelry) - I think I borrow this idea for our next course :-)

How showed many examples of his work - too many to write it in a blog entry - but well documented on his web page. It is a page worthwhile to check regularly as his group is outstandingly creative and productive.

Patrick made an insightful analogy between small screens and theater stage, arguing that the both have a similar problem: the lack space to show everything you would like to show due to a constraint space. The solutions in the theater can give a good inspiration for designing mobile UIs. He mentioned three typical solutions used in the theater:

Partially out of the frame - you show something only to a small part and the user's imagination will fill in for the rest. In the theater an example is a ocean liner where you only see a very small part but people imagine the rest. The halo visualization technique [1] is one example for this approach on a mobile device.
In and out points - people enter and leave the stage at given points providing an illusion of a comprehensible world around the stage (and in the UI case around the screen)
Direction of interaction to invisible targets - by talking/pointing/looking into a certain direction actors can create an impression of interacting with someone at a certain location - who is not really there and who's position is off the stage.

Technically I found the video he showed on using a depth cam extremely exciting. He speculated that this may solve many problems we have currently with using computer vision in difficult light conditions. In his example he showed the image recorded by a depth cam (based on time of flight) "filming" into direct sunlight - and as one would expect the sun does not matter. Looking forward to seeing some more results on this…

Looking at the current re-appearance of watch based computers and phones the idea of back side interaction becomes more and more interesting. In one design he showed how one can interact with a watch size device by interacting on buckle of the watch band. For more on back of device interaction see [2].

[1] Baudisch, P. and Rosenholtz, R. 2003. Halo: a technique for visualizing off-screen objects. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Ft. Lauderdale, Florida, USA, April 05 - 10, 2003). CHI '03. ACM, New York, NY, 481-488. DOI= http://doi.acm.org/10.1145/642611.642695

[2] Baudisch, P. and Chu, G. 2009. Back-of-device interaction allows creating very small touch devices. In Proceedings of the 27th international Conference on Human Factors in Computing Systems (Boston, MA, USA, April 04 - 09, 2009). CHI '09. ACM, New York, NY, 1923-1932. DOI= http://doi.acm.org/10.1145/1518701.1518995

Opening of MobileHCI 2010

2010-09-07T22:27:00.002+01:00

Over 300 participants from 5 continents came to Lisbon for MobileHCI 2010. This year's conference had a very high nigh number of submission and a very low acceptance rate (23% for full papers, below 20% for short papers). Looking back over the last 10 years it is amazing how this community has grown!

The statistics also shows the conference has a strong European focus (in contrast to CHI and UIST that are US dominated). Over the next years we want to make the Mobile HCI conference more international and attract more submissions and participation from around the world and in particular for the US. The plan is to have the conference 2012 in the US. Next year I will help (as paper co-chair) to run mobile HCI 2011 in Stockholm, Sweden. If you have ideas how to make the conference more attractive to people in America and Asia please let me know!

The deadline for submitting to mobile HCI 2011 is Janaury 28th, 2011.

SiMPE 2010, Keynote: Trends and Challenges in Mobile Interaction

2010-09-07T16:58:00.000+01:00

I was invited to give a keynote talk at the 5th Workshop on Speech in Mobile and Pervasive Environments that was held as a part of ACM MobileHCI 2010 in Lisbon, Portugal. Over the last years we looked at speech as an additional modality in the automotive user interface domain; besides this my experience with speech interfaces is limited.

My talk, with the title "Trends and Challenges in Mobile Interaction" looked at different issues in mobile interaction. In some parts I reflected on modalities and opportunities for speech interaction.

When characterizing mobile interaction I pointed out the following points:

Interacting while on the move
Interaction is one of the user's tasks (besides others - e.g. walking, standing in a crowd)
Environment in which the interaction takes place changes (e.g. on the train with varying light and noise conditions)
Interruptions happen frequently (e.g. boarding the bus, crossing the road)
Application usage is short (typically seconds to minutes)

My small set of key issues and recommendations for mobile UIs is:

Simple and Understandable
Perceptive and Context-Aware
Unobtrusive, Embedded and Integrated
Low Cognitive Load and Peripheral Usage
Users want to be in Control (especially on the move)

The presentations and discussion at the workshop were very interesting and I got a number of ideas for multimodal user interfaces - including speech.

One issue that made me think more was the question about natural language speech vs. specific speech commands. A colleague pointed me to Speech Graffiti [1] / Universal Speech Interface at CMU. I wonder if it would make sense to invent a Human Computer Interaction language (with a simple grammar and a vocabulary) that we could teach in a course over several weeks (e.g. similar effort than touch typing on a QUERTY keyboard) or as a foreign language at school to have a new effective means for interaction. Could this make us more effictive in interacting with information? Or should we try harder to get natural languge interaction working? Looking at the way (experienced) people use Google we can see that people adapt very successfully - probably faster than systems improve…

From some of the talks it seems that "Push to talk" seems to be a real issue for users and a reason for many user related errors in speech systems. Users do not push at the appropriate time, especially when there are other tasks to do, and hence utterances are cut off at the start and end. I would guess continuous recording of the speech and using the "push to talk" only as an indicator where to search in the audio stream may be a solution.

[1] Tomko, S. and Rosenfeld, R. 2004. Speech graffiti vs. natural language: assessing the user experience. In Proceedings of HLT-NAACL 2004: Short Papers (Boston, Massachusetts, May 02 - 07, 2004). Human Language Technology Conference. Association for Computational Linguistics, Morristown, NJ, 73-76. http://www.cs.cmu.edu/~usi/papers/HLT04.pdf

Lab visit in Chengdu, University of Electronic and Science Technology of China

2010-08-28T04:47:00.005+01:00

On the final day of the Sino-German Symposium on Wearable Computing in Chengdu Prof. Dongyi Chen invited us to see his lab. We drove to the new campus of the University of Electronic and Science Technology of China. Already the drive was impressive seeing the amount of building work happening in Chengdu, especially in the high tech area.

In the School of Computer Science and Engineering we visited a computer lab and got to see very interesting student projects in Prof. Dongyi Chen labs. The demos included a wireless controlled vehicle were the control is implemented on a mobile phone, industrial settings control applications using sensor nets, table top user interfaces and augmented reality applications on the table, different applications for wearable displays, and a wrist worn computer (developed from scratch).

The quality of the work by the students is impressive and so is the university campus (building and facilities). It shows a very clear determination to push science and education. We should probably talk to our government to consider investing more in research and higher education…
I hope this symposium will help us to start some more collaboration. As a next step we plan a summer school on Human Computer Interaction next year in Germany.

Public Displays in Chengdu

2010-08-27T19:25:00.000+01:00

This is a post to document the transition of public display systems around the work (hopefully there are more to follow). In the European pdnet project we investigate and develop a new communication medium based on public displays. The transformation that takes place in this domain is extremely quick, and hence I think it is interesting to keep some record on specific displays.I found it very interesting to see that there are many simple animations used in simple displays when they become digital. One example is the use of animation in traffic lights. There are two types of dynamic information: informative (e.g. showing the count-down to the next green light) and decorative (e.g. moving bicycle wheels as shown in the video).Currently there is a mix of traditional painted/printed displays, illuminated static displays, illuminated displays that change between discreet presentations and fully digital high resolution displays. The digital advertising displays in the inner city center of Chengdu are impressive. Looking at the photos it could be anywhere in world. It seems that public displays lose more and more their local character - very different from 20 years ago when they were still painted/printed in most places around the world.

Live experience - media consumption is social

2010-08-27T18:14:00.001+01:00

Before the dinner I decided to try a Chinese massage and it was astonishingly relaxing. It is one of those reminders that there are many things we need to experience and there is just no other way (at least so far) to gain a similar understanding…

The show after dinner showed to me how much a live presentation of artistic and musical performance transmits - it is so much richer than conserved/recorded media. Take as an example the shadow play - I really enjoyed it as live performance. In comparison to 3D animation movies it has little fidelity but it still works extremely well to engage people in the live presentation. But I could not imagine that I would watch it on TV - hence we probably miss something in creating the experience when playing/presenting conserved media. I would expect there is a lot potential in creating a social situation for digital media consumption that could improve the experience.

Chinese German Symposium on Wearable Computing in Chengdu

2010-08-27T11:02:00.000+01:00

The Sino-German Symposium on Wearable Computing in Chengdu provided an interesting opportunity to get together with colleagues in China that work on similar topics.
My talk was entitled "Interaction on the Move - Wearable User Interfaces" and look at a very high level perspective at mobile and wearable interaction. As the main objective of the symposium is to initiate collaboration I also included some slides on the other work we are doing.

Bernt Schiele looked back on his early work in 1998 at MIT with Sandy Pentland and reflected on how wearable computing has evolved. It seems than many of the scenarios that were originally envisioned are now realized on smart phones and it seems that an active usage model - people taking the device explicitly instead of having something that is always on and in their face. However looking back on some of the early visions (continuous capture, contextual support) they are still attractive and the technology may be there to realize them for real. One example of a system that could now be easily realized and may have a growing market could be a device similar to the StartleCam [1], providing personal safety services.

Feng Tian, one of the top HCI researchers in China gave an overview of their current work which I found very exciting (especially the projects related to sports and education). Hopefully there is a chance for future collaboration.

There is more information on the symposium and on the Sino-German collaboration at Wearcom.org.

[1] Healey, J. and Picard, R. W. 1998. StartleCam: A Cybernetic Wearable Camera. In Proceedings of the 2nd IEEE international Symposium on Wearable Computers (October 19 - 20, 1998). ISWC. IEEE Computer Society, Washington, DC, 42.

In the talks and conversation I saw a set of technologies I like to remember (and share) some of them:

Motion sensing suit from XSens
Personal Display Carl Zeiss Cinemizer 3D Plus
MotionNode - a 3-DOF inertial measurement unit (IMU)

Decorative Displays in Zürich Railway Station

2010-08-23T17:47:00.003+01:00

In the railway station in Zürich is a display that consists of 25.000 light units (=10x50x50 - by my counting). It seems that for each unit the color can be set. As there is some space between the lights one can see the "hidden" layers which makes it a sort of 3D image. So far it seems to be used only as an artistic/decorative display - or I did not get the meaning. There is also still "conventional" art in the railway station…

I recognized an interesting effect of human behavior. I looked at the electronic display for about 2 minutes and nobody took a photo of it. When I started taking the photos there were within 30 seconds 10 other people starting to take photos ;-) Perhaps we should create an application that makes this easier - consiting of three parts: (1) if someone takes a photo the photo application broadcasts this event, (2) an application running in the back ground monitoring when others take photos and records the location and (3) a page, folder or dynamic query that shows fotos for this location (and perhaps sorted by time difference to the recording of the event).

PhD defense of Michael Kuhn at ETH Zurich

2010-08-23T16:15:00.001+01:00

Michael Kuhn defended his PhD thesis on "Understanding and Organizing User Generated Data: Methods and Applications" at ETH Zurich and I had the honor to be one of the examiners. His thesis is a prime example how solid theoretical concepts and practical applications go well together. He investigated the similarities in different domains, including people, conferences, and music. I came first across his work at Mobile HCI [1]. He has published an interesting set of papers on his work, see his page at ETH.

As one of the datasets he used http://www.livejournal.com which is can be freely crawled. This is an interesting resource for doing research on social networks.

As part of his dissertation project he implemented several applications. I found the following two remarkable and very useful:

http://www.confsearch.org Looking for conference based on keywords? Searching for a conference in a field? Which conferences are related? Have a look and you will find some answers.
http://www.museek.ethz.ch a comprehensive mobile music application for Android.

PS: http://academic.research.microsoft.com/ is another conference/publication search site - not sure how much I believe in automating the rankings of scientists - as the sites lists me with Aalborg University - and even a change request did not help to put my affiliation right ;-)

[1] Goussevskaia, O., Kuhn, M., and Wattenhofer, R. 2008. Exploring music collections on mobile devices. In Proceedings of the 10th international Conference on Human Computer interaction with Mobile Devices and Services (Amsterdam, The Netherlands, September 02 - 05, 2008). MobileHCI '08. ACM, New York, NY, 359-362. DOI= http://doi.acm.org/10.1145/1409240.1409288

Call for Papers: IEEE Pervasive Computing special Issue on Automotive Pervasive Computing

2010-07-30T18:42:00.004+01:00

Next year will be a special issue of the IEEE Pervasive Computing magazine on Automotive Pervasive Computing. I am honored to edit this issue together with Joe Paradiso and Brian Noble :-) The submission deadline for full papers is October 1st and work in progress one month later - see the call for papers for details.

Cars have become an interesting and challenging microcosm for pervasive computing research and we invite articles relating to pervasive computing in the automotive context. Examples of relevant topics are:

Sensing and context in automotive environments

Pervasive sensor systems in the car
Use of sensors and context for automotive applications
Contextual vehicular applications
Collaborative sensing with multiple cars

Automotive user interfaces

Concepts for in-car user interfaces based on pervasive computing techology
Multi-modal interaction in the car
Detecting user intentions, emotions, and distraction
User interfaces for assistive functionality and autonomous driving
Applications of car to car communication

Pervasive computing applications in the car

Contextual information and navigation systems
Technologies to improve media consumption while driving
Communication appliances for drivers and passengers
In-car pervasive gaming for passengers and drivers

Experience with pervasive computing in the car

Experiences with pervasive computing technologies in cars
Case studies of automotive pervasive computing
Ethnographic work on the use of technologies in cars

For details see the cfp at: http://computer.org/pervasive/cfp3

Scientific papers as audio content?

2010-07-28T23:55:00.001+01:00

We have started to experiment with reading articles and providing them as MP3 files or podcasts (see the facebook page or the blogpost). So far I found for myself a number of places where I like it - from gym to car. Perhaps we should make it mandatory that the camera ready version of the paper is the PDF, the source, and an audio file (e.g. the paper read by the author or a description of the work - I guess the authors reading their papers could improve some papers as the authors would be finally read what they write ;-)

Coming across this sign in a bookshop in Essen made me smile - especially as we look into ways that may making reading feasible while driving [1].

[1] Kern, D., Marshall, P., and Schmidt, A. 2010. Gazemarks: gaze-based visual placeholders to ease attention switching. In Proceedings of the 28th international Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA, April 10 - 15, 2010). CHI '10. ACM, New York, NY, 2093-2102. DOI= http://doi.acm.org/10.1145/1753326.1753646

PS: to answer some of the questions about the audio files of research papers I got recently: Yes, I think it is nice to have real humans reading. Yes, I know that there are brilliant text to speech software (but as long as they do the Simpsons with actor's voices we are not there yet).

Handheld Laser Projector

2010-07-26T14:19:00.000+01:00

Enrico got for his project a handheld laser projector. The nice thing about this technology is that the projected image is always in focus - it does not matter if you move the projector or if you project on an uneven surface. The AAXA L1 Laser Pico Projector (even though there are plenty of things that can be improved, noise for a start) provides some inspiration what will become possible if these devices will be common in mobile phones. In Lancaster Enrico explored already 2007/2008 some of the usage scenarios and interaction techniques [1] and I am really curious of further ones. There was a workshop at Pervasive 2010 looking at current research on personal projectors [2].

With this piece of technology we can also move on the idea of a device where interactive shell and functional core of a product is separated. We have published the concept as work in progress at CHI [3] and perhaps it is now time to look into a realistic implementation using such a laser projector.

[1] Hang, A., Rukzio, E., and Greaves, A. 2008. Projector phone: a study of using mobile phones with integrated projector for interaction with maps. In Proceedings of the 10th international Conference on Human Computer interaction with Mobile Devices and Services (Amsterdam, The Netherlands, September 02 - 05, 2008). MobileHCI '08. ACM, New York, NY, 207-216. DOI= http://doi.acm.org/10.1145/1409240.1409263

[2] UBIPROJECTION 2010 http://eis.comp.lancs.ac.uk/workshops/ubiproject2010/

[3] Doering, T., Pfleging, B., Kray, C., and Schmidt, A. 2010. Design by physical composition for complex tangible user interfaces. In Proceedings of the 28th of the international Conference Extended Abstracts on Human Factors in Computing Systems (Atlanta, Georgia, USA, April 10 - 15, 2010). CHI EA '10. ACM, New York, NY, 3541-3546. DOI= http://doi.acm.org/10.1145/1753846.1754015

Teaching Ubicomp? What is it we should teach?

2010-07-23T16:00:00.000+01:00

Why should we teach Ubicomp? What are the core issues when teaching Ubicomp? How do we cope with the rapid changes in technologies if we provide practical exercises in our Pervasive Computing classes? What skills will students take away from the course?

As Ubicomp is still a young and dynamic subject it is inevitable that we have to ask these questions. To share our experiences in teaching we met at the ETH Zürich. Friedemann Mattern, Marc Langheinrich, Michael Rohs, Kay Römer, and many of our PhD students (and me ;-) spent two days in Zürich to collect materials and discuss the above questions. The hardest one is obviously the what is ubicomp question…

For me the key thing is that we teach about distributed computing systems that are aware and linked to the real world and which are used by humans. The systems aspect is key and I think which specific technologies, tools, methods, we teach are exchangeable. The second point I want to make in my pervasive computing class is to get students excited and aware of the potential of computing in the future and how we are at the heart of a major change that goes beyond technology.

We are currently compiling a Wiki with teaching materials which we hope will become public in the future (at least in parts). If you teach a ubicomp related course or if you know of course please feel free to add a comment with link to the webpage and we will try to include it in the collection.

PS: examples of the UI challenge of ubicomp are everywhere.

Feature or Bug - how do you end an annoying call?

2010-07-20T00:28:00.004+01:00

My first mobile phone in 1998 had a detachable antenna (you could just screw the antenna off). This design feature enabled a socially acceptable mechanism to end annoying conversations by losing the connection. When talking became boring you told the communication partner that it seems that the connection gets worse (while you unscrewed the antenna) and then when you detached the antenna the call was usually dropped.

Reading about the iPhone I thought about this long forgotten feature and expected that Apple listened to its users and that there is a next version of this feature - nowadays implemented as touch interaction (not screwing required) on the iPhone 4. The photo shows a phone where the feature is disabled by some rubber cover.

I was really disappointed that Apple tells us it is a bug…