Keynote by Cecilia Mascolo at Percom2016: Technology and Experiecne in the Physcial World

Cecilia Mascolo presents the keynote at Percom2016. Her opening statement is: “Technology must enhance and not substitute the physical experience”.

Cecilia makes the point that continuous sensing with mobile devices can overcome many issue that are well known with traditional studies (especially the classical problem of psychologist studying psychology students in a dark lab). One of here early papers (EmotionSense, see [1]) shows how we can move studies into the real world. This is not without difficulties, especially when you try to understand emotions.

Putting research apps into android market changes the game, large numbers of users become within reach. Higher numbers of participants require a clear purpose of the applications (Nielse Henze provide in [2] a nice recipe of how to do this). Her experience is that user engagement through gamification really worked. Even if the duration of participation of individuals is limited to weeks or months this generates very useful information. A short introduction to social sensing by Cecilia can be found in [3].

Different sensors have different energy and privacy cost and also different types of contributions. Correlating the accelerometer and happiness is really interesting. Users who are more active (not just movement, “being out and about”) are happier. Clustering accelerometer data and correlating it with other high level data opens exciting questions, e.g. health. Similarly correlating happiness and location leads to more surprising results: less happy at home and work, more when out and active. Looking at people’s personally and demographics shows that gender, age, employment, etc. has a clear correlation with activity and usage of communication.

Physical space matters! Using active badges they looked at how the change of physical space can impact peoples interactions [4]. The sensing approach allowed to understand how changes in physical space changes the behavior on a really fine grained level.

References:
[1] Rachuri, K. K., Musolesi, M., Mascolo, C., Rentfrow, P. J., Longworth, C., & Aucinas, A. (2010, September). EmotionSense: a mobile phones based adaptive platform for experimental social psychology research. In Proceedings of the 12th ACM international conference on Ubiquitous computing (pp. 281-290). ACM. http://csce.uark.edu/~tingxiny/courses/5013sp14/reading/Rachuri2010EMP.pdf
[2] Henze, N., Shirazi, A. S., Schmidt, A., Pielot, M., & Michahelles, F. (2013). Empirical research through ubiquitous data collection. Computer, 46(6), 0074-76. http://doi.ieeecomputersociety.org/10.1109/MC.2013.202
[3] Mascolo, C. (2010). The power of mobile computing in a social era. IEEE Internet Computing, 14(6), 76. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.466.2037&rep=rep1&type=pdf
[4] Brown, C., Efstratiou, C., Leontiadis, I., Quercia, D., Mascolo, C., Scott, J., & Key, P. (2014, September). The architecture of innovation: Tracking face-to-face interactions with ubicomp technologies. In Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (pp. 811-822). ACM. http://arxiv.org/pdf/1406.6829.pdf

Visit to Nokia Research Center Tampere, SMS, Physiological sensors

This trip was my first time in Tampere (nice to see sometimes a new place). After arriving yesterday night I got a quick cultural refresher course. I even met a person who was giving today a presentation to the president of Kazakhstan (and someone made a copy using a phone - hope he got back OK to Helsinki after the great time in the bar).

In the morning I met a number of people in Jonna Hakkila's group at the Nokia Research Center. The team has a great mix of backgrounds and it was really interesting to discuss the project, ranging from new UI concepts to new hardware platform - just half a days is much too short… When Ari was recently visiting us in Essen he and Ali started to implement a small piece of software that (hopefull) improves the experience when receiving an SMS (to Ali/Ari - the TODOs for the Beta-release we identified are: sound design, screen design with statistics and the exit button in the menu, recognizing Ok and oK, autostart on reboot, volume level controlable and respecting silent mode). In case you have not helped us with our research yet please fill in the questionnaire: http://www.pcuie.uni-due.de/uieub/index.php?sid=74887#

I gave a talk (see separate post on the next big thing) and had the chance to meet Jari Kangas. We discovered some common interest in using physiological sensing in the user interface context. I think the next steps in integrating physiological sensors into devices are smaller than expected. My expectation is that we rather detect simple events like "surprise" rather than complex emotion (at least in the very near future). We will see where it goes - perhaps we should put some more students on the topic…

Do you use Emoticons in your SMS? What are the first words in the SMS you receive?

We are curious in current practice in SMS use - and I hope for a good reason. Together with Jonna Hakkila and her group at Nokia Research we have discussed ideas how to make SMS a bit more emotional. Hopefully we have soon a public beta of a small program out. 

Till then it would be helpful to understand better how people use SMS and how the encode emotion in a very rudimentary way by :-) and :-( and alike. If you are curious, too and if you have 10 minutes it would be great to complete our survey: http://www.pcuie.uni-due.de/uieub/index.php?sid=74887#

Emotions in SMS and mobile communication has been a topic many people have been looking in, one of the early paper (in fact a design sketch not really a paper) was by Fagerberg et al.  [1] in our 2004 special issue on tangible UIs - an extended and more conceptual discussion of their work can be found in [2]; for more on their project see: http://www.sics.se/~petra/eMoto/

[1] Petra Fagerberg, Anna Ståhl, and Kristina Höök (2004) eMoto - Emotionally Engaging Interaction, Design Sketch in Journal of Personal and Ubiquitous Computing, Special Issue on Tangible Interfaces in Perspective, Springer.

[2] Ståhl, A., Sundström, P., and Höök, K. 2005. A foundation for emotional expressivity. In Proceedings of the 2005 Conference on Designing For User Experience (San Francisco, California, November 03 - 05, 2005). Designing For User Experiences, vol. 135. AIGA: American Institute of Graphic Arts, New York, NY, 33.

Workshop on User Experience at Nokia

Together with Jonna Hakkila’s group (currently run by Jani Mantyjarvi) we had a two day workshop at Nokia in Oulu discussion the next big thing* ;-)
* motto on the Nokia research centers web page

It seems that many people share the observation that emotions and culture play a more and more important role in the design of services and applications – even outside the research labs. One evening we looked for the Finnish experience… (photo by Paul)

Overall the workshop showed again how many ideas can be created in a very short time – hopefully we can follow up some of them and create some new means for communication. We plan to meet again towards the end of the year in Essen.

PS: Kiss the phone – some take it literarily: http://tech.uk.msn.com/news/article.aspx?cp-documentid=7770403

PPS: we talked about unanticipated use (some call it misuse) of technology, e.g. using the camera on the phone to take a picture of the inside of your fridge instead of writing a shopping list. Alternative uses is not restricted to mobile phones - see for yourself what you dishwasher may be good for.... http://www.salon.com/nov96/salmon961118.html