CHI 2011 in Vancouver, Keynote and Papers

In the opening keynote Howard Rheingold proclaimed that we are in a time for learners and he outlined the possibilities that arise from the interactive media that is available to us. In particular he highlighted the fact that people share and link content and to him this is at the heart of learning. Learning as a joined process where contributions by students - in different forms of media - become major a resource was one example.

I best liked his analogy on how little innovation there is in teaching. "If you take a warrior from 1000 years ago on a battlefield today - they will die - quickly. If you take a surgeon from a 1000 years ago and put them in a modern hospital - they will be lost. If you take a professor from 1000 years ago and put them in a University today he will exactly know what to do. " I am not sure about the 1000 years but it by 100 years the story works just as well. In essence he argued that there is a lot of potential for new approaches for teaching and learning.

After initially agreeing I gave it some more thoughts and perhaps the little change in learning and teaching shows that learning is very fundamental and technology is overrated in this domain? What is more effective than a teachers discussing in an exciting topic face to face with a small set of students - perhaps even while on a walk? Reminds me about things I read about the Greek teachers and there practices several thousand years ago … and it makes me looking forward to our summer school in the Italian Alps (http://www.ferienakademie.de/).

I found the SIGCHI Lifetime Achievement Award lectures very exciting and educational. Especially the talk by Larry Tesler provided deep insight into how innovation works in user interfaces - beyond the academic environment. He talked about the "invention" of cut and paste - very enjoyable!

This year we had a number of papers describing our research in CHI:

•  Elba reported on the field study in Panama using mobile phones to enhance teaching and learning [1]
• Ali presented work on how to increase the connectedness between people by simple means of iconic communication in the context of a sports game [2]
• Tanja showed how touch and gestural input on a steering wheel can reduce the visual distraction for a driver [3], and
• Gilbert (from LMU Munich) presented work on interaction with cylindrical screens [4].

The most inspiring and at the same time the most controversial paper for me was the possessed hand by Jun Rekimoto et al. [5]. He reported their results in using electro stimulation in order to move fingers of a hand.

Bill Buxton showed throughout the conference his collection of input and output devices (Buxton Collection). Seeing the collection physically is really exciting, but for all who did not have the chance there is a comprehensive online version with photos and details available at micosoft research: http://research.microsoft.com/en-us/um/people/bibuxton/buxtoncollection/

[1] Elba del Carmen Valderrama Bahamondez, Christian Winkler, and Albrecht Schmidt. 2011. Utilizing multimedia capabilities of mobile phones to support teaching in schools in rural panama. In Proceedings of the 2011 annual conference on Human factors in computing systems (CHI '11). ACM, New York, NY, USA, 935-944. DOI=10.1145/1978942.1979081 http://doi.acm.org/10.1145/1978942.1979081

[2] Alireza Sahami Shirazi, Michael Rohs, Robert Schleicher, Sven Kratz, Alexander Müller, and Albrecht Schmidt. 2011. Real-time nonverbal opinion sharing through mobile phones during sports events. In Proceedings of the 2011 annual conference on Human factors in computing systems (CHI '11). ACM, New York, NY, USA, 307-310. DOI=10.1145/1978942.1978985 http://doi.acm.org/10.1145/1978942.1978985

[3] Tanja Döring, Dagmar Kern, Paul Marshall, Max Pfeiffer, Johannes Schöning, Volker Gruhn, and Albrecht Schmidt. 2011. Gestural interaction on the steering wheel: reducing the visual demand. In Proceedings of the 2011 annual conference on Human factors in computing systems (CHI '11). ACM, New York, NY, USA, 483-492. DOI=10.1145/1978942.1979010 http://doi.acm.org/10.1145/1978942.1979010

[4] Gilbert Beyer, Florian Alt, Jörg Müller, Albrecht Schmidt, Karsten Isakovic, Stefan Klose, Manuel Schiewe, and Ivo Haulsen. 2011. Audience behavior around large interactive cylindrical screens. In Proceedings of the 2011 annual conference on Human factors in computing systems (CHI '11). ACM, New York, NY, USA, 1021-1030. DOI=10.1145/1978942.1979095 http://doi.acm.org/10.1145/1978942.1979095

[5] Emi Tamaki, Takashi Miyaki, and Jun Rekimoto. 2011. PossessedHand: techniques for controlling human hands using electrical muscles stimuli. In Proceedings of the 2011 annual conference on Human factors in computing systems (CHI '11). ACM, New York, NY, USA, 543-552. DOI=10.1145/1978942.1979018 http://doi.acm.org/10.1145/1978942.1979018

Floor for activity recognition

Patrick Baudisch showed in Paris at the Microsoft Software summit interesting photos of their DIY activity in creating a glass floor for tracking and activity recognition. With a fairly large glass pane in the floor they have created an interesting environment… I am sure there will be interesting things coming out of this installation.

Some years back in 2002 (and looking at the photos and the amount of hair I still had this seems long ago) in Lancaster we also looked in what to do with floors (and we were into DIY as well). We also considered arrangements with a floor tables and furniture on top. As you can see from Kristof, Hans and me on the photo it was a fun project.

The positive point in using load sensing is that you can track unobtrusive and potentially large scales with little instrumentation. We even considered the possibility to put a house on 4 load cells and do activity recognition based on this. We never got around to building the house ;-) The problem with load sensing is that you can only track one moving object/subject at the time.

Looking at the signature of the load measured and doing some signal processing we could detect events – unobtrusive and cheap – but only for single events.

Interested in more details? Have a look at the publications on load sensing [1], on the interaction [2], and at a patent [3] describing the basic technology.

[1] Schmidt, A., Strohbach, M., Laerhoven, K. v., Friday, A., and Gellersen, H. 2002. Context Acquisition Based on Load Sensing. In Proceedings of the 4th international Conference on Ubiquitous Computing (Göteborg, Sweden, September 29 - October 01, 2002). Springer-LNCS, London, 333-350.

[2] Schmidt, A.; Strohbach, M.; van Laerhoven, K. & Hans-W., G. 2003. Ubiquitous Interaction -  Using Surfaces in Everyday Environments as Pointing Devices, Universal Access Theoretical Perspectives, Practice, and Experience (UI4ALL 2003), Springer LNCS, 263-279.

[3] Schmidt, A., Strohbach, M., Van Laerhoven, K., Friday, A., Gellersen, H-W., Kubach, U.; Context acquisition based on load sensing. US Patent 7434459. US Patent Issued on October 14, SAP AG (DE), 2008

WP7 Tutorial - part 5: Orientation and Acceleration - X,Y,Z

Detecting gestures, orientations, and movement can be realized with the accelerometer. The accelerometer is a sensor that measures the acceleration in 3 dimensions (X, Y, and Z). If the device is not moved the accelerations measure are the gravity forces in each direction. If the device is accelerated the measured results are a combination of the acceleration and the gravity.

The accelerometer data can be accessed via the AccelerometerReadingEventArgs class. The class has values for the X, Y, and Z axis. The values are of type double and between -2 and 2 which related to acceleration "for each axis in gravitational units" - 1 is the gravitational force of the earth. See: http://msdn.microsoft.com/en-us/library/ff431744(v=vs.92).aspx and http://msdn.microsoft.com/en-us/library/ff431810(v=vs.92).aspx or on page 80ff, C. Petzold, Programming Windows Phone 7.

A typical exercise for understanding the accelerometer is to create a bubble level (a tool to measure if something is horizontal or vertical - e.g. for hanging pictures on the wall). You probably want to freshen up on arctan2 - at least I needed ;-)

See below the c# example reading out the accelerometer on a windows phone 7. You can also download the accelerometer project directory in a single ZIP-file.

using System;
using System.Windows;
using Microsoft.Phone.Controls;
using Microsoft.Devices.Sensors;

// A simple example to read the accelerometer and display the values
// In order to make it work you have to add the refercerence to 
// Microsoft.Devices.Sensors to your project. To do this right-click
// in the Solution Explorer on References and than choose add Reference
// in the dialog then select Microsoft.Devices.Sensors
// Albrecht Schmidt, University of Stuttgart

// for a more comprehensive example see: 
// http://msdn.microsoft.com/en-us/library/ff431810(v=vs.92).aspx
// http://msdn.microsoft.com/en-us/library/ff431744(v=vs.92).aspx
// and page 80ff, C. Petzold, Programming Windows Phone 7

namespace Accl_X_Y_Z
{
    public partial class MainPage : PhoneApplicationPage
    {
        Accelerometer accelerometer;
    
        public MainPage()
        {
            InitializeComponent();
            // create a new instance
            accelerometer = new Accelerometer();
            // register a callback function for when values change
            accelerometer.ReadingChanged += new EventHandler<AccelerometerReadingEventArgs>(accelerometer_ReadingChanged);
            // start the accelerometer
            accelerometer.Start();
        }

        void accelerometer_ReadingChanged(object sender, AccelerometerReadingEventArgs e)
        {
            // required as from here the textBlocks cannot be accessed
            Deployment.Current.Dispatcher.BeginInvoke(() => ChangeUI(e));
        }
        
        void ChangeUI(AccelerometerReadingEventArgs e)
        {
            // show the values on the screen
            textBlock1.Text = "X: " + e.X.ToString("0.000");
            textBlock2.Text = "Y: " + e.Y.ToString("0.000");
            textBlock3.Text = "Z: " + e.Z.ToString("0.000");
        }
    }
}

WP7 Tutorial - part 4: Storing Data on the Phone

If you want to save high scores or preferences you need persistent memory on the phone. On a traditional computer you would create a file and store your information in the file; another option on a Windows PC would be store such information in the registry. For security reasons there is no API to access the file system and there is no global persistent memory across applications on a WP7.

In general there are two ways to store data: (1) in a application specific storage (isolated storage) on the phone or (2) remotely on the internet (or to use another buzzword "in the cloud").
In this example the use of the phone isolated storage API is demonstrated. It is shown how to store and retrieve name-value pairs on the phone (to people who programmed Java ME this is conceptually similar to the record store).

For more details see: http://msdn.microsoft.com/en-us/library/cc221360(v=VS.95).aspx and page 126ff, C. Petzold, Programming Windows Phone 7. It is also possible to created an isolated storage for files see: http://msdn.microsoft.com/en-us/library/system.io.isolatedstorage.isolatedstoragefile(v=VS.95).aspx

See below the c# example using the local storage on a windows phone 7. You can also download the IsolatedStorageSettings project directory in a single ZIP-file.

using System;
using System.Windows;
using Microsoft.Phone.Controls;
using System.IO.IsolatedStorage;

// example of how to save to and load from the isolated application storage 
// this helps to create persistence storage within a single application
// in this example it is shown how to do it for a string 
// Albrecht Schmidt, University of Stuttgart  


// For more details see:
// http://msdn.microsoft.com/en-us/library/cc221360(v=VS.95).aspx
// page 126ff, C. Petzold, Programming Windows Phone 7

// storing files/directory structures see:
// http://msdn.microsoft.com/en-us/library/system.io.isolatedstorage.isolatedstoragefile(v=VS.95).aspx

namespace PhoneStorage
{
    public partial class MainPage : PhoneApplicationPage
    {
        // Constructor
        public MainPage()
        {
            InitializeComponent();
        }

        #region Save and Load Parameters from the Application Storage
        void saveToAppStorage(String ParameterName, String ParameterValue)
        {
            // use mySettings to access the Apps Storage
            IsolatedStorageSettings mySettings = IsolatedStorageSettings.ApplicationSettings;

            // check if the paramter is already stored
            if (mySettings.Contains(ParameterName))
            {
                // if parameter exists write the new value
                mySettings[ParameterName] = ParameterValue;
            }
            else
            {
                // if parameter does not exist create it
                mySettings.Add(ParameterName, ParameterValue);
            }
        }

        String loadFromAppStorage(String ParameterName)
        {
            String returnValue = "_notSet_";
            // use mySettings to access the Apps Storage
            IsolatedStorageSettings mySettings = IsolatedStorageSettings.ApplicationSettings;

            // check if the paramter exists
            if (mySettings.Contains(ParameterName))
            {
                // if parameter exists write the new value
                mySettings.TryGetValue<String>(ParameterName, out returnValue);
                // alternatively the following statement can be used:
                // returnValue = (String)mySettings[ParameterName];
            }

            return returnValue;
        }
        #endregion

        private void button1_Click(object sender, RoutedEventArgs e)
        {
            saveToAppStorage("myIdentifer1", "Last used @ " + System.DateTime.Now.ToString("HH:mm:ss"));
            textBox1.Text = "saved...";
        }

        private void button2_Click(object sender, RoutedEventArgs e)
        {
            textBox1.Text = loadFromAppStorage("myIdentifer1");
        }

    }
}

Online German Language Corpus, UCREL Summer School

At the University of Leipzig a German Language corpus is available (Projekt Deutscher Wortschatz). The database can be queried from different programming languages and access is also possible via a web service. Requests can ask for co-occurrences of words, base forms, about words that often occur to the right and to the left of the word, word frequency, synonyms and much more. If you develop text input systems this may be a very useful resource, see the web services overview page (with links to downloads), the list of web-service-requests offered or have a look at some php-examples.

You can try the service interactively at http://wortschatz.uni-leipzig.de/abfrage/. See the pictures for an example query on the term Internet. They also feature a German-English dictionary.

Since I shared and office at Lancaster University with Paul Rayson from UCREL (University Centre for Computer Corpus Research on Language) I find corpus linguistics an interesting topic. By the way UCREL runs a Summer School in Corpus Linguistics from 13 to 15 July 2011 - would love to go there...

WP7 Tutorial - part 3: Using Location

In this example the use of the location API is demonstrated. The API is a high level interface to geo location. How the location is determined (e.g. GPS, GSM cell information) is of no concern to the developer.

The basic approach is to create an instance of GeoCoordinateWatcher and register two callback functions: one for when the status changes and one for when the location changes. The program demonstrates how these call backs are set up and how from within those function the user interface is updated with the received information. If the status is changes, the program checks what the current status is, and shows this in the status line (textBlock8.Text). If the position is changed then the new position information (Position.Location.Longitude, Position.Location.Latitude) - and additional information such as Speed, Altitude, Course, Accuracy are shown.

As an exercise you can build an application that shows you how close you are to a given target. In two input fields you enter the longitude and latitude of the destination (e.g. a geo cache location). And then you can calculate the difference from the current position to the target location and visualize or sonify the distance.

There is another example (Geo coordinate watcher) how to use this API on the Microsoft msdn website. In C. Petzold's book there is also a good example, see page 91ff.

See below the c# example using geo location on a windows phone 7. You can also download the geolocation project directory in a single ZIP-file.

using System;
using System.Collections.Generic;
using System.Windows;
using Microsoft.Phone.Controls;
using System.Device;
using System.Device.Location;

// the example shows the basic functionality of the location device
// you need to add in the solution explorer a reference to System.Device
// right click on References in the solution explorer, click Add Reference, and then
// System.Device
// Albrecht Schmidt, University of Stuttgart

// for a more comprehensive example see: 
// http://msdn.microsoft.com/en-us/library/system.device.location.geocoordinatewatcher.aspx
// http://msdn.microsoft.com/en-us/library/ff431744(v=vs.92).aspx
// and page 91ff, C. Petzold, Programming Windows Phone 7

namespace Geo_Location
{
    public partial class MainPage : PhoneApplicationPage
    {
        GeoCoordinateWatcher watcher;

        // Constructor
        public MainPage()
        {
            InitializeComponent();
        }

        // the initialize and start button is pressed
        private void button1_Click(object sender, RoutedEventArgs e)
        {
            // initialize the geo watcher with defaul accuracy (battery saving)
            // user GeoPositionAccuracy.High for higher accuracy
            watcher = new GeoCoordinateWatcher(GeoPositionAccuracy.Default);
            // set movement threhold - as distance in meters - default is 0
            watcher.MovementThreshold = 10;

            // add a handler that is called when position is changed more than MovementThreshold
            watcher.PositionChanged += new EventHandler<GeoPositionChangedEventArgs<GeoCoordinate>>(watcher_PositionChanged);
            // a handler for status change
            watcher.StatusChanged += new EventHandler<GeoPositionStatusChangedEventArgs>(watcher_StatusChanged);
            
            // Start reading location data
            watcher.Start();
        }

        void watcher_StatusChanged(object sender, GeoPositionStatusChangedEventArgs e)
        {
            // you cannot change the UI in this function -> you have to call the UI Thread
            Deployment.Current.Dispatcher.BeginInvoke(() => ChangeStatusUI(e));
        }

        void ChangeStatusUI(GeoPositionStatusChangedEventArgs e)
        {
            String statusType="";
            if ((e.Status) == GeoPositionStatus.Disabled)
            {
                statusType = "GeoPositionStatus.Disabled";
            }
            if ((e.Status) == GeoPositionStatus.Initializing)
            {
                statusType = "GeoPositionStatus.Initializing";
            }
            if ((e.Status) == GeoPositionStatus.NoData)
            {
                statusType = "GeoPositionStatus.NoData";
            }
            if ((e.Status) == GeoPositionStatus.Ready)
            {
                statusType = "GeoPositionStatus.Ready";
            }
            textBlock8.Text = statusType;
        }

        void watcher_PositionChanged(object sender, GeoPositionChangedEventArgs<GeoCoordinate> e)
        {
            // you cannot change the UI in this function -> you have to call the UI Thread
            Deployment.Current.Dispatcher.BeginInvoke(() => ChangeUI(e));
        }

        void ChangeUI(GeoPositionChangedEventArgs<GeoCoordinate> e)
        {
            textBlock1.Text = "Longitute: " + e.Position.Location.Longitude;
            textBlock2.Text = "Latitute:  " + e.Position.Location.Latitude;
            textBlock3.Text = "Speed:     " + e.Position.Location.Speed;
            textBlock4.Text = "Altitude:  " + e.Position.Location.Altitude;
            textBlock5.Text = "Course:  " + e.Position.Location.Course;
            textBlock6.Text = "Vertical Accuracy:  " + e.Position.Location.VerticalAccuracy;
            textBlock7.Text = "Horizontal Accuracy:  " + e.Position.Location.HorizontalAccuracy;
            textBlock8.Text = "location updated at " + System.DateTime.Now.ToString("HH:mm:ss");
        }

        // the stop button clicked ... stop the watcher
        private void button2_Click(object sender, RoutedEventArgs e)
        {
            if (watcher != null) { watcher.Stop(); }
            textBlock8.Text = "location reading stopped";
        }
    }
}

Percom 2011 in Seattle, keynote

This year's Percom conference was held in Seattle and offered an exciting and diverse program. Have a look the program to see it for yourself. The two keynotes were both looking at the implications of pervasive computing and communication - especially when thinking about the data is collected and how the data may be used.

Alex Pentland from MIT talked about their work on reality mining. The work looks at how one can capture interactions between people and between people and their environment and how such information can be exploited. One example he gave was on looking at the effect of face to face communication on the performance on workers. The basics insights of this work are thrilling and thinking it through it becomes obvious that we are at the start of new era of mankind. The arguments he made that we can contain and control such information I did not find convincing and I think it may be dangerous to tell decision makers in politics that we can provide solutions. I see no way (that is not restricting people's freedom massively or which reduces productivity massively) that would allow to control the information that will become available through pervasive computing… and all the solutions I have heard either will plainly not work or would require a global agreement over data protection laws…

The keynote on the second day was by Derek McAuley from Nottingham University. One of his topics was on product history and how the availability of product history has the potential to increase the value of products. I think this is a very powerful concept and we will in the near future see this commercially exploited.
Furthermore Dereck discussed interesting issues that come up with crowd sourcing and participatory sensing. One central point is where the data is hold and who controls the data collected. Especially in the context of cloud services this becomes transparent and important at the same time. With regard to the implementation is does not matter; however from a legal perspective it may make a serious difference whether you cloud service runs in German, the US, or on a ship somewhere in the Atlantic. An example he gave are navigation systems in cars which have a back channel. The cars sent back information about their speed and whereabouts and the information is used to predict the state of the road, which is then used to improve the navigation. He raised the questions what happens if this information is held somewhere were legislation has no control? I think this is going to happen and there is no real approach against it…
He made a case that end-users (individuals) should be able to bring together information about them and make use of it. On principle I like this idea to put the individual into control and allow them to exploit this data. For me this is however not a solution for data protection, as a certain part of individuals will sell their data - and in a free country there is probably very little society can do against it.

In summary - we are heading towards an exciting future!

PS: Percom 2012 will be in Lugano with Silvia and Marc chairing the conference. And I have the honor to serve as program chair. See the web page for more information (will be available soon) or the photo of the call for papers here.

WP7 Tutorial - part 2: Vibration

This examples shows how to activate the vibration motor / vibration actuator in the phone. The calls Microsoft.Devices.VibrateController.Default.Start and Microsoft.Devices.VibrateController.Default.Stop are used to switch the actuator on and off.
When switching the vibration on the parameter sets the duration for which it should be on. The duration is between 0 and 5 seconds. With the function TimeSpan.FromMilliseconds(duration), where duration is a number, the parameter can be set easily.

The standard API only supports to switch on and off the vibration. We experimenting with the code you can explore how to have vibrations of different intensity. To do this you have to switch on and off the vibration (e.g. 100 ms on then 50 ms off) - basically doing pulse-width modulation.

There is more information on the vibration controller on the Microsoft site.

See below the c# example for controling the vibration a windows phone 7.
You can also download the vibration project directory in a single ZIP-file.

using System;
using System.Windows;
using Microsoft.Phone.Controls;

// example of how switch on the vibration motor for a given time
// another call to switch it off
// Albrecht Schmidt, University of Stuttgart

// see:
// http://msdn.microsoft.com/en-us/library/microsoft.devices.vibratecontroller.default(v=VS.92).aspx


namespace Vibration
{
 public partial class MainPage : PhoneApplicationPage
 {
     // Constructor
     public MainPage()
     {
         InitializeComponent();
     }

     private void button1_Click(object sender, RoutedEventArgs e)
     {
         double duration;
         duration = Convert.ToDouble(textBox1.Text);
         if (duration > 5000)
         {
             duration = 5000;
         }
         // starts the vibrations (valid duration are between 0 and 5 seconds)
         Microsoft.Devices.VibrateController.Default.Start(TimeSpan.FromMilliseconds(duration));
     }

     private void button2_Click(object sender, RoutedEventArgs e)
     {
         Microsoft.Devices.VibrateController.Default.Stop();
     }
 }
}

WP7 Tutorial - part 1: Phone calls and SMS

Over the next weeks I like to share some examples I have created while learning to program the Windows Phone 7 platform. The tutorial is mainly explaining some of the APIs and components I found interesting (in particular related to context-awareness and human computer interaction).

If you are really new to programming on this platform the App-Hub is probably a good place to start. There are also plenty of helpful examples on the Microsoft web page. As prerequisite for this tutorial I assume that you have successfully installed Visual Studio (Express), the windows phone development tools, and that you have managed to get your first "Hello World" written, compiled, and deployed. There is also plenty of material on youtube that helps to get started. The Programming Windows Phone 7 book by Charles Petzold (free available as PDF) is also a good starting point.

Several examples that follow in the tutorial (e.g. vibration, accelerometer) having a real phone to test the programs is highly recommended.

The Windows Phone 7 is a phone and hence I start with a program that makes use of the basic phone functionality: making a phone call and sending an SMS.

This example also highlights the approach taken in several of the APIs. Your program (a third party application) is restricted to transfers control to the phones basic functions when making a call or sending an SMS. Basically the API call opens the phone/SMS applications with the parameters you hand over. From a security point of view this is nice as an application can not send SMS or do phone calls without the user recognizing (and agreeing to) it. From a programming perspective this has disadvantages as automation of functionality (e.g. always sent an SMS when I am coming closer than 10 km to home) is not possible.

The example demonstrates how to access interactively the phone book (using the PhoneNumberChooserTask), how to initiate a phone call task (using Microsoft.Phone.Tasks.PhoneCallTask), and how to initiate a SMS task (using Microsoft.Phone.Tasks.SmsComposeTask).
The PhoneNumberChooserTask is an example of a callback - used a lot when programming for WP7. The basic concept is to register a function that is called when an event happens. In this case the event is that the number is chosen and then the function myPhoneNumberChooser_Completed is called. The nice thing with Visual Studio is that you do not have to type this (or remember the syntax) - just use the TAB-key after you typed +=

See below the c# example for sending SMS and making a phone call. You can also download the Phone/SMS project directory in a single ZIP-file.

using System;
using System.Collections.Generic;
using System.Windows;
using System.Windows.Controls;
using Microsoft.Phone.Controls;
using Microsoft.Phone.Tasks;

// example how to invoke a phone call and an SMS from a program
// the basic approach is that you can set the phone number (and
// for SMS the message body) and then you call the phone/sms
// application
// Albrecht Schmidt, University of Stuttgart
// more on how to invoke phone tasks (photo, email, ...)
// http://msdn.microsoft.com/en-us/library/ff769543(v=vs.92).aspx

namespace Phone_Call_and_SMS
{
public partial class MainPage : PhoneApplicationPage
{
  // Constructor
  public MainPage()
  {
      InitializeComponent();
  }

  // choose a phone number
  private void button1_Click(object sender, RoutedEventArgs e)
  {
      PhoneNumberChooserTask myPhoneNumberChooser;
      myPhoneNumberChooser = new PhoneNumberChooserTask();
      myPhoneNumberChooser.Completed += new EventHandler<PhoneNumberResult>(myPhoneNumberChooser_Completed);
      myPhoneNumberChooser.Show();
  }

  // call back after the phone number is choosen
  void myPhoneNumberChooser_Completed(object sender, PhoneNumberResult e)
  {
      if (e.TaskResult == TaskResult.OK)
      {
          textBox1.Text = e.PhoneNumber.ToString();
      }
  }

  // making a phone call (opening the sms application with presets)
  private void button2_Click(object sender, RoutedEventArgs e)
  {
      Microsoft.Phone.Tasks.PhoneCallTask phonecall = new Microsoft.Phone.Tasks.PhoneCallTask();
      phonecall.PhoneNumber = textBox1.Text; // set phone number
      phonecall.Show();
  }

  // sending an sms (opening the sms application with presets)
  private void button3_Click(object sender, RoutedEventArgs e)
  {
      Microsoft.Phone.Tasks.SmsComposeTask sms = new Microsoft.Phone.Tasks.SmsComposeTask();
      sms.To = textBox1.Text; // set phone number
      sms.Body = textBox2.Text; // set body
      sms.Show();
  }
}
}

Source code formating in blogspot

I am putting together a Windows Phone 7 tutorial ... and then I came across the issue of how to include code in the blog posts. Here is the solution.

First get the code nicely formatted - the following tools may help
http://jsbeautifier.org/
http://www.prettyprinter.de/

Then include the prettyprint function into your Blogspot template. The following Link show how:
http://lukabloga.blogspot.com/2008/10/to-test-new-highlighting.html

You have alter the template you can use the following command to include code in the HTML code with the pre-tag and the class=prettyprint.

Alternatively you can convert source code into HTML:
http://www.tohtml.com:
http://www.neathighlighter.com/

more links at
http://techbrij.com/3/code-syntax-highlighting-for-a-free-hosted-blogger-blog

Visualization - Tokyo Underground Map

I spotted this visualization of an underground line in Tokyo and thought that I can use this as an interesting introductory example in information visualization. The very simple question is: what does this visualization communicate to the user? The bi-lingual labeled picture is quite rich in the way it communicates the information. Take as one example the visualization of "Where am I?" and "Where is the train going to?". This is visualized in multiple and redundant ways: (1) the station where the user is highlighted with a red background color, (2) the visualization of the underground line has a break at the current station, (3) the stations the train has already passed are lighter colored (pink) as the stations to come, and (4) there is an arrow showing the direction.

Overall I think this is a good example of how to efficiently communicate such information. But be careful if you travel … there are also much more confusing signs around here.

How will computing change the world? Our view in Computing Now.

Together with Marc Langheinrich and Kristian Kersting we wrote an article on how computing is going to change our world [1] and featured in Computing Now. We discuss how upcoming technologies will change the our perception. Besides others we make the bold statement "By the middle of this century, the boundaries between direct and remote perception will become blurred".


We discuss how our perception is extended and augmented by technical means and how this will eventually lead to a new augmented sense of ubiquitous perception. We expect this will radically change the way we live and hence ethical considerations are central. We make the argument that ethics become a major design factor. We are looking forward to feedback on this vision - even if you disagree.

[1] Albrecht Schmidt, Marc Langheinrich, Kritian Kersting, "Perception beyond the Here and Now," Computer, vol. 44, no. 2, pp. 86-88, Feb. 2011, doi:10.1109/MC.2011.54 (PDF)

Lukowicz Consumer Confidence Index

Paul Lukowicz taked in the symposium on Interaction with Smart Artifacts in Tokyo about "Large scale, context aware socio-technical systems". As a number of other talks in our symposium he also pointed out that value of the massive amounts of data that become available by mobile sensing and crowd interaction.

Paul gave one example how such sensing data could be used. I think this is really interesting that is why I share here the Paul "Lukowicz Consumer Confidence Index" (CCIL) which is based on detecting the whereabouts of people. It assumes that we can estimate the number of people shopping at various places based on the use of their mobile devices.

Interesting touch technology: PhotoelasticTouch

Hideki Koike talked at the Symposium on Interaction with Smart Artifacts in Tokyo. One of the examples he showed is the PhotoelasticTouch technology. The idea is to use a transparent gel and the effect of photoelasticity to detect interaction on a surface, see [1] for details. [2] give a brief overview (the picture below is taken from this publication).
There is a nice youtube videp the shows the concept very well: http://www.youtube.com/watch?v=RJaEg0xB98E

[1] Toshiki Sato, Haruko Mamiya, Hideki Koike, and Kentaro Fukuchi. 2009. PhotoelasticTouch: transparent rubbery tangible interface using an LCD and photoelasticity. In Proceedings of the 22nd annual ACM symposium on User interface software and technology (UIST '09). ACM, New York, NY, USA, 43-50. DOI=10.1145/1622176.1622185 http://doi.acm.org/10.1145/1622176.1622185

[2] One page abstract: http://www.vogue.is.uec.ac.jp/PET/photoelasticTouch1Page.pdf

Lab Tour on arrival in Tokyo

After a long day/night of travelling I arrived in Tokyo. Arriving at the hotel we were met by our Japanese Colleague Yoshito Tobe and guided us to a Lab tour in the afternoon. We went by train to the Morikawa Labs in the Komaba Research Campus at the University of Tokyo.

At the lab students from different group showed us their work and discussed their ideas with us. To mention only a few things I got to try out a tutoring system for Japanese calligraphy, we saw prototypes for phone based urban sensing, and saw an implementation of a system that communicates between two devices using accelerometers and vibration motors [1].

After the tour we went up a tower building to look at the sunset above Tokyo and we even had a view on Mount Fuji. And as the observation floor is a touristy place there are all sorts of interesting things - and I operated a nice machine to get a puri-CUBE.

Some more photos are available at: http://tinyurl.com/LabTok11 (facebook account required) or public on flickr.


[1] vib-connect: A Device Selecting Interface Using Vibration by Hiroshi Nakahara et al. Demo at IOT 2010.

Keyboard I/O hack

During my research visit at Microsoft Research in Cambridge with Steve Hodges group I spend a lot of time building things and programming (Windows Phone 7, C#, microcontroller...) - which I really enjoyed!

In this post I want to share a mini project, I did together with Nic Villar between two coffees … it reminded me somehow of the Friday afternoon projects (=2h projects you do because they are fun and sometimes you learn something) while I did my PhD.

Perhaps some context first: working on Microsoft .NET Gadegteer we discussed many ideas of how to ease code/system creation for physical and embedded computing. Looking into it I came across a number of interesting approaches for software development that runs in the browser. Examples include: http://ideone.com/ (a online compiler for a large number of languages), http://wonderfl.net/ is a online development environment for Flash that allows to build on other people's code by forking their project, and http://mbed.org/ is a online tool for rapid prototyping of a microcontroller system.

Now to the mini project: a keyboard hack that supports input as well as output. You find many example of keyboard hacks on the web - basically you take out the PCB of the keyboard an replace the keys with something more interesting… here is ours that supports input (which is common) as well as output - from the computer (or web browser) to the environment. Using AQW210EH allows more flexibility in what you drive and what you create the inputs with. The attractive thing in using a keyboard is that it will work without a driver - you just plug it in and it works.

To control it I wrote a java applet and some JavaScript code. Basically the java-applet is controlling the Scroll-Lock LED and from the website this is called via JavaScript (try here if it works on your computer - if it works your CAPS, NUM, and SCROLL LED will blink once the web page is open). I tried it on some Windows machines and it worked well - the java applet seems not to work on a Mac (so you probably have to find another way to set you Scroll Lock LED programmatically). You can extend the output to CAPSLOCK and NUMLOCK - but these are sometimes used - in contrast to the SCROLL LOCK. Input is simple - by connecting one of the rows to one of the colloms you generate a letter comes in as if you would type it. The nice thing with USB is that you can have multiple keyboards connected at the same time (however the LEDs are synchronized between them).

In C# you can use the SendKeys.Send("{SCROLLLOCK}") command (for details see: http://msdn.microsoft.com/en-us/library/system.windows.forms.sendkeys.send.aspx,
http://support.microsoft.com/Default.aspx?id=177674) The SendKeys.Send may only give an pulse to the LED and hence you may need to call a system function.

Using a JavaApplet it looks like this.

import java.applet.*;
import java.awt.*;
import java.awt.Toolkit;
import java.awt.event.KeyEvent;
public class keyLed extends Applet {
   Font f = new Font("TimesRoman",Font.BOLD,20);
   String strMessage;
   Toolkit tk = Toolkit.getDefaultToolkit();
   public void init() {
       setBackground(Color.white);
       setStrMessage("Applet loaded.");
   }
   public void paint(Graphics g) {
       g.setFont(f);
       g.drawString(this.strMessage, 100 , 25);
   }
   public void ScrollLedOn() {
       tk.setLockingKeyState(KeyEvent.VK_SCROLL_LOCK,true);
       setStrMessage("Scroll on");
   }
   public void ScrollLedOff() {
       tk.setLockingKeyState(KeyEvent.VK_SCROLL_LOCK,false);
       setStrMessage("Scroll off");
   }
}

And in HTML/JavaScript you call it:

<APPLET CODEBASE="." CODE="keyLed.class" WIDTH=300 HEIGHT=50 NAME="keyLedApplet">
</APPLET>
<script language="JavaScript" type="text/javascript"> 
function LEDOn()
{
document.keyLedApplet.ScrollLedOn();
}
 
function LEDOff()
{
document.keyLedApplet.ScrollLedOff();
}
 </script>