Sunday 17 April 2011

Floor for activity recognition

Patrick Baudisch showed in Paris at the Microsoft Software summit interesting photos of their DIY activity in creating a glass floor for tracking and activity recognition. With a fairly large glass pane in the floor they have created an interesting environment… I am sure there will be interesting things coming out of this installation.

Some years back in 2002 (and looking at the photos and the amount of hair I still had this seems long ago) in Lancaster we also looked in what to do with floors (and we were into DIY as well). We also considered arrangements with a floor tables and furniture on top. As you can see from Kristof, Hans and me on the photo it was a fun project.

The positive point in using load sensing is that you can track unobtrusive and potentially large scales with little instrumentation. We even considered the possibility to put a house on 4 load cells and do activity recognition based on this. We never got around to building the house ;-) The problem with load sensing is that you can only track one moving object/subject at the time.

Looking at the signature of the load measured and doing some signal processing we could detect events – unobtrusive and cheap – but only for single events.

Interested in more details? Have a look at the publications on load sensing [1], on the interaction [2], and at a patent [3] describing the basic technology.

[1] Schmidt, A., Strohbach, M., Laerhoven, K. v., Friday, A., and Gellersen, H. 2002. Context Acquisition Based on Load Sensing. In Proceedings of the 4th international Conference on Ubiquitous Computing (Göteborg, Sweden, September 29 - October 01, 2002). Springer-LNCS, London, 333-350.

[2] Schmidt, A.; Strohbach, M.; van Laerhoven, K. & Hans-W., G. 2003. Ubiquitous Interaction -  Using Surfaces in Everyday Environments as Pointing Devices, Universal Access Theoretical Perspectives, Practice, and Experience (UI4ALL 2003), Springer LNCS, 263-279.

[3] Schmidt, A., Strohbach, M., Van Laerhoven, K., Friday, A., Gellersen, H-W., Kubach, U.; Context acquisition based on load sensing. US Patent 7434459. US Patent Issued on October 14, SAP AG (DE), 2008

Monday 11 April 2011

WP7 Tutorial - part 5: Orientation and Acceleration - X,Y,Z

Detecting gestures, orientations, and movement can be realized with the accelerometer. The accelerometer is a sensor that measures the acceleration in 3 dimensions (X, Y, and Z). If the device is not moved the accelerations measure are the gravity forces in each direction. If the device is accelerated the measured results are a combination of the acceleration and the gravity.

The accelerometer data can be accessed via the AccelerometerReadingEventArgs class. The class has values for the X, Y, and Z axis. The values are of type double and between -2 and 2 which related to acceleration "for each axis in gravitational units" - 1 is the gravitational force of the earth. See: http://msdn.microsoft.com/en-us/library/ff431744(v=vs.92).aspx and http://msdn.microsoft.com/en-us/library/ff431810(v=vs.92).aspx or on page 80ff, C. Petzold, Programming Windows Phone 7.

A typical exercise for understanding the accelerometer is to create a bubble level (a tool to measure if something is horizontal or vertical - e.g. for hanging pictures on the wall). You probably want to freshen up on arctan2 - at least I needed ;-)

See below the c# example reading out the accelerometer on a windows phone 7. You can also download the accelerometer project directory in a single ZIP-file.

using System;
using System.Windows;
using Microsoft.Phone.Controls;
using Microsoft.Devices.Sensors;

// A simple example to read the accelerometer and display the values
// In order to make it work you have to add the refercerence to 
// Microsoft.Devices.Sensors to your project. To do this right-click
// in the Solution Explorer on References and than choose add Reference
// in the dialog then select Microsoft.Devices.Sensors
// Albrecht Schmidt, University of Stuttgart

// for a more comprehensive example see: 
// http://msdn.microsoft.com/en-us/library/ff431810(v=vs.92).aspx
// http://msdn.microsoft.com/en-us/library/ff431744(v=vs.92).aspx
// and page 80ff, C. Petzold, Programming Windows Phone 7

namespace Accl_X_Y_Z
{
    public partial class MainPage : PhoneApplicationPage
    {
        Accelerometer accelerometer;
    
        public MainPage()
        {
            InitializeComponent();
            // create a new instance
            accelerometer = new Accelerometer();
            // register a callback function for when values change
            accelerometer.ReadingChanged += new EventHandler<AccelerometerReadingEventArgs>(accelerometer_ReadingChanged);
            // start the accelerometer
            accelerometer.Start();
        }

        void accelerometer_ReadingChanged(object sender, AccelerometerReadingEventArgs e)
        {
            // required as from here the textBlocks cannot be accessed
            Deployment.Current.Dispatcher.BeginInvoke(() => ChangeUI(e));
        }
        
        void ChangeUI(AccelerometerReadingEventArgs e)
        {
            // show the values on the screen
            textBlock1.Text = "X: " + e.X.ToString("0.000");
            textBlock2.Text = "Y: " + e.Y.ToString("0.000");
            textBlock3.Text = "Z: " + e.Z.ToString("0.000");
        }
    }
}

WP7 Tutorial - part 4: Storing Data on the Phone

If you want to save high scores or preferences you need persistent memory on the phone. On a traditional computer you would create a file and store your information in the file; another option on a Windows PC would be store such information in the registry. For security reasons there is no API to access the file system and there is no global persistent memory across applications on a WP7.

In general there are two ways to store data: (1) in a application specific storage (isolated storage) on the phone or (2) remotely on the internet (or to use another buzzword "in the cloud").
In this example the use of the phone isolated storage API is demonstrated. It is shown how to store and retrieve name-value pairs on the phone (to people who programmed Java ME this is conceptually similar to the record store).

For more details see: http://msdn.microsoft.com/en-us/library/cc221360(v=VS.95).aspx and page 126ff, C. Petzold, Programming Windows Phone 7. It is also possible to created an isolated storage for files see: http://msdn.microsoft.com/en-us/library/system.io.isolatedstorage.isolatedstoragefile(v=VS.95).aspx

See below the c# example using the local storage on a windows phone 7. You can also download the IsolatedStorageSettings project directory in a single ZIP-file.

using System;
using System.Windows;
using Microsoft.Phone.Controls;
using System.IO.IsolatedStorage;

// example of how to save to and load from the isolated application storage 
// this helps to create persistence storage within a single application
// in this example it is shown how to do it for a string 
// Albrecht Schmidt, University of Stuttgart  


// For more details see:
// http://msdn.microsoft.com/en-us/library/cc221360(v=VS.95).aspx
// page 126ff, C. Petzold, Programming Windows Phone 7

// storing files/directory structures see:
// http://msdn.microsoft.com/en-us/library/system.io.isolatedstorage.isolatedstoragefile(v=VS.95).aspx

namespace PhoneStorage
{
    public partial class MainPage : PhoneApplicationPage
    {
        // Constructor
        public MainPage()
        {
            InitializeComponent();
        }

        #region Save and Load Parameters from the Application Storage
        void saveToAppStorage(String ParameterName, String ParameterValue)
        {
            // use mySettings to access the Apps Storage
            IsolatedStorageSettings mySettings = IsolatedStorageSettings.ApplicationSettings;

            // check if the paramter is already stored
            if (mySettings.Contains(ParameterName))
            {
                // if parameter exists write the new value
                mySettings[ParameterName] = ParameterValue;
            }
            else
            {
                // if parameter does not exist create it
                mySettings.Add(ParameterName, ParameterValue);
            }
        }

        String loadFromAppStorage(String ParameterName)
        {
            String returnValue = "_notSet_";
            // use mySettings to access the Apps Storage
            IsolatedStorageSettings mySettings = IsolatedStorageSettings.ApplicationSettings;

            // check if the paramter exists
            if (mySettings.Contains(ParameterName))
            {
                // if parameter exists write the new value
                mySettings.TryGetValue<String>(ParameterName, out returnValue);
                // alternatively the following statement can be used:
                // returnValue = (String)mySettings[ParameterName];
            }

            return returnValue;
        }
        #endregion

        private void button1_Click(object sender, RoutedEventArgs e)
        {
            saveToAppStorage("myIdentifer1", "Last used @ " + System.DateTime.Now.ToString("HH:mm:ss"));
            textBox1.Text = "saved...";
        }

        private void button2_Click(object sender, RoutedEventArgs e)
        {
            textBox1.Text = loadFromAppStorage("myIdentifer1");
        }

    }
}