Showing posts with label hardware. Show all posts
Showing posts with label hardware. Show all posts

Thursday, 3 March 2011

Keyboard I/O hack

During my research visit at Microsoft Research in Cambridge with Steve Hodges group I spend a lot of time building things and programming (Windows Phone 7, C#, microcontroller...) - which I really enjoyed!

In this post I want to share a mini project, I did together with Nic Villar between two coffees … it reminded me somehow of the Friday afternoon projects (=2h projects you do because they are fun and sometimes you learn something) while I did my PhD.

Perhaps some context first: working on Microsoft .NET Gadegteer we discussed many ideas of how to ease code/system creation for physical and embedded computing. Looking into it I came across a number of interesting approaches for software development that runs in the browser. Examples include: http://ideone.com/ (a online compiler for a large number of languages), http://wonderfl.net/ is a online development environment for Flash that allows to build on other people's code by forking their project, and http://mbed.org/ is a online tool for rapid prototyping of a microcontroller system.

Now to the mini project: a keyboard hack that supports input as well as output. You find many example of keyboard hacks on the web - basically you take out the PCB of the keyboard an replace the keys with something more interesting… here is ours that supports input (which is common) as well as output - from the computer (or web browser) to the environment. Using AQW210EH allows more flexibility in what you drive and what you create the inputs with. The attractive thing in using a keyboard is that it will work without a driver - you just plug it in and it works.

To control it I wrote a java applet and some JavaScript code. Basically the java-applet is controlling the Scroll-Lock LED and from the website this is called via JavaScript (try here if it works on your computer - if it works your CAPS, NUM, and SCROLL LED will blink once the web page is open). I tried it on some Windows machines and it worked well - the java applet seems not to work on a Mac (so you probably have to find another way to set you Scroll Lock LED programmatically). You can extend the output to CAPSLOCK and NUMLOCK - but these are sometimes used - in contrast to the SCROLL LOCK. Input is simple - by connecting one of the rows to one of the colloms you generate a letter comes in as if you would type it. The nice thing with USB is that you can have multiple keyboards connected at the same time (however the LEDs are synchronized between them).

In C# you can use the SendKeys.Send("{SCROLLLOCK}") command (for details see: http://msdn.microsoft.com/en-us/library/system.windows.forms.sendkeys.send.aspx,
http://support.microsoft.com/Default.aspx?id=177674) The SendKeys.Send may only give an pulse to the LED and hence you may need to call a system function.

Using a JavaApplet it looks like this.

import java.applet.*;
import java.awt.*;
import java.awt.Toolkit;
import java.awt.event.KeyEvent;
public class keyLed extends Applet {
Font f = new Font("TimesRoman",Font.BOLD,20);
String strMessage;
Toolkit tk = Toolkit.getDefaultToolkit();
public void init() {
setBackground(Color.white);
setStrMessage("Applet loaded.");
}
public void paint(Graphics g) {
g.setFont(f);
g.drawString(this.strMessage, 100 , 25);
}
public void ScrollLedOn() {
tk.setLockingKeyState(KeyEvent.VK_SCROLL_LOCK,true);
setStrMessage("Scroll on");
}
public void ScrollLedOff() {
tk.setLockingKeyState(KeyEvent.VK_SCROLL_LOCK,false);
setStrMessage("Scroll off");
}
}



And in HTML/JavaScript you call it:
<APPLET CODEBASE="." CODE="keyLed.class" WIDTH=300 HEIGHT=50 NAME="keyLedApplet">
</APPLET>
<script language="JavaScript" type="text/javascript"> 
function LEDOn()
{
document.keyLedApplet.ScrollLedOn();
}
 
function LEDOff()
{
document.keyLedApplet.ScrollLedOff();
}
 </script>

Tuesday, 25 January 2011

TEI Studio, making devices with Microsoft Gadgeteer

Nic Villar and James Scott from Microsoft Research in Cambridge, UK, run a Studio on building interactive devices at TEI 2011 in Madeira. Studio's are mixture of tutorials and hands-on workshops and there were very interesting ones at this year's conference - it was very hard to choose.

Microsoft Gadgeteer is a modular platform for creating devices. It includes a main board, displays, a camera, sensors, interaction elements, motors and servos, memory board, and USB-connectors as well as power supply options. It is programmable in C# using Visual Studio. The development involves the connecting up of the hardware and the writing of the software. A "hello world" example is a simple digital camera. You connect up the main board with the camera, a display, and a button. Writing less than 20 lines of code you implement the functionality of the camera. It follows an asynchronous approach. When the buttons is pressed the camera is instructed to take a picture. An event handle is registered that is called when the camera is ready capturing the image. And in the code in the handler you take the image and show it on the display. 10 years back we worked on a European project Smart-Its [1] and started with basic electronic building blocks and the idea of open source hardware where people could create their devices. Looking now at a platform like Gadgeteer one can see that there has been great progress in Ubicomp research over that time. It is great to see how Nic Villar has been pursuing this idea from the time he did his BSc thesis with Smart-Its in our lab in Lancaster, with Pin-and-Play/Voodoo I/O [2] in his PhD work, to the platform he is currently working on and the vision beyond [3].

It was amazing how quickly all groups got this examples working. For me, with programming experience - but no real experience in C# and with little practice in programming over the last few years - it was surprising how quickly I got into programming again. To me this was mainly due to the integration with visual studio - especially the suggestions made by the IDE were very helpful. James said the design rational for the Visual Studio integration was "you should never see just a blinking cursor" - and I think this was well executed.

All groups made over the day a design. I liked the "robot-face" that showed minimal emotions… basically if you come to close you can see it getting unhappy. My mini-project was a camera that use an ultrasonic range sensor and a larger display. It takes photos when someone comes close and shows the last 8 photos on the display - overwriting the first one when it comes to the end of the screen. Interested in how little code is required to do this? Check out the source code I wrote.


Two more studios I would have loved to attend: Amanda run a Studio on creating novel (and potentially bizarre) game controllers and Daniela offered a Studio to explore what happens if bookbinding meets electronics.

[1] Holmquist, L. E., Gellersen, H., Kortuem, G., Schmidt, A., Strohbach, M., Antifakos, S., Michahelles, F., Schiele, B., Beigl, M., and Mazé, R. 2004. Building Intelligent Environments with Smart-Its. IEEE Comput. Graph. Appl. 24, 1 (Jan. 2004), 56-64. DOI= http://dx.doi.org/10.1109/MCG.2004.1255810

[2] Van Laerhoven, K., Villar, N., Schmidt, A., Gellersen, H., Håkansson, M., Holmquist, L. E. 2003. In-Home Networking - Pin&Play: The Surface as Network Medium, IEEE Communications Magazine, vol. 41, no. 4, April 2003.

[3] Steve Hodges and Nicolas Villar, The Hardware Is Not a Given, in IEEE Computer, IEEE, August 2010

Wednesday, 10 November 2010

PARC - touching computing history

At PARC I had the chance to talk to people about some of our current projects. Les Nelson has done interesting work on public displays [1]. This work is highly relevant to ideas we pursue in the pdnet project and it was great to get a first person view from the researchers involved.

Being at PARC history of computing is all around you! Seeing the original Ethernet cable, tapes from Alan Kay or Lucy Suchman, the Alto computer, one of the original laser printer, and different Ubicomp artifacts from Mark Weiser's group really makes you feel that this is a special place for anyone interested in personal computing and ubicomp.

[1] Elizabeth F. Churchill, Les Nelson, and Gary Hsieh. 2006. Cafe life in the digital age: augmenting information flow in a cafe;-work-entertainment space. In CHI '06 extended abstracts on Human factors in computing systems (CHI '06). ACM, New York, NY, USA, 123-128. DOI=10.1145/1125451.1125481 http://doi.acm.org/10.1145/1125451.1125481

Saturday, 28 August 2010

Lab visit in Chengdu, University of Electronic and Science Technology of China


On the final day of the Sino-German Symposium on Wearable Computing in Chengdu Prof. Dongyi Chen invited us to see his lab. We drove to the new campus of the University of Electronic and Science Technology of China. Already the drive was impressive seeing the amount of building work happening in Chengdu, especially in the high tech area.

In the School of Computer Science and Engineering we visited a computer lab and got to see very interesting student projects in Prof. Dongyi Chen labs. The demos included a wireless controlled vehicle were the control is implemented on a mobile phone, industrial settings control applications using sensor nets, table top user interfaces and augmented reality applications on the table, different applications for wearable displays, and a wrist worn computer (developed from scratch).

The quality of the work by the students is impressive and so is the university campus (building and facilities). It shows a very clear determination to push science and education. We should probably talk to our government to consider investing more in research and higher education…
I hope this symposium will help us to start some more collaboration. As a next step we plan a summer school on Human Computer Interaction next year in Germany.

Friday, 12 March 2010

Invited Talk by Nicolas Villar

Nicolas visited us in Essen to give the opening talk of our German meeting on tangible interaction. In his talk he first showed some examples of the hardware and sensors group at Microsoft research in Cambridge, most notably the SenseCam (which we learned is licensed and will be soon commercially available).

In the main part of the talk Nicolas presented a modular embedded architecture that allows developers to create custom made digital systems with fairly little effort. By integrating physical development (3D printing), functional blocks and software development the approach aims at empowering developers to create entirely new devices. His examples were impressive, e.g. creating a fully functional game console in a few hours.

Assuming that electronics become really small and cheap and that displays can be directly printed I can see that this approach makes a lot of sense - the question is just how long will it take before we rather use a (nowadays) powerful ARM processor, instead of a logic circuit with 10 gates. I would imagine that from a economic perspective it will less than 20 years before this makes sense.

We talked about energy harvesting and hier is a link to a potential interesting component: LTC3108.

Thursday, 14 May 2009

Rubber-like stretchable display

Jörg just sent me a link on a rubber-like stretchable display that is published in Nature Material. There is a previous press release with some photos [2]. This is a significant step towards new nteractive devices, such as the one suggested in the GUMMI project [3].

[1] Stretchable active-matrix organic light-emitting diode display using printable elastic conductors, Tsuyoshi Sekitani et al., Nature Materials, doi: 10.1038/nmat2459
http://www.nature.com/nmat/journal/vaop/ncurrent/abs/nmat2459.html

[2] http://www.ntech.t.u-tokyo.ac.jp/Archive/Archive_press_release/press_stretchable/documents/press_release_en.pdf

[3] Schwesig, C., Poupyrev, I., and Mori, E. 2004. Gummi: a bendable computer. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Vienna, Austria, April 24 - 29, 2004). CHI '04. ACM, New York, NY, 263-270. DOI= http://doi.acm.org/10.1145/985692.985726

Tuesday, 13 January 2009

Mechanical Computing, Beauty of Calculating Machines

Instead of covering the history of calculating machines in the DSD lecture, we took the train and went to the Arithmeum in Bonn to the see the artefacts live and to play with some of them.



We started with early means for counting and record keeping. The tokens and early writings did not use numbers as abstract concepts, rather as representatives of concrete objects – this is very inspiring, especially from a tangible interaction point of view. The knots, as used in south America, show impressively how the tools for calculation have to fit the context people live in. Interestingly all these artefacts highliht how the ability to calculate and store information is related to the ability to do trade – quite a good motivation for the setup we have in Essen business studies and computer science within one faculty.


I was again impressed by the ingenuity by the early inventers of calculating tools and machines. There is an interesting separation between calculating tools and machines – the first ones require the user to take care of the carry and the second do it by themselves. We tried out replicas of Napier’s calculating tool and Schickard’s calculating machine.


The beauty and the mechanical precision required of those early machines is impressive. These prototypes (most of them took years and massive funds to be built complete) can teach us something for research today. These inventors had visions and the will to get it implemented, even without a clear application or business model in mind. They were excited by the creating of systems than can do things, machines could not do before. From the professions of the inventors (e.g. Philipp Matthäus Hahn was a clergyman)  it becomes apparent that at these times some considered religion and calculation as closely related - which to mondern understanding is very very alien.


Seeing the Hollerith machine that was used for the US census more than 100 years ago can teach you a lot about data processing. Punch cards, electrical reading and electrical counters (using mainly relays) were the basis for this technology. Looking at the labels on the counters showed that the US has a long tradition in collecting data that is after some time is not seen as political correct ;-)


Having learned binary calculations during the DSD course it was nice to see a machine that did binary additions, using small steel balls and gravity. On each place (1,2,4,8, …) there is space for one ball. If a second one comes to this place one moves up to the next place (carry) and one is discarded. This is implemented with very simple mechanics and the working prototype (recently build) is based on designs of Schickard (but he never built - if I am correct).


Moving on with binary systems and finally to silicon, we got to see the Busicom 141 - a desk calculator that uses the Intel 4004. It is impressive to see that this is not even 40 years ago – starting with 2300 transistors and 180kHz. 

you can find the full set of photos at: http://foto.ubisys.org/dsd0809/

Tuesday, 16 December 2008

One Ez430-F2013 for each student in DSD

This term we teach digital system design and besides the essential (gates, flip-flops, 2-complements, alu, data path, etc) we decided to include some practical parts. In the first part we introduced Verilog. We did the exercises with Icarus Verilog (free, text-based verilog) and there are more powerful tools available, e.g. ISE WebPACK.

In the second part we have practical exercises in assembly language using the Ez430-F2013 development kit. We borrowed one kit to each student (just in case someone is bored over Christmas) and the current task is to complete and assembly program (incomplete sample) that the LED shows repeatedly "Hello" in Morse code. This is just the start - perhaps we do some more interesting stuff in January. The development kit is really interesting - especially given the fact that it is only 20€ (including hardware, compiler, and IDE). The MCU is small - but still good enought to generate a video signal (eagle CAD files, some assembly code). 

Friday, 6 June 2008

Is it easier to design for touch screens if you have poor UI designers?

Flying back from Sydney with Qantas and now flying to Seattle with Lufthansa I had to long distance flights in which I had the opportunity to study (n=1, subject=me, plus over-shoulder-observation-while-walking-up-and-down-the-aisle ;-) the user interface for the in-flight entertainment.

The 2 systems have very different hardware and software designs. The Qantas infotainment system is a regular screen and interaction is done via a wired moveable remote control store in the armrest. The Lufthansa system uses a touch screen (It also has some hard buttons for volume in the armrest). Overall the content on the Qantas system comprised of more content (more movies, more TV-shows) including real games.

The Qantas system seemed very well engineered and the remote control UI worked was greatly suited for playing games. Nevertheless the basic operation (selecting movies etc.) seemed more difficult using the remote control compared to the touch screen interface. In contrast the Lufthansa system seems to have much room for improvement (button size, button arrangement, reactions times of the system) but it appeared very easy to use.

So here are my hypotheses:

Hypothesis 1: if you design (public) information or edutainment systems (excluding games) using a touch screen is a better choice than using an off-screen input device.

Hypothesis 2: with UI design team of a given ability (even a bad UI design team) you will create a significantly better information and edutainment systems (excluding games) if you use a touch screen than using an off-screen input device.

From the automotive domain we have some indications that good off-screen input device are really hard to design so that they work well (e.g. in-build-car navigation system). Probably I should find a student to proof it (with n much larger than 1 and other subjects than me).

PS: the Lufthansa in-flight entertainment runs on Windows-CE 5.0 (the person in front of me had mainly the empty desktop with the Win CE logo showing) and it boots over network (takes over 6 minutes).

Friday, 4 April 2008

Work on our new lab space started – ideas for intelligent building material

This week work on our new lab space started :-) With all the drilling and hammering leaving for CHI in Florence seemed like perfect timing. Our rooms are located in a listed historical building and hence planning is always a little bit more complicated but we are compensated by working in a really nice building.

As I was involved in the planning space for the lab we had the opportunity to integrate a space dedicated to large interactive surfaces where we can explore different options for interaction.

Seeing the process of planning and carrying out indoor building work ideas related to smart building materials inevitably spring to mind. Much work goes into communication between different people involved in the process and into establishing and communicating the current status (structure, power routing, ventilation shafts, insulation, etc.) of the building. When imagine that brick, fixture, panel, screw and cable used could provide information about its position and status we could create valuable applications. Obviously always based on the assumption that computing and communication gets cheaper… I think it could be an interesting student project to systematically assess what building material would most benefit from sensing (or self-awareness) and processing and what applications this would enable; and in a second step create and validate a prototype.

Thursday, 21 February 2008

OLPC – cute and interesting – but what type of computer is it?

After the conference I had finally some time to try out my new XO Laptop (OLPC). It is fairly small, has a rubber keyboard and a very good screen. It can be used in laptop and e-book mode. A colleague described it as somewhere between a mobile phone and a notebook-computer – first I did not get it – but after using it I fully understand.

There is good documentation out – the getting started manual at laptop.org provides a very good entry point. Getting it up and running was really easy (finding the key for my WIFI-Access point at home was the most difficult part ;-)

There are two interesting wikis with material online at olpcaustria.org and laptop.org. I am looking forward to trying the development environments supplied with the standard distribution (Pippy and Etoys).

I would expect when Vivien get up in the morning and sees it I will be second in line for exploring the XO further. It is really designed in a way that makes it attractive for children. To say more about about the usability (in particular the software) I need to explore it more...

To me it is not understandable why it is so difficult to get them in Europe. I think the buy 1 and donate 1 approach was very good (but again this was only in the US)...

Wednesday, 8 August 2007

Museum Audio Guides – is there a way to make this a good experience?

We visited the archeology and Stone Age museum in Bad Buchenau http://www.federseemuseum.de/. For our visit we rented their audio guide system – they had one version for kids and one for adults. The audio guides were done very well and the information was well presented.

Nevertheless such devices break the joint experience of visiting a museum! We had three devices – and we stood next to each other listening but not talking to each other. Even though it may transport more information than the written signs it makes a poorer experience than reading and discussing. I wonder how one would design a good museum guide… There are plenty of projects but so far I have not seen the great solution.

Wednesday, 11 July 2007

MP3 Player Module for Prototyping

Just talked to Matthias Kranz and he told me about an interesting MP3-Player module from http://www.roguerobotics.com - uMP3. It is not cheap but very versatile and it seems easy to use with a PIC. Looking at the documentation of the uMP3 I saw there is even a function to control the playback speed. It seems this is an interesting tool for prototyping and teaching labs!