Monday, 17 December 2012

Silvia Miksch talking about time oriented visual analytics


It seems this term we picked a good slot for the lecture. On Thursday we had Prof. Silvia Miksch from Vienna University of Technology visiting our institute. We took this chance for another guest lecture in my advanced HCI class. Silvia presented a talk with the title “A Matter of Time: Interactive Visual Analytics of Time-Oriented Data and Information”. She first introduced the notion of interactive visual analytics and then systematically showed how time oriented data can be visually presented.

I really liked how Silvia motivated visual analytics and could not resist to adapt it with a Christmas theme. The picture shows three representations (1) numbers, always 3 grouped together, (2) a plot of the numbers where the first is the label and the second and the third are coordinates, and (3) a line connecting the labels in order. Her example was much nicer, but I missed to take a photo. And it is obvious that you do not put it on the same slide... Nevertheless I think even this simple Christmas tree example shows the power of visual analytics. This will go in my slide set for presentations in schools ;-)

If you are more interested in the details of the visualization of time oriented data, please have a look at the following book: Visallization of Time-Oriented Data, by Wolfgang Aigner, Silvia Miksch, Heidrun Schumann, and Christian Tominski. Springer, 2011. http://www.timeviz.net [2]. After the talk there was an interested discussion about the relationship and fundamental difference between time and space. I think this is worthwhile further discussion.


Another direction to follow up is tangible (visual) analytics. It would be interesting to assess the contributions to understanding of further modalities when interactively exploring data, e.g. haptics and sound. Some years back Martin Schrittenloher (one of my students in Munich) visited Morten Fjeld for his project thesis and experimented with force feedback sliders [1], … perhaps we should have this as a project topic again! An approach would be to look specifically at the understanding of data when force-feedback is presented on certain dimensions.

References
[1] Jenaro, J., Shahrokni, A., Schrittenloher, and M., Fjeld, M. 2007. One-Dimensional Force Feedback Slider: Digital platform. In Proc. Workshop at the IEEE Virtual Reality 2007 Conference: Mixed Reality User Interfaces: Specification, Authoring, Adaptation (MRUI07), 47-51
[2] Wolfgang Aigner, Silvia Miksch, Heidrun Schumann, and Christian Tominski. Visallization of Time-Oriented Data. Springer, 2011. http://www.timeviz.net

Tuesday, 11 December 2012

A proposal to replace non-archival publications

In the CHI community we have the notion of non-archival publications. Some years back this concept may have been good but I find it harder and harder to understand. Over the month I had several people about this concept and in Paris I discussed it with several colleagues, who are involved in SIGCHI. Here are some of the thoughts – hopefully as a starting point for further discussion.

 First a short introduction to the concept of Non-archival publications: non-archival publications in the “CHI world” are papers that are published and shown at the conference, but that must not be held against a later publication. In essence these papers are considered as not-published when reviewing an extended version of the paper. A typical example is to publish a work in progress (WIP) paper in one year showing outlining the concept, the research path you have started to take, and some initial findings. Than in the following year you publish a full paper that includes all the data and a solid analysis. In principles this a great way of doing research, getting feedback from the community on the way, and publishing then larger piece of work. Elba in our group did this very well: a WIP in CHI2010 [1] and then the full paper in CHI2011 [2]. This shows there is value to it and understand the motivation why the concept of non-archival publications was created.

Over the last years however I have seen a number points that highlight that the concept of non-archival publication is everything but not straightforward to deal with. The following points are from experience in my group over the last years.

1) Non-archival publications are in fact archival. Once you assign a document a DOI and include them in a digital library (DL) these publications are archived. The purpose of a digital library and the DOI is that things will live on, even if the people’s websites are gone. The point that the authors keep the copyright and can publish it again does not chance the fact that the paper is archived. It is hard to explain someone from another community (e.g. during a TPC meeting of Percom) that there is a paper which has a DOI, is in the ACM DL, it counts into the download and citation statistics of the author in the ACM DL and is indexed by Google Scholar, and yet it has to be considered as not published, when assessing a new publication.

2) Non-archival publications may be the only publication on a topic. Sometimes people have a really cool idea and some initial work and they publish it as WIP (non-archival). Then over the years the authors do not get around to write the full paper, e.g. because they did not get the funding to do it. Hence the non-archival work in progress paper is the only publication that the authors have about this work. As the believe it is interesting they and probably other people will reference this work – but referencing something that is non-archival is questionable, but not in this case as in fact is archival as it is in the DL with DOI. Here is an example from our own experience: Sometimes back we had in our view a cool idea to chance the way smart objects can be created [3] – we did initial prototypes but did not have funding for the full project (we still work on getting it). The WIP is the only “paper” we “published” on it and hence we keep it in our CVs.

3) Chances in authorship between non-archival and full paper. Academia is a dynamic environment and hence things are started in one place and continued somewhere else. In this process the people doing the research are very likely to chance. To account for this we typically include a reference to the first non-archival publication to acknowledge the earlier contributions made. We have one example were we had an idea for navigation system that we explored in Munich very superficial and wrote up a WIP [4]. Enrcio then moved on to Lancaster and did a serious system and study – and as he is a nice person he references the WIP to acknowledged that some other people were involved in initial phase of creating the idea [5]. And by doing so he increased Antonio’s and my citation count, as we list the WIP paper on our Google scholar page.

4) Non-archival publication are part of people’s citation count and h-index. When assessing the performance of individuals academia seems to move more and more towards “measurable” data, hence we see that citation counts and h-index may play a role. I have one “publication”, it was a poster at ISWC 2000 about a wearable RFID reader [6], that has 50+ citations and it hence impacts my h-index (on Google). For ISWC2000 posters were real publications in the IEEE DL, but this could have equally been a WIP at CHI. Hence there is the question: should non-archival material be part of the quantitative assessment of impact?

I have some further hypothetical points (inspired by the real world) that highlight some of the issues I see with the concept of non-archival publications:

Scenario A) Researcher X has great idea for a new device and publishes a non-archival paper including the idea, details about the way the implementation, some initial results, and a plan how she will do the study at Conf201X. She has a clear plan to complete the study and publish the full paper at Conf201X+1. She falls short in time due to be ill for a few months and manages to submit only a low quality full paper. Researcher Y talks to Research X at the conference is impressed and reads the non-archival version of the paper. He likes it and has some funds available hence he decides to do a follow-up building on this research. He hires 3 interns for the summer, gets 20 of the devices build, does a great study, and submits a perfect paper. The paper of Researcher Y is accepted and the paper of researcher X is not. My feeling would be that in this case Y should at least reference the non-archival paper of X, hence non-archival papers should be seen as previous work.

Scenario B) A researcher starts a project, creates a systems and does an initial qualitative study. He publishes the results as non-archival paper (e.g. WIP) including a description of the quantitative study to be conducted. Over the next month he does the quantitative study - it does not provide new insights, but confirms the initial findings. He decides to write a 4 page note in two column format that is over 95% the same text as the 6 page previously published WIP, just with the addition, of one paragraph that a qualitative study was conducted which confirmed the results. In this case having both papers in the digital library feels not write. The obvious solution would be to replace the work in progress by the note.

Here is a proposal how non-archival publication could be replaced: 
  • Everything that is published in the (ACM) digital library and which has an DOI is considered an archival publication (as they are in fact are) 
  • Publications carry labels such as WIP, Demo, Note, Full paper, etc. 
  • Scientific communities can decided to have certain venues that can be evolutionary, e.g. for SIGCHI this would be to my current understanding WIP, Interactivity, and workshops. 
  • Evolutionary publications can be replaced by “better” publications by the authors, e.g. an author of a WIP can replace this WIP in the next year with a Full paper or a Note, the DOI stays the same 
  • To ensure accountability (with regard to the DOI) the replaced version remain in the appendix of the new version, e.g. the full paper has then as appendix the WIP it replaces 
  • If evolutionary publications are not replaced by the author they stay as they are and other people have to consider these as previous work 
  • Citations accumulated along the evolutionary path are accumulated on the latest version include. 
  • Authors can decide (e.g. when the project team changes, when the results a contradictory to the initial publication, when significant parts of the system chance, when authors chance) to not go the evolutionary path. In this case they are measured against the state of the art, which includes their own work.

In the CHI context this could be as follows: you have a WIP in year X, in year X+1 you decided to replace the WIP by the accepted Full paper that extended this WIP, in year X+3 you decided to extend Full Paper with your accepted ToCHI paper. When people download the ToCHI paper they will have the full conference paper and the WIP in the appendix. The citations that are done on the WIP and on the full paper are included in the citations of the journal paper. In a case where you combine conference several papers into a consolidated journal paper, you would create a new instance not replacing any of it or you may replace one of the conference papers.

This approach does not solve all the problems but I hope it is a starting point for a new discussion.

Just claiming stuff that is in the ACM DL and has a DOI is not archival feels like we create our own little universe in which we decide that gravity is not relevant…

UPDATE - Discussion in facebook (2012-12-11):
---
Comment by Alan Dix:
It seems there are three separate notions of 'archival':
(i) doesn't count as prior publication for future, say, journals
(ii) is recorded in some stable way to allow clear citation
(iii) meets some minimum level against some set of quality criteria

In the days before people treated conferences as if they were journal publications. It was common to have major publications in university or industrial lab 'internal' report series. These were often cited, and if they made it to journals, it was years later. The institutions distributed and maintained the repositories, hence they were archival by defn (ii). Conference and workshop papers likewise were and have always been cited widely whether or not they were officially declared 'archival'.

Conference papers, even if from prestigious conferences such as CHI are NOT usually archival by defn (iii) - or at least cannot be guaranteed to - as it is not a minimal standard in all criteria, more a balance between criteria, if something is really novel and important, but maybe not 100% solid it would and *should* be conference publishable, but shuld not be jiurnal publishable until *everything* hits minimum standard may not be fantastic against any though - faultless != best

As for (i) that is about venue, politics and random rubbish rules. For a conference the issue is "is there enough new for the delegates to see?" (unless the conference is pretending to be 'archival' meaning (ii), but we should ignore such disingenuous venues).

For a journal, it would quite valid to publish a paper absolutely identical (copyright issues withstanding) one that had previously been published (and is archival by (i)) as its job is to ensure (ii).

This was common in the past with internal reports and common again now with eprints services providing pre-prints during submission as well as pre-publication.

In a web world *all* conference contributions are archival by defn (i) and *none* are by def. (ii).

Conferences are news channels, journals and quality agencies ... and when the two get confused the discipline is in crisis.

Comment by Eva Hornecker
Reading Alan's response I am reminded I used to learn the distinction between 'grey' literature (citable, e.g. technical reports) and white/black (not sure anymore which is which) that is either informal and not archived (e.g. workshop position papers) or fully published and peer reviewed. Difference with WiPs etc. is they are peer reviewed (although only gently)

Comment by Rod Murray-Smith
I guess there is also a question about whether WIPs are really still being used as "works in progress", or more frequently as a way to attend the conference despite the paper not being lucky enough to get in. Do we have any stats on % of papers which are recycled from the main conference, as by submitting them to that, authors are claiming that these are ready for archival. Similar issues for many workshop papers.

Comment by Alan Dix

Of course workshop position papers are often web 'archived' (my criteria (ii)), and some even heavily refereed ... indeed many people would prefer a CHI workshop paper on their CV than a more heavily refereed conference paper elsewhere ... I guess about brand, like a Nike holdall.

There is another orthogonal issue too which is about the level and surety of the process, which is pretty independent of the clarity and kind of criteria. You may have a poor quality journal that is using similar criteria to a better journal, but simply having a lower bar and perhaps, because of quality of reviewing, lower level of confidence. I'm sure both Fiat and Ferrari have quality control, just the level different.

In some ways I am happier with low quality journals that you now are low quality (and therefore readers apply caveat emptor) than high quality conferences, where it is easy for readers to assume high quality = all OK.

This is why I always feel that all reviewing processes should have a non-blind point, as a paper with a fantastic idea, but major methodological flaw, is fine if produced by an unknown person in and unknown institution (as readers will take it with a inch of salt), but should be rejected if from a major name in the field (as it is more liely to be taken as a pattern of how to do it by readers).

Alternatively anonymous refereeing + anonymous publication

... and none of this is about the absolute value, significance, etc. of the work, quality control is about stopping the bad apples, not making good ones.

Comment by Susanne Boll

I fully agree. Coming from the Multimedia community initially, I never understood this concept. SIGMM and the annual conferences will publish anything that undergoes a peer-review. Full papers are the most prestiguous one, short papers (4 pages) are for smaller contributions or more focused work. Workshops are THE platform to start new topics in the field and of course the work is peer-reviewed and published. For example, the Multimedia Information Retrieval run for several years and gained more and more interest in the field until it finally became an own conference.

I also found it strange this year that I reviewed a full paper for one year but had a deja vu as the work was already shown in the interactivity session the year before. This not only makes it difficult to judge novelty but also is contradictory to the blind review. Maybe have a look how other SIG conferences such as Multimedia handle it.

Comment Amanda Marisa Williams

I'm intrigued -- no time at the moment but it's bookmarked for later today. Def wanna have this conversation with some CHI veterans since I have some concerns about the archival/non-archival distinction as well.

Comment by Bo Begole
I think the crux of the issue is simply that we shouldn't use the term "archival" at all - as you point out, anything published on the DL with a DOI is "archived". It's an archaic term. More properly, we should use accurate terms to describe the level of review. CHI uses the terms "refereed" "juried" and "curated" for different levels http://chi2013.acm.org/authors/call-for-participation/#refereed which map to ACM categories of CHI refereed is roughly equiv to "refereed, formally reviewed" CHI juried is equiv to "reviewed" and CHI curated is roughly equiv to "unreviewed". CHI also uses the ACM criteria regarding republishability of content

Comment by Chris Schmandt
What Bo says is good, but this distinction is lost on the masses. It's a "CHI paper" no matter what venue. And even in the old day when we had the separate "abstracts" volume, only the few in the know could recognize the difference between the short ...
---
 
References
[1] Elba del Carmen Valderrama Bahamóndez and Albrecht Schmidt. 2010. A survey to assess the potential of mobile phones as a learning platform for panama. In CHI '10 Extended Abstracts on Human Factors in Computing Systems (CHI EA '10). ACM, New York, NY, USA, 3667-3672. DOI=10.1145/1753846.1754036 http://doi.acm.org/10.1145/1753846.1754036
[2] Elba del Carmen Valderrama Bahamondez, Christian Winkler, and Albrecht Schmidt. 2011. Utilizing multimedia capabilities of mobile phones to support teaching in schools in rural panama. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 935-944. DOI=10.1145/1978942.1979081 http://doi.acm.org/10.1145/1978942.1979081
[3] Tanja Doering, Bastian Pfleging, Christian Kray, and Albrecht Schmidt. 2010. Design by physical composition for complex tangible user interfaces. In CHI '10 Extended Abstracts on Human Factors in Computing Systems (CHI EA '10). ACM, New York, NY, USA, 3541-3546. DOI=10.1145/1753846.1754015 http://doi.acm.org/10.1145/1753846.1754015
[4] Enrico Rukzio, Albrecht Schmidt, and Antonio Krüger. 2005. The rotating compass: a novel interaction technique for mobile navigation. In CHI '05 Extended Abstracts on Human Factors in Computing Systems (CHI EA '05). ACM, New York, NY, USA, 1761-1764. DOI=10.1145/1056808.1057016 http://doi.acm.org/10.1145/1056808.1057016
[5] Enrico Rukzio, Michael Müller, and Robert Hardy. 2009. Design, implementation and evaluation of a novel public display for pedestrian navigation: the rotating compass. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, New York, NY, USA, 113-122. DOI=10.1145/1518701.1518722 http://doi.acm.org/10.1145/1518701.1518722
[6] Albrecht Schmidt, Hans-W. Gellersen, and Christian Merz. 2000. Enabling Implicit Human Computer Interaction: A Wearable RFID-Tag Reader. In Proceedings of the 4th IEEE International Symposium on Wearable Computers (ISWC '00). IEEE Computer Society, Washington, DC, USA, 193-194.

Monday, 10 December 2012

Congratulation to Dr. Florian Alt (No. 6)

Florian Alt defended his PhD thesis “A Design Space for Pervasive Advertising on Public Displays” at the University of Stuttgart. Over the last years Florian work at the crossroads of interactive public displays and pervasive advertising. His research output during the last years and while working on the http://pd-net.org project was amazing, see his DBLP entry.

The dissertation will be soon available online. If you are curious about his work right now, there are a few papers or a book you should read. A high level description of the findings is described in a paper published in IEEE Computer on Advertising on Public Display Networks [1]. The initial paper that paved the way towards understanding design space of public displays [2] is providing a comprehensive descriptions of ways for interaction with public displays. One of the highlights of the experimental research is the paper “Looking glass: a field study on noticing interactivity of a shop window” [3], which was done during Florian’s time at Telekom Innovation Laboratories in Berlin (it received a best paper award at CHI 2012). Towards the end of the thesis everyone realizes that evaluation is a most tricky thing, hence there is one paper on “How to evaluate public displays” [4]. If you are more interested on the advertising side, have a look at the book he co-edited with Jörg Müller and Daniel Michelis: Pervasive Advertising by Springer Verlag, 2011, available as kindle version at Amazon.

Florian joined my research group already back in Munich as a student researcher, where we explored ubiquitous computing technologies in a hospital environment [5]. He followed to Fraunhofer IAIS to do his MSc thesis, where he created a web annotation system that allowed parasitic applications on the WWW [6]. I nearly believed him lost, when he moved to New York – but he came back to start his PhD in Duisburg-Essen… and after one more move in 2011 to the University of Stuttgart he graduated last week! Congratulations! He is no. 6 following Dagmar Kern, Heiko Drewes, Paul Holleis, Matthias Kranz, and Enrico Rukzio. The photo shows the current team in Stuttgart – when looking at the picture it seems there are soon more to come ;-)

References
[1] Alt, F.; Schmidt, A.; Müller, J.; , "Advertising on Public Display Networks," Computer , vol.45, no.5, pp.50-56, May 2012. DOI: 10.1109/MC.2012.150, URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=6193076&isnumber=6197765
[2] Jörg Müller, Florian Alt, Daniel Michelis, and Albrecht Schmidt. 2010. Requirements and design space for interactive public displays. In Proceedings of the international conference on Multimedia (MM '10). ACM, New York, NY, USA, 1285-1294. DOI=10.1145/1873951.1874203 http://doi.acm.org/10.1145/1873951.1874203
[3] Jörg Müller, Robert Walter, Gilles Bailly, Michael Nischt, and Florian Alt. 2012. Looking glass: a field study on noticing interactivity of a shop window. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 297-306. DOI=10.1145/2207676.2207718 http://doi.acm.org/10.1145/2207676.2207718
[4] Florian Alt, Stefan Schneegaß, Albrecht Schmidt, Jörg Müller, and Nemanja Memarovic. 2012. How to evaluate public displays. In Proceedings of the 2012 International Symposium on Pervasive Displays (PerDis '12). ACM, New York, NY, USA, , Article 17 , 6 pages. DOI=10.1145/2307798.2307815 http://doi.acm.org/10.1145/2307798.2307815
[5] A. Schmidt, F. Alt, D. Wilhelm, J. Niggemann,  and H. Feussner,  Experimenting with ubiquitous computing technologies in productive environments. Journal Elektrotechnik und Informationstechnik. 2006, 135-139.
[6] Florian Alt, Albrecht Schmidt, Richard Atterer, and Paul Holleis. 2009. Bringing Web 2.0 to the Old Web: A Platform for Parasitic Applications. In Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part I (INTERACT '09). Springer-Verlag, Berlin, Heidelberg, 405-418. DOI=10.1007/978-3-642-03655-2_44 http://dx.doi.org/10.1007/978-3-642-03655-2_44

Wednesday, 5 December 2012

Call for Papers: Augmented Human Conference 2013 (AH2013)

In 2013 the 4th Augmented Human Conference will talk place in Stuttgart, Germany. The submission deadline is January 8, 2013 and the conference is in cooperation with ACM SIGCHI. The papers will be published in the ACM digital library. Andreas Bulling and Christian Holz are the program chairs and there is a fabulous technical program committee.

With AH2013 we continue a Conference that over last years has ventures beyond the usual things in human computer interaction and pervasive computing. Improving and augmenting human abilities is at the core of the conference, ranging from navigation systems, to actuator that help human movement, to improved or novel senses. This may include hardware, sensors, actuators, and software, such as web based applications or mobile apps.

We are curious about technologies and solutions that make humans smarter and augment human capabilities. Over the last years the conference has highly valueed novel contributions, inspiring ideas, forward thinking applications and new concepts. Originality, ingenuity, creativity, novelty come in this context before rigorous evaluations and flawless statistical analysis of the study data. We are looking forward to your contributions. Please the web page at http://www.hcilab.org/ah2013/
Thanks to Patrick Lühne for the great designs!