Tuesday, 9 June 2015

Security and privacy implications of memory aids

Ever considered what you risk when you use pervasive computing based memory aids? Can other people twist what you remember? Imagine someone tampers with your photos you keep in the cloud - will this change what you remember? Will this alter your memory. In the European Project RECALL we investigated pervasive memory augmentation through computing technologies. In an article for IEEE Pervasive Magizine we investigated privacy and security implications if memory aids.


The full article is at the moment available for free at http://www.computer.org/web/computingnow/content?g=53319&type=article&urlTitle=security-and-privacy-implications-of-pervasive-memory-augmentation

Davies, Nigel, Adrian Friday, Sarah Clinch, Corina Sas, Marc Langheinrich, Geoff Ward, and Albrecht Schmidt. "Security and privacy implications of pervasive memory augmentation." Pervasive Computing, IEEE 14, no. 1 (2015): 44-53.

Monday, 1 June 2015

Crowdfunding - Interviews with Amanda Williams and Khai Truong

In the next issue of the IEEE Pervasive Magazine I will publish in the column on Innovations in Ubicomp Products the article: "Crowdfunding for Ubicomp Products: Interviews with Amanda Williams and Khai Truong" - Here are already the full interviews.

Interview with Amanda Williams

 Albrecht: You had a successful kickstarter project with a "lamp" – can you shortly describe the product (perhaps you have one or two photos, too)? And who was on the team?

Amanda: Our kickstarter campaign was for a smart, programmable lamp named Clyde. Clyde is a unique, jellyfishlike desk lamp that does both bright task lighting and colored ambient lighting (see figure CLYDE). He comes equipped with environmental sensors that he automatically detects and reacts to, so whether you want him to be "touchy feely" or "afraid of the dark", you can change his personality without programming. But if you do want to program (or learn how) Clyde is Arduino compatible and open-source, and we provide a number of guides and tutorials at http://fabule.com/clyde/.

Our team during the crowdfunding and manufacturing process was composed of myself, Bruno Nadeau, and Angela Gabereau. I did a lot of the interaction design, electrical engineering, marketing, and customer engagement; Bruno handled the design of the physical enclosure, design for manufacture, firmware, and testing -- he spent the most time on the factory floor; Angela is an all-round great software engineer, so she contributed to firmware development, coding up some of our test software, and setting up the back end on our website for ecommerce and customer support.

Albrecht: Why did you go with crowdfunding? What were the advantages for you over other funding approaches?

Amanda: If you're starting a company with a hardware product, your typical startup funding options tend not to match your cost structure because they are really optimized for software companies. If you're creating a software product, your major expense is just developer time; in the early stages you'll have founders working for nothing, which means your expenses are almost nil. As little as $25-50k in seed funding from an angel or an accelerator can get a software startup to the point where it has traction, customers, and revenue - if the founders handle everything really well. This is virtually impossible for a hardware startup, no matter how disciplined the founders are, because they will have to deal with the costs of materials, manufacturing, and shipping physical products. And certifications, which are a significant cost and a real hurdle for people who don't have prior manufacturing experience. Costs vary depending on the product, but I think it's pretty rare to be able to do this well for less than $100k, and even that is a stretch for all but the easiest hardware projects.

So you have this chicken and egg problem. Investors (quite reasonably) don't want to part with a ton of money until they see that your business can actually sell a product; but it takes a lot of money to manufacture a product that you can sell. This is where crowdfunding comes in. It can provide the money you need to cover non-recoverable expenses, and if it goes well, it shows potential investors that you have a market for your product.

There are a number of risks here. Crowdfunding can help you succeed, but it can also give you enough rope to hang yourself with. A failed campaign -- or even a fairly lackluster success -- can actually hurt your chances of attracting investors. It's also extremely common for first-time project creators to underestimate their expenses. People worry a lot about their campaign failing, but that's not the worst outcome. The worst outcome is if it succeeds, you take on the obligation to deliver what you promised, and then you run out of money without delivering. In my experience, I see it becoming increasingly common for startups to start fundraising the moment their campaign wraps up, without waiting to deliver the actual product. I think that's the smart thing to do. It's a moment where your company and your product is all potential: you've shown you have traction, but you haven't yet had the chance to make any ugly manufacturing mistakes. Securing conventional funding to supplement the crowd-funding helps insure against the possibility of running out of cash before you deliver.

Albrecht: Can you give a short overview of your timeline - from idea, to running the kickstarter campaign, to delivering the final products?


Amanda: Our original idea evolved a lot over time. Around January-April 2012 we were working part-time on these LED solder kits, playing around with the idea of an electrical kit that we could build up into various different mechanical configurations. So we made a little solder kit of PCBs with a few through-hole components, as well a couple different laser-cut enclosure kits: one of which would let you build a soft ambient light, the other of which would let you build a little desklamp with an adjustible neck -- identical PCBs in both. We brought these kits to Maker Faire in the Bay Area and to a big "Maker Carnival" in Beijing, the first such event to be held in China. We were a bit surprised to see how much people were drawn to the attractive enclosure design we'd worked on, people who aren't usually that into solder kits. So we started considering how we might make these kits more approachable to a broader audience.

We submitted our idea to HAXLR8R, a hardware accelerator based in Shenzhen, and were accepted to the program. This meant that Bruno and I moved to Shenzhen in January 2013 to spend almost four months doing intensive product development. While we were there, we had access to fabrication techniques that we couldn't have afforded back home. We could cheaply CNC or vacuum cast really professional-looking prototypes, the kind of thing that you could imagine on a store shelf. We could also prototype more sophisticated electronics much faster. This enabled us to move away from solder kits (4 out of 5 people who wanted to buy our product at previous maker faires didn't know how to solder) to an Arduino-compatible board with auto-detectable sensor modules. We now had a product with a really gentle learning curve, allowing users to customize without any special tools at all, but still open enough for really deep hardware or software hacking. Also, we were able to make it look really cool!

We ran our Kickstarter campaign from May-June 2013 and made almost $150k, more than triple our goal. We thought we could deliver the product by Christmas, but because of our inexperience, we underestimated how much work was still required to design for manufacture, set up a test plan the factory could follow, finalize and purchase everything on the bill of materials, design supplementary materials like packaging and manuals, test for electromagnetic compatibility, and ship to 30 different countries. During this time we had to re-design our board because the microprocessor we were using suddenly shot up in price. We had to do more mold revisions than anticipated, and switch materials to a plastic that would look good when back lit by a bunch of really bright LEDs. Since we were using several different materials in our product -- injection-molded plastic, die-cut silicone, metal gooseneck tubing, off-the-shelf screws and nuts, and custom PCBs -- we had to work with a number of different, but interdependent suppliers. If one supplier has a delay, the others fall like dominoes. In hindsight, a lot of this stuff looks obvious, but our experience wasn't really atypical for first time product creators.

We shipped Clyde out to our backers and post-Kickstarter pre-orders in July of 2014. One typically thinks that shipping is the finish line, but the reality is, that's when you start seeing a lot of customer support work. It's a computational product, so of course, you have to answer questions about set-up, drivers (if anyone wants to plug it in to program from their computer), how the sensors work, how to assemble, etc. Sometimes things get held up in customs, sometimes packages go missing and need to be replaced. So that will take up a lot of your time for a few months, and customer support is never really an engineer's favorite task.

Albrecht: What was the minimum number of items that were required in order to make sense to start production and why? Is there an "Economy of scale" for Ubicomp products?

Amanda: There is ABSOLUTELY an economy of scale for Ubicomp products, though it depends a lot on your method of production. More and more manufacturers in China and North America are willing to do small runs, but it will inevitably cost more per piece.

When we were calculating our Kickstarter goal and reward levels, we calculated how much we would need for runs of 300, 500, and 1000. We ended up producing a run of 1000, plus 100 wooden special edition Clydes. (Note: I don't recommend Shenzhen for woodworking.) The cost of tooling for injection molding is high no matter what (we paid $10,000, and that was a remarkably good deal), but the more you make, the more that cost gets amortized across many units; and the unit cost itself also go down. If you've ever prototyped hardware and shopped for components somewhere like Digikey or even Sparkfun, you're probably familiar with the idea of price breaks for electronic components: each capacitor costs half as much if you buy 1000 instead of 10. The same logic applies to injection molded plastic parts, too. Every time you do a manufacturing run, a skilled technician has to set up the injection molding machine. They have to get the mold temperature, plastic temperature, injection speed, and release of air pressure calibrated exactly right for the size and shape of your product, and this takes a lot of trial and error. Once that's done, it's easy to spit out one unit after another; but you can see why manufacturers would prefer to do 10,000 units at once rather than 1,000 ten times. The calibration process also generates plastic waste that can't be recycled since it's often warped and burnt.

Method of production matters a lot. Injection molding, as described above, has a very high up-front cost and a low per-piece cost. However, if you happened to be making something using CNC milling, you'd have a fairly low set-up fee, and a relatively high per-piece cost. This really needs to be taken into account when you set up a crowdfunding campaign. Campaigns that go way over their funding goal are actually MORE likely to experience delays, and I suspect that a lot of pain could be avoided if creators thought ahead about whether their production methods scale well or not.


Albrecht: Ubicomp products typically include design, hardware and software (and often networking) and I guess you had the know-how for prototyping yourself / in your team. How did you transfer the expertise for prototyping to production?


Amanda: We learned the hard way. Bruno is extremely organized and detail-oriented, and it is absolutely crucial to have someone on your team like that. I turned out (to my own surprise) to be ruthless - kind of tactically evil, in fact - when it came to negotiating prices with suppliers; you probably also want someone like that on your team.

If you can hire a senior engineer with lots of manufacturing experience, that would be ideal. Most of us don't really have the budget for that, so we muddle through.

That said, we had great mentors. Bunnie Huang helped advise us on some manufacturing issues, and he's been a wonderful resource to us, other hardware startups, and even graduate students looking to manufacture creative hardware. Our own contract manufacturer (AQS), since they often work with startups, was willing to show us the ropes -- not everyone would have been so patient. It costs a bit more to work with a factory like that, but I'd say it was worth it. And we were also working in an environment where we had frequent contact with other teams who were doing similar work to us. So we could always talk to someone who was a year, six months, two months ahead of us in the process, and anticipate what might go wrong. And in turn, we helped teams that were a few months behind us.

Learning to manufacture is not easy, and it's not at all formalized. Most of us learn by forming professional networks and getting help from our friends. You have to be resourceful, and not shy about asking for help.

Albrecht: With crowdfunding you have customers for a product often already at the idea stage. How does this impact the product development and the final implementation?

Amanda: It's great! And it sucks! 

Most backers are really supportive. They back campaigns not just to acquire an end product, but to engage with the process of invention and design and production. So you can get great feedback from people regarding what features they want you to prioritize. We opened up our source code and saw people creating new modes of interaction for Clyde, and it was awesome. I'm eternally grateful to the 965 people who gave us a chance.

But it does add a lot of pressure. Even if people are nice, you feel the weight of your promise to give them what they want, and you feel pretty awful about every delay. I love learning new things, but it's more pleasurable without all the pressure!

Albrecht: How is crowdfunding changing how we make ubiquitous computing products? Are there new classes of products that become economically viable?


Amanda: Crowdfunding, along with a growing class of manufacturers that are willing to do small runs, allow us to actually bring more long-tail, niche devices into existence. Clyde's a pretty niche product. It's a quirky design. It's not meant to be one-size-fits all. 1100 was a pretty good number to produce. If we'd been required to do a minimum order quantity of 10,000, it just wouldn't have happened, but this way, about 1000 people get a neat little product that powerfully appeals to them. But I think also these small manufacturing projects have to be quite passion-driven, since the up front work to design a 1000-unit run is not much less than the work required to set up something much larger. You probably won't get filthy rich doing this stuff, but if you handle things right, you won't go broke, either.

This new set of capabilities makes the boundary between research and product-development a lot more porous. You could conceivably do a run of a hundred or so devices for a research project, and deploy them in the wild for validation. Universities and industrial research labs spin off research projects into startups sometimes; crowdfunding and small-run manufacturing can allow you to validate these projects before taking a big risk on them.

Albrecht: With crowdfunding you give your idea away (or at least you make it public) before you can ship. Were you afraid of others being faster in creating "your" product and having it on the market first? Did you take precautions to avoid competition?


Amanda: Protecting our IP wasn't a huge deal for us, but it can be for others. At $150K, we were below the threshold of interest for most copycats to try to reverse engineer our product -- if we'd raised more than half a million I'd have been a lot more worried. But anyway, we made it all open source. Preventing copying wasn't a huge priority for us. It is for others.

In the US you get a one year grace period between going public with your idea and filing a patent for it. This doesn't apply everywhere, so if you want to patent, submit it before you launch a crowdfunding campaign. However, patent protection is far from air-tight. Whether your patent protects you or not is really for the courts to decide, if you ever get into a lawsuit. Do you have the budget to hire a lawyer for this? A better lawyer than Xiaomi can afford? If you have a US or European patent, that won't count for anything in China. You can get a patent in Hong Kong if you're willing to translate everything, and the People's Republic of China will *ostensibly* honor it. But I will eat every ounce of our plastic waste if Chinese courts EVER rule in favor of a foreign company suing a Chinese company for patent infringement. In fact, I believe you have to set up a China-based subsidiary to even be allowed to sue.

So if patents are not that useful, what do you do to protect your product? You have a couple of options. 1) Move fast. Invent new things faster than people can copy your old things. But watch out, those shanzhai manufacturers are pretty quick. 2) Don't make incredibly simple pure-hardware products. Those are the easiest things to copy. They're right in the shanzhai manufacturer comfort zone. If you're doing something incredibly original in hardware, something that required lots of R&D to get right, that can help a lot. Or you can put a lot of the value in software and the creation of an awesome UX. If you're worried about how honest your manufacturer is, don't give them the source code -- just have them flash the binaries onto your microprocessors, or if you're really paranoid, do it yourself (we know someone who did that). The hardest thing for a competitor to steal is a fantastic user community.

That said, I really respect those shanzhai guys. Silvia Lindtner and Bunnie Huang have both written about what that work looks like on the ground. They are extremely clever about using their hardware ecosystem to reduce costs, and they create niche (or not-so-niche) products for markets that mainstream companies aren't touching, like a minimalist mobile phone that retails for 12 USD! That means the cost of materials, labor, and shipping have to add up to no more than 4 USD. That is pretty crazy.


Albrecht: How did you continue after "kick starting" you project / company?


Amanda: Manufacturing Clyde to fulfil our Kickstarter was a marvelous learning experience. Once we finished up that project, we realized it had given us a ton of ideas for tools that we wished we'd had during the manufacturing process. Effective collaborative tools for manufacturing just didn't seem to exist yet, at least not in any form usable to the increasing number of startups and small-scale creators that need them. We're working on an exciting new project now (up on fabule.com) for a hosted Bill of Materials management tool. The Bill of Materials is a keystone document that is an absolutely crucial point of communication between a creator and their manufacturer, at all stages of production, from prototyping, to sourcing, to assembly, testing, and subsequent manufacturing runs. Right now everyone is using Excel, and spending hours doing tedious and error-prone revisions, and maintaining multiple parallel copies for different purposes and different audiences. We've spoken to manufacturers who've had to spend hours coaching startups on how to provide them with the specific information they need to even be able to provide a quote. We know we can do better. At this point we're decent domain experts in hardware manufacturing, but we also have a really deep background in UX and software development, which means that we're really well-positioned to build a tool that can help guide first-timers to do the manufacturing collaboration correctly.

In the hardware ecosystem right now, we're seeing a revolution in prototyping technologies and fundraising techniques. The first two steps in creating a ubicomp product have gotten much, much easier, and it's opened the field up to a lot more people. However as soon as you face the problem of manufacturing, you find that that part is about as hard to navigate as it ever was. So now we're tackling that step, trying to find ways to standardize, educate, and create smoother collaborations between creators, manufacturers, and suppliers.


Albrecht: Further comments?


Amanda: When we make ubicomp products, we are often trying to make "normal" objects smarter. We stuff processors, and wifi or bluetooth, and batteries, and tiny circuit boards into everyday things like lamps, jewelry, cat toys, bicycle handlebars, cribs, etc. But the manufacturers who know how to make lamps, jewelry, cat toys, bicycle handlebars, and cribs don't necessarily know ANYTHING about manufacturing and testing computational hardware. And the manufacturers who know how to make and test PCBs don't necessarily know how to work with the materials that you'd need to make your thing. So when we try to make smarter everyday things, we often find ourselves in the position of having to educate our manufacturer, or at least we have to be a good bridge between manufacturers who have very different areas of expertise.


Short bio:

Amanda Williams is Co-Founder and CEO of Fabule Fabrications. She's in charge of creating beautiful interactions, beautiful hardware, and customer development. She has a Ph.D. in Information and Computer Sciences from UC Irvine and a B.S. in Symbolic Systems from Stanford University. Amanda has worked at Xerox PARC, Adobe, Intel Research, and Microsoft Research. Indecisively, she loves both qualitative user research and hardware design. She has been accused of eating like a trucker. 




Interview with Khai Truong 

Albrecht: You had a successful indiegogo project with a on-screen keyboard – can you shortly describe the product (perhaps you have one or two photos, too)? And who was on the team?


Khai: Minuum is an text input method which reduces a full size keyboard down to a single dimension (or row) of keys. For on-screen keyboards, this reduces the amount of space that would be taken up by the input method, giving more real estate to the rest of the application. The challenge introduced with enabling this is how to allow users to still type quickly as it becomes harder for them to precisely hit keys on this reduced sized keyboard. To this end, the backend of the product is a disambiguation engine that predicts what the user is typing without requiring the user to always hit the exact keys in the word that they are typing. By reducing text entry to being simply selection of keys placed on a single row, this same mode of typing can be carried over to a variety of other platforms and devices. 

The initial members of the team were Will Walmsley, Xavier Snellgrove and myself. Severin Smith joined the team later as a co-founder. Will, Xavier, and Severin remain a core member of the company. I've taken a step back as and I am acting as a scientific advisor to the company. The company, in the meantime, has grown to include more developers, designers, as well as marking, communications and business development people.


Albrecht: Why did you go with crowdfunding? What were the advantages for you over other funding approaches?


Khai: Of course, there is the financial benefits of crowdfunding. Because it's obvious, it probably does not need to be discussed. But there are several other reasons that might not be as clear, yet are important to consider.

One advantage with crowdfunding is that it is full of people who want to support new ideas. In a sense, these are people who are likely to be early adopters of the technology. They might be technology savvy. And they might be people who are open to the possibility that the product is still be refined.

A second advantage with crowdfunding is the buzz that it provides for the product. For Minuum, while the crowdfunding campaign occurred, the Youtube video about the product was watched by over a million people (there were two versions of the video because the audio in the initial version needed to be improved--the combined viewership of the two videos went over a million in a few weeks). This helped to advertise the product. The media also caught wind of the campaign and helped to promote awareness about the product because of the crowdfunding effort as well.

I think the final advantage is that it provides a way for gaining feedback about the product/idea from the potential users while the development of the product is still occurring. That can help to shape the product.


Albrecht: Can you give a short overview of your timeline … from idea, to running the indiegogo campaign, to delivering the final products?


Khai: The timeline actually started well before the campaign. For Minuum, the timeline might be somewhat unique as well. But the timeline went like this. 

I taught an HCI course that Will Walmsley took. For the course, students were asked to develop and evaluate an "eyes-free typing method." Will and his project team worked on a rotational gesture-based keyboard. The basic premise of the keyboard is that the user holds the phone in their hand. The user rotates their wrist along the forearm's axis to select letters (arranged alphabetically), and taps the screen to type that letter. As you can imagine, it's hard to imagine the user being able to easily and accurately selecting a letter in this way. He and his team designed and developed a rough prototype which demonstrated that with a disambiguation engine behind it, it was possible to handle the imprecisions (i.e., errors) involved with being able to correctly select letters using wrist rotations. 

The technique was interesting enough that Will and I continued to work on it as a research project. Over this period of time, we brought on another student in the lab (Xavier Snellgrove). Using an iterative design process over the course of a year, we continued to develop, evaluate and refine the prototype in numerous ways, including: dramatically improving the disambiguation algorithm, supporting more input gestures, providing auditory feedback and minimal visual feedback, and using two different alphabet layouts (a layout which adapted the familiar "QWERTY" layout to a single dimension and an expert/optimal "ENBUD" layout). The research was published in a TOCHI 2014 paper, but the interesting things that we learned were people could learn to use the method to perform eyes-free typing and that users were able to use the QWERTY layout to achieve comparable input rates to the optimal ENBUD layout (thus, the user's  familiarity with QWERTY could be leveraged rather than requiring the user to learn a completely new layout).

The point of explaining the above is to provide some context for what happens next.

Once we completed the Rotext prototype, we thought it was an interesting eyes-free typing method and thought about turning it into a product. Will was still a Master's student (and furthermore, he was not my Masters student) and needed to finish his Master's degree. I was also going to do my sabbatical leave at UC Irvine. So while we were interested in commercializing, we decided to wait until he finished his Master's before proceeding. However, when I started my sabbatical, I was contacted by the partnership office at University of Toronto. I was told that the university was starting a program to help incubate and commercialize early stage technology that summer. They were interested in supporting one of my projects. 

This lead me to have a conversation with Will about whether he would be interested in going forward with commercializing, what we would commercialize, and what our timeline would be like. While we thought eyes-free typing was interesting, we felt that it would be a hard interaction method for many people to be able to learn and adopt. We felt that the most interesting part of the work was the disambiguation engine's ability to support imprecise typing. Furthermore, we felt that the disambiguation engine could be used to support typing along any one dimensional axis. I had also attempted another startup a few years earlier on the 1Line Keyboard (a concept that we studied in a UIST 2009 paper). While that startup effort was not nearly as successful, I had stumbled into an interesting user problem: how to give the user more screenspace while typing. We targetted the tablet platform with the 1Line keyboard, which was only still growing at the time. So as a result, the 1Line company never picked up the momentum we had hoped for it. However, Will and I thought that the concept of using his disambiguation engine on a reduced size keyboard be really useful for mobile phones, which has a very small screen and so any added screen space we could give back to the applications in use would be really valuable.  We decided to call the product Minuum - because we were designing a mini sized keyboard with keys laid along a single continuum. We also felt that this keyboard could be the product that helps users become familiar to concept of imprecise typing along a single axis, and then they could adapt and transition that typing practice over to other devices/platforms/form factors (such as, typing on a small watch or gesture-based typing like the Rotext work).

We entered the UTEST program that summer to begin designing the Minuum keyboard.  Over the course of the summer, Will and I developed and refined our concepts for the Minuum keyboard and the SDK which would allow us and other developers to use the engine to support imprecise 1-dimensional typing on other devices/platforms/form factors. It was over the summer that we started to think about crowdfunding as the way to launch the product and bring awareness to the product and the company. Will and I started some high-level plans about what the crowdfunding campaign might involve, but it wasn't until we had the prototype that details of the crowdfunding effort was thought out more.

One of the reasons why we decided to go with crowdfunding was because the UTEST program gave us some initial funding to help us pay Will for the summer and fall. This meant that we could afford to ask for not a huge amount of money from the crowdfunding campaign to complete the product. So basically we were working in stealth mode the summer and fall of that year. We spent a significant amount of time on the keyboard design, the various features we were going to support. Will then started to port the engine so that it works on a number of platforms (including Android) and I developed the Android input method. By the end of the year we had a mostly demonstrable prototype.  It wasn't a finished product, but it was close enough and became the premise for what we showed in the crowdfuding video. When the next year rolled around, I had returned to Toronto. Xavier had joined the team and they worked on the launch.

We had to start to put together a website, develop the concept video, bring on people to help with marketing and communications (press releases, etc). I remember we had set the launch date to be early in the year, but we ended up delaying it because getting all these materials ready took a significant effort. Just to give you an example, the video wasn't finished until the early morning of the day of the launch. We still ended up needing to update the audio track in the video to make it easier to comprehend a day or two after the launch.  We ended up launching mid March (instead of earlier) and the campaign ran for a month.

With the launch came a lot of press and a number of people/companies contacting us. While Will and the team took all the press requests, feedback about the idea was also coming in. This really helped to provide an understanding of what people liked and wanted/asked for, etc.

What's involved in releasing a product is making the software/system run robustly. This is quite different from creating a research prototype. Coding and testing of the product, tracking of bugs, etc. is quite important and requires a lot of resources (including time).  So while it seemed like a lot of time often goes by before a product actually gets released after we hear about it initially, it's primarily because taking it from a concept/research prototype to a deliverable product still requires a significant amount of development/testing/debugging. For us, the beta version was release June of that year (or about 3 months after the launch).

The team has gone onto participating in the Y Combinator accelerator program and grown in size some (adding developers). They've developed an iPhone version of the keyboard.  They've developed smartwatch versions of the system demonstrated additional experimental uses of the SDK (such as Glass typing, or typing using a remote control, or a leap motion device, etc.). They've done a marvelous job of growing the company and the product in exciting ways. More information about all of these different efforts can be found: http://minuum.com/mediaroom/ .


Albrecht: With crowdfunding you have customers for a product often already at the idea stage. How does this impact the product development and the final implementation?


Khai: User feedback at any stage of development is always valuable. The most important one that crowdfunding provides is whether potential users find the overall concept appealing or not. Different from traditional product development, where we might have fully built the product, then try to put it on the market and see if people *might* buy the product, crowdfunding allows for companies to get insight into whether such customers exist already.

During the campaign, funders will discuss features that they are excited about and what they want and hope to have included in the product.  The funders also get early access to the product (the beta version).  This provides them with the opportunity to give us additional feedback before the first non-beta version is completed. All of this information helps us to understand user requirements and prioritize features. Obviously, not everything can be included in the initial version that gets released, but we can develop a list of features that gets added to subsequent versions.


Albrecht: You funded the development of software. In your research you created typical ubicomp systems – what challenges do you see in using crowd funding for ubicomp products that include hardware and software?


Khai: This is a really interesting question. Minuum was primarily a software product, which has very different costs to a hardware product. In particular, producing hardware in certain volume changes the costs involved. The same is not true with software. 

So in a sense, funding the development of a hardware product is more difficult. But what I think crowdfunding does for the funding of a product that includes a hardware component is that it mitigates some of the expense incurred for initially producing at smaller volumes. What makes it hard for a company, when hardware is involved, is that the manufacturing of a large amount of their product might not be wise early on. The challenge is then of course the pricing associated with the product, and what to ask from crowdfunders so that it seems reasonable. 
  
Albrecht: How is crowdfunding changing how we make ubiquitous computing products? Are there new classes of products that become economically viable?


Khai: I think crowdfunding allows for us to take concepts that we have done some basic research and development on, and turn into actual products when previously we would need to secure the funds to do so first. Even if we know that an idea can work and can be turned into a product, can we find people who believe in us enough to fund that development? Crowdfunding allows for enough people who believe in the idea to share some of that cost. 

Ultimately, I think this can benefit ubicomp products that have some hardware component. The cost to create that product is high. We can show people large bulky prototypes of the same concept, but would a large bulky prototype be enough to convince an investor to provide the significant funding needed for the development of that product? Maybe, if they could be convinced that there is a particularly large market for the finished product. And that's what I envision crowdfunding doing, it helps to incur some of that cost. It helps to demonstrate market size. And this helps the company grow and develop the products enough so that they can reach the final design, price point, experience etc. that would be otherwise hard to achieve for their product.


Albrecht: With crowdfunding you give your idea away (or at least you make it public) before you can deliver it. Were you afraid of others being faster in creating "your" product and having it on the market first? Did you take precautions to avoid competition?

Khai: We had patents filed already before the launch. 

We also didn't want to overpromise and not be able to deliver as well. So we had many of the capabilities and much of the backend for the product created already. They were not fully tested. They didn't do everything the released version supported, and they didn't do everything that is supported in subsequent versions either. But enough of it was completed that we knew a) we could do it, and b) we were "months" away from an actual release. That would make it hard for others to catch up.

Short bio:
Khai N. Truong, Ph.D. is a professor at the University of Toronto. His research lies at the intersection of Human-Computer Interaction (HCI) and Ubiquitous Computing (UbiComp), specifically examining the mutual impact of usability and technical constraints on the design of applications and interaction techniques for novel, off-the-desktop computing systems that may be commonplace in 5-10 years. He received his PhD in computer science from the Georgia Institute of Technology.
 

Friday, 23 August 2013

Usable Privacy - did you install the seat belt in your car?

In a recent chat with the journalist Eva Wolfangel we discussed why digital security and privacy is so little usable and why many computer scientists seem not to understand the problem. Reading several articles in newspapers I got really annoyed by many of my CS colleagues who:
(1) blame the user for not taking enough care of the data and for making little effort in installing the encryption modules into their email programs and
(2) focusing on new technologies and better encryption and better algorithms to improve security and not considering the entire system including the human user.

Eva wrote an interesting and comprehensive article on usable security in Spectrum der Wissenschaft (it is German and the full version is online at her website). In the following I am sharing the some of the thougts..

@1: Mount the Seatbelts Yourself 
Technically I agree that encryption is not really complicated to install and that most people using computers could learn how to keep their data safe and how to communicate using encryption. From my experience in the real world I see that they chose not to learn it and I completely disagree that this is the user’s fault. Making the end user responsible for security and privacy is in my view entirely and utterly wrong.

Photo by Wikipedia/Michiel1972
Consider this (obviously fictional) example that applies “user responsibly for safety” to another widely used product and shows how strange the idea is:
When you get a new car there are already fixtures and wholes prepared where you can attach the seat belts. In order to get the seats belts which you can than mount in your car, you just have to fill in a post-card (you get with the car) and send it to the manufacturer of your favorite seat belts. You get then the safety belts mailed to you home – free of charge – together with a 2-page manual how to fix them in the car. The only thing you need is a screwdriver and a wrench. It is so easy that really everyone can make their car safe within 30 minutes.

It is very clear and little surprising to anyone that this is not how we do it with cars. We have agreed that the car company is responsible for the safety of the car. Economically the above example would make it cheaper for the manufacturer – probably not all people would claim their seatbelt and the company saves the effort in mounting it. Nevertheless car companies still have to provide you with a build in seat belt if they want to sell their car in Germany…

@2: Live in a Bunker 
Again from a technical perspective it is of great importance that the algorithms are secure and the encryption is strong. Nevertheless this is in my view not the key problem. Take the following example. What is better a 20 random character string password or a 4 digit PIN? From a technical perspective this is clear – however most people will be able to remember a 4 digit PIN (without writing it down) but not many will be able to remember a 20 random character string. Hence the overall system with the PIN – if well designed – may be “better” than the apparently save password based solution (as people will write it down or email it to themselves).

In the physical world we are used to complex (social) systems that allow us to live in a secure environment. In Germany people generally live in houses and flats where people who are determined can break in (e.g. using a sledge hammer on the door, a stone from the front yard on the window, or using more subtle methods). Even though people could fortify their house most people I know value their windows and easy access to their house and do not live in a bunker or add seven additional locks to their front doors – they balance risk and comfort. In traditional environments we rely on the whole system: we expect that neighbors will keep an open eye, forced entry will leave traces, police will try to find a burglar and that they will be punished, and that for most people the risk of committing a crime is not worth the potential benefit.

From a society perspective we similarly balance risk and freedom. If a purse is stolen in a small town the police will not seal off the area and check each person and search each house. Traditionally this is not possible due to the effort involved but also due to our understanding that the actions taken by law enforcement has to follow the proportionality principle. In Germany we do not consider imposing a curfew, even though one could imagine that this would even more reduce the crime rate.

I think we should take the physical and social world as example and inspiration to create usable and secure systems that offer privacy to the end user.

Overall I think security and privacy in digital systems is much more a human computer interaction problem than most people (especially from the security community) think! If you read German you may want to look at the article Eva Wolfangel wrote on the topic.

Friday, 21 June 2013

Reading List: Developing Ubiquitous Computing Devices

 

Together with Thomas Kubitza I was teaching a class in the UBI summer school on Developing Ubiquitous Computing Devices. The summer school was held in Oulu and organized by Timo Ojala.

In total the summer school include the following 4 courses:

  • EXPERIENCE-DRIVEN DESIGN OF UBIQUITOUS INTERACTIONS IN URBAN SPACES Prof. Kaisa Väänänen-Vainio-Mattila, Tampere University of Technology, Finland & Dr. Jonna Häkkilä, University of Oulu, Finland
  • DESIGNING MOBILE AUGMENTED REALITY INTERFACES Prof. Mark Billinghurst, University of Canterbury, New Zealand 
  • DEVELOPING UBIQUITOUS COMPUTING DEVICES Prof. Albrecht Schmidt, University of Stuttgart, Germany 
  • URBAN RESOURCE NETWORKS Prof. Malcolm McCullough, University of Michigan, USA 
There was more than work... if you are curious have a look at flickr for photos and more photos.
 
As some people asked for the reading list for our course on Developing Ubiquitous Computing Devices, I thought I post it here.... The reading list is also available as PDF for download.

The reading list comprises 4 areas that are relevant to our course. We expect that you have come across the original paper by Marc Weiser, introducing the concept of ubiquitous computing [1].

In the first part we have included papers that provide an overview of interaction concepts that are relevant in the context of ubiquitous computing. In particular this is tangible interaction [2a] [2b], reality based interaction [3], embedded interaction [4]. The concept of informative art [5] is introduced as well as the notion of persuasive technologies [16].This part is concluded with an overview of interaction with computers in the 21st century [6].

In the second part we have included a paper on how to create smart devices [7], which gives an overview of sensors that may be useful for creating novel and reactive devices. In [8] sensing is extended to context and context-awareness. In the third part we introduce the .NET Gadgeteer platform [9] and show some trends in the development of ubiquitous computing devices: how can we create new products once we can fabricate things [10] and enclosures [10b] and how ubicomp technologies enable new devices and devices concepts [11].

The final part provides some ideas for application scenarios that we plan to assess during the course. In [12] a concept of how to change a bed into a communication media is presented and in [13] a social alarm clock is presented. A recent study [14] shows the impact of technology on communication and in [15] an overview of novel alarm clocks and sleep monitoring devices is given.

References
[1] Weiser, M. (1991). The computer for the 21st century. Scientific american,265(3), 94-104. http://wiki.daimi.au.dk/pca/_files/weiser-orig.pdf
[2a] Ishii, H., & Ullmer, B. (1997, March). Tangible bits: towards seamless interfaces between people, bits and atoms. In Proceedings of the ACM SIGCHI Conference on Human factors in computing systems (pp. 234-241). ACM. http://doi.acm.org/10.1145/258549.258715 http://labs.rightnow.com/colloquium/papers/tangiblebits.pdf
[2b] Ishii, H. (2008, February). Tangible bits: beyond pixels. In Proceedings of the 2nd international conference on Tangible and embedded interaction (pp. xv-xxv). ACM. http://doi.acm.org/10.1145/1347390.1347392
[3] Jacob, R. J., Girouard, A., Hirshfield, L. M., Horn, M. S., Shaer, O., Solovey, E. T., & Zigelbaum, J. (2008, April). Reality-based interaction: a framework for post-WIMP interfaces. In Proceedings of the SIGCHI conference on Human factors in computing systems (pp. 201-210). ACM. http://doi.acm.org/10.1145/1357054.1357089 http://research.cs.queensu.ca/~audrey/papers/chi08.pdf
[4] Kranz, M., Holleis, P., & Schmidt, A. (2010). Embedded interaction: Interacting with the internet of things. Internet Computing, IEEE, 14(2), 46-53. http://dx.doi.org/10.1109/MIC.2009.141 http://pure.ltu.se/portal/files/39756776/FINAL_PRINT_w2iot_preprint.pdf
[5] Ferscha, A. (2007). Informative art display metaphors. In Universal Access in Human-Computer Interaction. Ambient Interaction (pp. 82-92). Springer Berlin Heidelberg. http://www.pervasive.jku.at/Research/Publications/_Documents/InformativeArtDisplayMetaphors-ferscha2007.pdf
[6] Schmidt, A., Pfleging, B., Alt, F., Sahami, A., & Fitzpatrick, G. (2012). Interacting with 21st-Century Computers. Pervasive Computing, IEEE, 11(1), 22-31. http://www.hcilab.org/wp-content/uploads/schmidt-ieeepc-21century.pdf http://dx.doi.org/10.1109/MPRV.2011.81
[7] Schmidt, A., & Van Laerhoven, K. (2001). How to build smart appliances?.Personal Communications, IEEE, 8(4), 66-71. http://www.comp.lancs.ac.uk/~albrecht/pubs/pdf/schmidt_ieee_pc_08-2001.pdf
[8] Schmidt, A. (2013). Context-Aware Computing: Context-Awareness, Context-Aware User Interfaces, and Implicit Interaction. The Encyclopedia of Human-Computer Interaction, 2nd Ed. http://www.interaction-design.org/encyclopedia/context-aware_computing.html
[9] Villar, N., Scott, J., Hodges, S., Hammil, K., & Miller, C. (2012). . NET gadgeteer: a platform for custom devices. In Pervasive Computing (pp. 216-233). Springer Berlin Heidelberg. http://research.microsoft.com/pubs/163162/Gadgeteer%20Pervasive%202012%20Proof.pdf
[10] Schmidt, A., Doring, T., & Sylvester, A. (2011). Changing How We Make and Deliver Smart Devices: When Can I Print Out My New Phone?. Pervasive Computing, IEEE, 10(4), 6-9. http://www.hci.simtech.uni-stuttgart.de/wp-content/uploads/schmidt2011changing.pdf http://dx.doi.org/10.1109/MPRV.2011.68
[10b] Weichel C., Lau M., Gellersen,H. (2013). Enclosed: A Component-Centric Interface for Designing Prototype Enclosures. Tangible, embedded, and embodied interaction conference (TEI 2013) http://doi.acm.org/10.1145/2460625.2460659 http://www.csweichel.de/papers/2013-enclosed.pdf
[11] Hodges, S., Villar, N., Scott, J., & Schmidt, A. (2012). A New Era for Ubicomp Development. Pervasive Computing, IEEE, 11(1), 5-9. http://dx.doi.org/10.1109/MPRV.2012.1 http://research.microsoft.com/pubs/163175/ANewEraForUbiCompDevelopment-IEEEPervasiveComputing.pdf
[12] Dodge, C. (1997, March). The bed: a medium for intimate communication. InCHI'97 extended abstracts on Human factors in computing systems: looking to the future (pp. 371-372). ACM. http://doi.acm.org/10.1145/1120212.1120439
[13] Schmidt, A., Shirazi, A. S., & van Laerhoven, K. (2012). Are You in Bed with Technology?. Pervasive Computing, IEEE, 11(4), 4-7. http://dx.doi.org/10.1109/MPRV.2012.63
[14] Schmidt, A. (2006). Network alarm clock (The 3AD International Design Competition). Personal and Ubiquitous Computing, 10(2-3), 191-192. http://dx.doi.org/10.1007/s00779-005-0022-y http://old.hcilab.org/documents/Schmidt_NetworkAlarmClock.pdf
[15] Shirazi, A. S., Clawson, J., Hassanpour, Y., Tourian, M. J., Schmidt, A., Chi, E. H., Borazio, M., & Van Laerhoven, K. (2013). Already Up? Using Mobile Phones to Track & Share Sleep Behavior. International Journal of Human-Computer Studies. http://www.sciencedirect.com/science/article/pii/S1071581913000244
[16] Fogg, B. J. (2009, April). A behavior model for persuasive design. In Proceedings of the 4th international conference on persuasive technology (p. 40). ACM. http://bjfogg.com/fbm_files/page4_1.pdf

Appendix: .NET Gadgeteer Links (optional)

Tuesday, 4 June 2013

Keynote at PerDis2013: Proxemic Interactions by Saul Greenberg


Saul Greenberg presented the opening keynote at PerDis2013, the second international symposium on pervasive displays, held at Google in Mountain View, US.

Saul gave a brief history motivating the challenges that arise from the move to interactive ubiquitous computing environments. The degrees of freedom for interaction, when moving from graphical user interfaces to ubiquitous computing environments, are massively increased and the social context becomes central.

The other line of motivation Saul used is the notion of proxemics as studied in social science. The primary element is the distance between people. By physical proximity a lot in the interaction between people is determined. Interpersonal relationships are at the heart of the theory by Edward Hall, who explored this already in the 1960ties ([1], for a short overview and introduction see the Wiki-Pages on Edward Hall and on Proxemics). It is interesting (and not undisputed) to see that people in computer science have moved the notion of proxemics beyond human-to-human interaction to include technologies.

Saul outlined the dimensions for proximic interactions:
  • Distance 
  • Movement 
  • Location 
  • Orientation 
  • Identity 

In a paper in ACM Interactions Saul provides a really good and easy to read introductory text to proximic interactions – which is also well suitable for teaching [2]. There is more on the dimensions, the overall concept of proximic interactions, and potential applications in a 2010 paper they presented at ITS [3]. One of the aspects they have looked into in their work is at supporting proxemic interactions through a toolkit [4]. For more details we can be looking forward to the PhD thesis of Nicolai Marquardt, who worked in Saul’s group and who will defend in a few weeks.

Proxiemic interaction is a hot topic and several researchers have started to explore this space. There is also a Dagstuhl Seminar on the topic later this year (http://www.dagstuhl.de/13452) orgamized by Saul Greenberg, Kasper Hornbæk, Aaron Quigley, and Harald Reiterer.

[1] Hall, E. T., & Hall, E. T. (1969). The hidden dimension (p. 119). New York: Anchor Books. http://courses.arch.ntua.gr/fsr/137555/Hall-The-Hidden-Dimension.pdf
[2] Greenberg, S., Marquardt, N., Ballendat, T., Diaz-Marino, R., & Wang, M. (2011). Proxemic interactions: the new ubicomp?. interactions, 18(1), 42-50. http://grouplab.cpsc.ucalgary.ca/grouplab/uploads/Publications/Publications/2011-ProxemicInteraction.Interactions.pdf
[3] Ballendat, T., Marquardt, N., & Greenberg, S. (2010, November). Proxemic interaction: designing for a proximity and orientation-aware environment. In ACM International Conference on Interactive Tabletops and Surfaces (pp. 121-130). ACM. http://www.cs.ucf.edu/courses/cap6121/spr11/readings/proxemic.pdf
[4] Marquardt, N., Diaz-Marino, R., Boring, S., & Greenberg, S. (2011, October). The proximity toolkit: prototyping proxemic interactions in ubiquitous computing ecologies. In Proceedings of the 24th annual ACM symposium on User interface software and technology (pp. 315-326). ACM. http://curis.ku.dk/ws/files/44312111/marquardt.UIST_2011.proximity_toolkit.pdf


Monday, 17 December 2012

Silvia Miksch talking about time oriented visual analytics


It seems this term we picked a good slot for the lecture. On Thursday we had Prof. Silvia Miksch from Vienna University of Technology visiting our institute. We took this chance for another guest lecture in my advanced HCI class. Silvia presented a talk with the title “A Matter of Time: Interactive Visual Analytics of Time-Oriented Data and Information”. She first introduced the notion of interactive visual analytics and then systematically showed how time oriented data can be visually presented.

I really liked how Silvia motivated visual analytics and could not resist to adapt it with a Christmas theme. The picture shows three representations (1) numbers, always 3 grouped together, (2) a plot of the numbers where the first is the label and the second and the third are coordinates, and (3) a line connecting the labels in order. Her example was much nicer, but I missed to take a photo. And it is obvious that you do not put it on the same slide... Nevertheless I think even this simple Christmas tree example shows the power of visual analytics. This will go in my slide set for presentations in schools ;-)

If you are more interested in the details of the visualization of time oriented data, please have a look at the following book: Visallization of Time-Oriented Data, by Wolfgang Aigner, Silvia Miksch, Heidrun Schumann, and Christian Tominski. Springer, 2011. http://www.timeviz.net [2]. After the talk there was an interested discussion about the relationship and fundamental difference between time and space. I think this is worthwhile further discussion.


Another direction to follow up is tangible (visual) analytics. It would be interesting to assess the contributions to understanding of further modalities when interactively exploring data, e.g. haptics and sound. Some years back Martin Schrittenloher (one of my students in Munich) visited Morten Fjeld for his project thesis and experimented with force feedback sliders [1], … perhaps we should have this as a project topic again! An approach would be to look specifically at the understanding of data when force-feedback is presented on certain dimensions.

References
[1] Jenaro, J., Shahrokni, A., Schrittenloher, and M., Fjeld, M. 2007. One-Dimensional Force Feedback Slider: Digital platform. In Proc. Workshop at the IEEE Virtual Reality 2007 Conference: Mixed Reality User Interfaces: Specification, Authoring, Adaptation (MRUI07), 47-51
[2] Wolfgang Aigner, Silvia Miksch, Heidrun Schumann, and Christian Tominski. Visallization of Time-Oriented Data. Springer, 2011. http://www.timeviz.net

Tuesday, 11 December 2012

A proposal to replace non-archival publications

In the CHI community we have the notion of non-archival publications. Some years back this concept may have been good but I find it harder and harder to understand. Over the month I had several people about this concept and in Paris I discussed it with several colleagues, who are involved in SIGCHI. Here are some of the thoughts – hopefully as a starting point for further discussion.

 First a short introduction to the concept of Non-archival publications: non-archival publications in the “CHI world” are papers that are published and shown at the conference, but that must not be held against a later publication. In essence these papers are considered as not-published when reviewing an extended version of the paper. A typical example is to publish a work in progress (WIP) paper in one year showing outlining the concept, the research path you have started to take, and some initial findings. Than in the following year you publish a full paper that includes all the data and a solid analysis. In principles this a great way of doing research, getting feedback from the community on the way, and publishing then larger piece of work. Elba in our group did this very well: a WIP in CHI2010 [1] and then the full paper in CHI2011 [2]. This shows there is value to it and understand the motivation why the concept of non-archival publications was created.

Over the last years however I have seen a number points that highlight that the concept of non-archival publication is everything but not straightforward to deal with. The following points are from experience in my group over the last years.

1) Non-archival publications are in fact archival. Once you assign a document a DOI and include them in a digital library (DL) these publications are archived. The purpose of a digital library and the DOI is that things will live on, even if the people’s websites are gone. The point that the authors keep the copyright and can publish it again does not chance the fact that the paper is archived. It is hard to explain someone from another community (e.g. during a TPC meeting of Percom) that there is a paper which has a DOI, is in the ACM DL, it counts into the download and citation statistics of the author in the ACM DL and is indexed by Google Scholar, and yet it has to be considered as not published, when assessing a new publication.

2) Non-archival publications may be the only publication on a topic. Sometimes people have a really cool idea and some initial work and they publish it as WIP (non-archival). Then over the years the authors do not get around to write the full paper, e.g. because they did not get the funding to do it. Hence the non-archival work in progress paper is the only publication that the authors have about this work. As the believe it is interesting they and probably other people will reference this work – but referencing something that is non-archival is questionable, but not in this case as in fact is archival as it is in the DL with DOI. Here is an example from our own experience: Sometimes back we had in our view a cool idea to chance the way smart objects can be created [3] – we did initial prototypes but did not have funding for the full project (we still work on getting it). The WIP is the only “paper” we “published” on it and hence we keep it in our CVs.

3) Chances in authorship between non-archival and full paper. Academia is a dynamic environment and hence things are started in one place and continued somewhere else. In this process the people doing the research are very likely to chance. To account for this we typically include a reference to the first non-archival publication to acknowledge the earlier contributions made. We have one example were we had an idea for navigation system that we explored in Munich very superficial and wrote up a WIP [4]. Enrcio then moved on to Lancaster and did a serious system and study – and as he is a nice person he references the WIP to acknowledged that some other people were involved in initial phase of creating the idea [5]. And by doing so he increased Antonio’s and my citation count, as we list the WIP paper on our Google scholar page.

4) Non-archival publication are part of people’s citation count and h-index. When assessing the performance of individuals academia seems to move more and more towards “measurable” data, hence we see that citation counts and h-index may play a role. I have one “publication”, it was a poster at ISWC 2000 about a wearable RFID reader [6], that has 50+ citations and it hence impacts my h-index (on Google). For ISWC2000 posters were real publications in the IEEE DL, but this could have equally been a WIP at CHI. Hence there is the question: should non-archival material be part of the quantitative assessment of impact?

I have some further hypothetical points (inspired by the real world) that highlight some of the issues I see with the concept of non-archival publications:

Scenario A) Researcher X has great idea for a new device and publishes a non-archival paper including the idea, details about the way the implementation, some initial results, and a plan how she will do the study at Conf201X. She has a clear plan to complete the study and publish the full paper at Conf201X+1. She falls short in time due to be ill for a few months and manages to submit only a low quality full paper. Researcher Y talks to Research X at the conference is impressed and reads the non-archival version of the paper. He likes it and has some funds available hence he decides to do a follow-up building on this research. He hires 3 interns for the summer, gets 20 of the devices build, does a great study, and submits a perfect paper. The paper of Researcher Y is accepted and the paper of researcher X is not. My feeling would be that in this case Y should at least reference the non-archival paper of X, hence non-archival papers should be seen as previous work.

Scenario B) A researcher starts a project, creates a systems and does an initial qualitative study. He publishes the results as non-archival paper (e.g. WIP) including a description of the quantitative study to be conducted. Over the next month he does the quantitative study - it does not provide new insights, but confirms the initial findings. He decides to write a 4 page note in two column format that is over 95% the same text as the 6 page previously published WIP, just with the addition, of one paragraph that a qualitative study was conducted which confirmed the results. In this case having both papers in the digital library feels not write. The obvious solution would be to replace the work in progress by the note.

Here is a proposal how non-archival publication could be replaced: 
  • Everything that is published in the (ACM) digital library and which has an DOI is considered an archival publication (as they are in fact are) 
  • Publications carry labels such as WIP, Demo, Note, Full paper, etc. 
  • Scientific communities can decided to have certain venues that can be evolutionary, e.g. for SIGCHI this would be to my current understanding WIP, Interactivity, and workshops. 
  • Evolutionary publications can be replaced by “better” publications by the authors, e.g. an author of a WIP can replace this WIP in the next year with a Full paper or a Note, the DOI stays the same 
  • To ensure accountability (with regard to the DOI) the replaced version remain in the appendix of the new version, e.g. the full paper has then as appendix the WIP it replaces 
  • If evolutionary publications are not replaced by the author they stay as they are and other people have to consider these as previous work 
  • Citations accumulated along the evolutionary path are accumulated on the latest version include. 
  • Authors can decide (e.g. when the project team changes, when the results a contradictory to the initial publication, when significant parts of the system chance, when authors chance) to not go the evolutionary path. In this case they are measured against the state of the art, which includes their own work.

In the CHI context this could be as follows: you have a WIP in year X, in year X+1 you decided to replace the WIP by the accepted Full paper that extended this WIP, in year X+3 you decided to extend Full Paper with your accepted ToCHI paper. When people download the ToCHI paper they will have the full conference paper and the WIP in the appendix. The citations that are done on the WIP and on the full paper are included in the citations of the journal paper. In a case where you combine conference several papers into a consolidated journal paper, you would create a new instance not replacing any of it or you may replace one of the conference papers.

This approach does not solve all the problems but I hope it is a starting point for a new discussion.

Just claiming stuff that is in the ACM DL and has a DOI is not archival feels like we create our own little universe in which we decide that gravity is not relevant…

UPDATE - Discussion in facebook (2012-12-11):
---
Comment by Alan Dix:
It seems there are three separate notions of 'archival':
(i) doesn't count as prior publication for future, say, journals
(ii) is recorded in some stable way to allow clear citation
(iii) meets some minimum level against some set of quality criteria

In the days before people treated conferences as if they were journal publications. It was common to have major publications in university or industrial lab 'internal' report series. These were often cited, and if they made it to journals, it was years later. The institutions distributed and maintained the repositories, hence they were archival by defn (ii). Conference and workshop papers likewise were and have always been cited widely whether or not they were officially declared 'archival'.

Conference papers, even if from prestigious conferences such as CHI are NOT usually archival by defn (iii) - or at least cannot be guaranteed to - as it is not a minimal standard in all criteria, more a balance between criteria, if something is really novel and important, but maybe not 100% solid it would and *should* be conference publishable, but shuld not be jiurnal publishable until *everything* hits minimum standard may not be fantastic against any though - faultless != best

As for (i) that is about venue, politics and random rubbish rules. For a conference the issue is "is there enough new for the delegates to see?" (unless the conference is pretending to be 'archival' meaning (ii), but we should ignore such disingenuous venues).

For a journal, it would quite valid to publish a paper absolutely identical (copyright issues withstanding) one that had previously been published (and is archival by (i)) as its job is to ensure (ii).

This was common in the past with internal reports and common again now with eprints services providing pre-prints during submission as well as pre-publication.

In a web world *all* conference contributions are archival by defn (i) and *none* are by def. (ii).

Conferences are news channels, journals and quality agencies ... and when the two get confused the discipline is in crisis.

Comment by Eva Hornecker
Reading Alan's response I am reminded I used to learn the distinction between 'grey' literature (citable, e.g. technical reports) and white/black (not sure anymore which is which) that is either informal and not archived (e.g. workshop position papers) or fully published and peer reviewed. Difference with WiPs etc. is they are peer reviewed (although only gently)

Comment by Rod Murray-Smith
I guess there is also a question about whether WIPs are really still being used as "works in progress", or more frequently as a way to attend the conference despite the paper not being lucky enough to get in. Do we have any stats on % of papers which are recycled from the main conference, as by submitting them to that, authors are claiming that these are ready for archival. Similar issues for many workshop papers.

Comment by Alan Dix

Of course workshop position papers are often web 'archived' (my criteria (ii)), and some even heavily refereed ... indeed many people would prefer a CHI workshop paper on their CV than a more heavily refereed conference paper elsewhere ... I guess about brand, like a Nike holdall.

There is another orthogonal issue too which is about the level and surety of the process, which is pretty independent of the clarity and kind of criteria. You may have a poor quality journal that is using similar criteria to a better journal, but simply having a lower bar and perhaps, because of quality of reviewing, lower level of confidence. I'm sure both Fiat and Ferrari have quality control, just the level different.

In some ways I am happier with low quality journals that you now are low quality (and therefore readers apply caveat emptor) than high quality conferences, where it is easy for readers to assume high quality = all OK.

This is why I always feel that all reviewing processes should have a non-blind point, as a paper with a fantastic idea, but major methodological flaw, is fine if produced by an unknown person in and unknown institution (as readers will take it with a inch of salt), but should be rejected if from a major name in the field (as it is more liely to be taken as a pattern of how to do it by readers).

Alternatively anonymous refereeing + anonymous publication

... and none of this is about the absolute value, significance, etc. of the work, quality control is about stopping the bad apples, not making good ones.

Comment by Susanne Boll

I fully agree. Coming from the Multimedia community initially, I never understood this concept. SIGMM and the annual conferences will publish anything that undergoes a peer-review. Full papers are the most prestiguous one, short papers (4 pages) are for smaller contributions or more focused work. Workshops are THE platform to start new topics in the field and of course the work is peer-reviewed and published. For example, the Multimedia Information Retrieval run for several years and gained more and more interest in the field until it finally became an own conference.

I also found it strange this year that I reviewed a full paper for one year but had a deja vu as the work was already shown in the interactivity session the year before. This not only makes it difficult to judge novelty but also is contradictory to the blind review. Maybe have a look how other SIG conferences such as Multimedia handle it.

Comment Amanda Marisa Williams

I'm intrigued -- no time at the moment but it's bookmarked for later today. Def wanna have this conversation with some CHI veterans since I have some concerns about the archival/non-archival distinction as well.

Comment by Bo Begole
I think the crux of the issue is simply that we shouldn't use the term "archival" at all - as you point out, anything published on the DL with a DOI is "archived". It's an archaic term. More properly, we should use accurate terms to describe the level of review. CHI uses the terms "refereed" "juried" and "curated" for different levels http://chi2013.acm.org/authors/call-for-participation/#refereed which map to ACM categories of CHI refereed is roughly equiv to "refereed, formally reviewed" CHI juried is equiv to "reviewed" and CHI curated is roughly equiv to "unreviewed". CHI also uses the ACM criteria regarding republishability of content

Comment by Chris Schmandt
What Bo says is good, but this distinction is lost on the masses. It's a "CHI paper" no matter what venue. And even in the old day when we had the separate "abstracts" volume, only the few in the know could recognize the difference between the short ...
---
 
References
[1] Elba del Carmen Valderrama Bahamóndez and Albrecht Schmidt. 2010. A survey to assess the potential of mobile phones as a learning platform for panama. In CHI '10 Extended Abstracts on Human Factors in Computing Systems (CHI EA '10). ACM, New York, NY, USA, 3667-3672. DOI=10.1145/1753846.1754036 http://doi.acm.org/10.1145/1753846.1754036
[2] Elba del Carmen Valderrama Bahamondez, Christian Winkler, and Albrecht Schmidt. 2011. Utilizing multimedia capabilities of mobile phones to support teaching in schools in rural panama. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 935-944. DOI=10.1145/1978942.1979081 http://doi.acm.org/10.1145/1978942.1979081
[3] Tanja Doering, Bastian Pfleging, Christian Kray, and Albrecht Schmidt. 2010. Design by physical composition for complex tangible user interfaces. In CHI '10 Extended Abstracts on Human Factors in Computing Systems (CHI EA '10). ACM, New York, NY, USA, 3541-3546. DOI=10.1145/1753846.1754015 http://doi.acm.org/10.1145/1753846.1754015
[4] Enrico Rukzio, Albrecht Schmidt, and Antonio Krüger. 2005. The rotating compass: a novel interaction technique for mobile navigation. In CHI '05 Extended Abstracts on Human Factors in Computing Systems (CHI EA '05). ACM, New York, NY, USA, 1761-1764. DOI=10.1145/1056808.1057016 http://doi.acm.org/10.1145/1056808.1057016
[5] Enrico Rukzio, Michael Müller, and Robert Hardy. 2009. Design, implementation and evaluation of a novel public display for pedestrian navigation: the rotating compass. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, New York, NY, USA, 113-122. DOI=10.1145/1518701.1518722 http://doi.acm.org/10.1145/1518701.1518722
[6] Albrecht Schmidt, Hans-W. Gellersen, and Christian Merz. 2000. Enabling Implicit Human Computer Interaction: A Wearable RFID-Tag Reader. In Proceedings of the 4th IEEE International Symposium on Wearable Computers (ISWC '00). IEEE Computer Society, Washington, DC, USA, 193-194.