Many academic conferences award one or more “best paper”, “best demo” or “best poster” awards. The awardees are either selected by the program committee or by an anonymous audience vote.
However, in my opinion, we should get rid of these awards.
For three reasons I think these are a bad idea.

1. The big question regarding such awards is: “In which way are they useful for the community”. Like in education, an intervention (the award) should have a lasting positive effect either on the awardee or the community. Sure, the individual author who receives an award might feel happy for a short while. However, this positive effect might also be reached by just patting them on the back and saying “Great Work!”. There is no evidence that such awards lead to higher achievements. Quite to the contrary, a number of publications claim that awards and incentives actually have overall negative effects on individuals and communities, lowering their performance [1].

2. Awards are a bad metric for great research. Bartneck and Hu have pointed out that (on average) papers which got a CHI best paper awards did not get more citations than a random sample [2]. It seems that even a commitee of experts is unable to predict which papers will have the highest impact. If detecting great work is not even possible for papers with their fixed structure, why should it work for posters or demos? Especially demos are so diverse that a one-size-fits-all award is plain wrong. Is an artful, thought-provoking demo better than a demo of a novel, extremely versatile sensing technology?

3. Undersampling is another problem. How many of the conference attendees have seen all posters and demos? I would guess that, for any poster session, not a single attendee has read all poster titles. Likewise, it is hard to judge the quality of a demo without understanding it. For understanding a demo, you have to try it out for some time. With hundreds of other attendees trying out the same demos, there is just no time for this. Therefore, almost all votes for a best demo or poster can consider only a small subset. Nearly noone is able to make a qualified decision. Variables like poster/demo placement or group dynamics might have more of an effect on the votes than any kind of actual “quality” of a poster or demo.

Overall, best poster/paper/demo awards are neither shown effective nor valid nor at least fair. Why are we then clinging to them?

The TEI conference – which has a very intensive and diverse demo and poster session – has opted not to have any awards – for more or less these reasons.

[1] Kohn, A. (1993). Why incentive plans cannot work. Harvard Business Review, September 1993

[2] Bartneck, C., & Hu, J. (2009). Scientometric Analysis Of The CHI Proceedings. Proceedings of the Conference on Human Factors in Computing Systems (CHI2009), Boston, pp.699-708

I’m attending ITS 2010 – the ACM International Conference on Interactive Tabletops and Surfaces 2010 in Saarbrücken, Germany. This is a short collection of interesting stuff I’ve seen and heard on day 2 (Monday, 6. November 2010).

(The demo and poster session is like a huge, dark playground with (literally) tons of amazing touch interfaces.)

Monday was the first day of paper presentations. There was a wealth of papers on several topics. Therefore my account is very selective. You can get all papers at the conference website.

The day started very relaxing with “Tafelmusik“, two musicians with a digital audio sequencer and a table full of objects that make sounds. See their website for a photo. By sampling them and continuously replaying these sounds they created a sound landscape – sometimes soothing and sometimes fascinating.

Brad Paley gave a keynote covering a wide range of topics but centering about ways to visualize information. Some of his claims:

• “CHI” considered harmful: instead “computer mediated human-to-human interaction”
• Color is bad for encoding data
• Consistence *impairs* performance
• 15:1 increases in information density, 20:1 speed-ups can be easily reached
• “users” considered harmful

While Brad did not explicitly say so, I think in their entirety these claims only apply to UIs for expert users, however.

Afterwards, Malte Weiss (RWTH Aachen) presented “BendDesk: Dragging Across the Curve” [PDF]. He and Simon Voelker built a desk with an interactive surface bent partly upwards. Malte kindly mentioned Curve – our research on this topic. We are currently figuring out how to connect both prototypes for remote interaction.

In the same session, Yvonne Jansen presented MudPad [PDF], a tactile display using ferrofluid and magnets.

Antti Virolainen presented an interactive surface made out of ice (FTIR in ice is probably not possible).

Hrvoje Benko (Microsoft Research) presented another spherical multitouch surface – but this time a large dome where you walk inside [PDF]. Interesting link from his talk: worldwidetelescope.org

In the afternoon, Dietrich Kammer (TU Dresden) presented an interesting theoretical framework for describing gestures [PDF].

For me, the demo and poster session is always the highlight of a conference. At ITS 2010 it took place at DFKI. There was a wealth of really cool demos and interesting posters. As I had to present my own poster (“Some Thoughts on a Model of Touch-Sensitive Surfaces” [PDF]), I did not find time to have a look at every demo. However, there was an amazing mixture of art, high-tech hardware, and applications. See the photos on Facebook!

While I liked some demos and posters more than others, I did not fill out my voting sheet for best poster or demo. More on this later.

Photo taken from the official ITS 2010 Facebook album:

I’m attending ITS 2010 – the ACM International Conference on Interactive Tabletops and Surfaces 2010 in Saarbrücken, Germany. This is a short collection of interesting stuff I’ve seen and heard on day 1 (Sunday, 6. November 2010).

(people testing the multitouch device they just built)

Sunday started with four tutorials:

Sheelagh Carpendale (University of Calgary) gave an introduction into qualitative evaluation and observation of tabletop interaction. We did an experiment where she would tell us a story and we should draw a continuous line somehow representing the mood of the characters in the story. I found it quite interesting that there were completely different looking drawings but some drawings looked quite similar. I wonder whether you could find out which persons have a similar understanding of a story. Looks like a great method for match-making.

Uli von Zadow (Archimedes Solutions) gave a very interesting overview of multitouch sensing APIs and implementation details of common processing stages (e.g. it is not a good idea to put successive processing stages on different processors as this makes the processor cache pretty much useless).

Florian Echtler (Hochschule München) presented an overview of multitouch sensing techniques [I did not attend the talk as I’m pretty familiar with his work].

And finally, Anne Roudaut (HPI Potsdam) organized a really cool “build your own multitouch” session where about 50 participants built a simple touch-sensitive surface using FTIR with visible light and a cheap webcam. You can find the instructions online on her Acrylicpad page. They got really cheap (5 EUR) webcams on eBay. However, they only worked with Windows XP – and of course Linux 🙂

Afterwards we had a nice get-together at the Ratskeller in Saarbrücken.

Photo taken from the official Facebook album:

Day 1, Day 2