Two Quotes on Research and Teaching

In a Hacker News discussion, I just stumbled upon two quotes which – in my view – beautifully capture the essence of the symbiosis of academic teaching and research:

“If you pay a man a salary for doing research, he and you will want to have something to point to at the end of the year to show that the money has not been wasted. In promising work of the highest class, however, results do not come in this regular fashion, in fact years may pass without any tangible result being obtained, and the position of the paid worker would be very embarrassing and he would naturally take to work on a lower, or at any rate a different plane where he could be sure of getting year by year tangible results which would justify his salary. The position is this: You want one kind of research, but, if you pay a man to do it, it will drive him to research of a different kind. The only thing to do is to pay him for doing something else and give him enough leisure to do research for the love of it. “

— J. J. Thompson

“I don’t believe I can really do without teaching. The reason is, I have to have something so that when I don’t have any ideas and I’m not getting anywhere I can say to myself, “At least I’m living; at least I’m doing something; I am making somecontribution” — it’s just psychological.

When I was at Princeton in the 1940s I could see what happened to those great minds at the Institute for Advanced Study, who had been specially selected for their tremendous brains and were now given this opportunity to sit in this lovely house by the woods there, with no classes to teach, with no obligations whatsoever. These poor bastards could now sit and think clearly all by themselves, OK? So they don’t get any ideas for a while: They have every opportunity to do something, and they are not getting any ideas. I believe that in a situation like this a kind of guilt or depression worms inside of you, and you begin to worry about not getting any ideas. And nothing happens. Still no ideas come.

Nothing happens because there’s not enough real activity and challenge: You’re not in contact with the experimental guys. You don’t have to think how to answer questions from the students. Nothing!

In any thinking process there are moments when everything is going good and you’ve got wonderful ideas. Teaching is an interruption, and so it’s the greatest pain in the neck in the world. And then there are the longer period of time when not much is coming to you. You’re not getting any ideas, and if you’re doing nothing at all, it drives you nuts! You can’t even say “I’m teaching my class.”

If you’re teaching a class, you can think about the elementary things that you know very well. These things are kind of fun and delightful. It doesn’t do any harm to think them over again. Is there a better way to present them? The elementary things are easy to think about; if you can’t think of a new thought, no harm done; what you thought about it before is good enough for the class. If you do think of something new, you’re rather pleased that you have a new way of looking at it.

The questions of the students are often the source of new research. They often ask profound questions that I’ve thought about at times and then given up on, so to speak, for a while. It wouldn’t do me any harm to think about them again and see if I can go any further now. The students may not be able to see the thing I want to answer, or the subtleties I want to think about, but they remind me of a problem by asking questions in the neighborhood of that problem. It’s not so easy to remindyourself of these things.

So I find that teaching and the students keep life going, and I would never accept any position in which somebody has invented a happy situation for me where I don’t have to teach. Never.”

— R.Feynman


Thoughts on Peer Review in the CHI Community

I just filled out the CHI reviewer pre-review questionnaire. The final question asks about general thoughts on the CHI review process. I have documented my answer below. None of the potential improvements I mention are really novel – they have been implemented at other conferences or journals before.

Edit (01.10.2015, 09:15 UTC+2): I would not expect such changes to be made for CHI 2016 or 2017. Instead, some of these changes could be tried out at smaller conferences first in order to work out a usable implementation.

(For the record: this is not about open access. While I am a fan of open source/science/access/…, I think that the ACM’s approach (much freedom for authors, affordable access to the Digital Library and individual papers, offering an OA option) is very reasonable.)

In general, I would like to see four changes to the current review process:

Post-Publication Peer Review

I find the currently practiced pre-publication peer review quite problematic. I have reviewed or otherwise seen plenty of papers that contained interesting findings but were rejected (sometimes rightfully) due to some flaw or another.
A huge share of these rejected papers was never published in another venue, the insight contained in them (and errors from which others could learn) lost to the community. Many other papers were only published one or more years later – thereby delaying all research that could build on it or try to replicate its findings.

I would very much like CHI to adopt a post-publication peer review or a similar approach that improves speed and visibility. For example, a process similar to alt.chi (in some ways) would be great, where all submitted papers are made public or semi-public and peer review then selects those that are to be presented at the conference. Ideally, reviews would also be published for all papers. This would also make it harder for authors to re-submit their paper unaltered to another conference without addressing the issues mentioned by the reviewers. I hate when I give extensive feedback to a paper and the authors do not even bother to fix the spelling errors that I pointed out when they submit it to another conference.

Open Peer Review

Quite often I read a paper which incorrectly quotes a paper of mine or which omits important related work. With the current review process, the quality of the reviews depends on the ACs ability and desire to find the right reviewers.  Allowing any researcher to provide a review for a submitted paper would make sure that domain experts can chime in and point out flaws or interesting use cases that the other reviewers missed. This has already been tried at alt.chi, too.

Modular Peer Review

The standard PCS review form asks the reviewer how they would rate their expertise on a four-point Likert scale. This is a rather simplistic measure. While I feel very competent to judge novelty or validity of a sensing technique, I can not honestly tell whether the correct statistical tests were used in the evaluation of this technique. Similarly, I won’t be able to tell whether all relevant related work has been cited in a paper on design techniques but can certainly offer my opinion on further applications or spelling errors.

I would prefer to describe my expertise more accurately and also focus only on certain aspects of a paper as a reviewer. By telling the AC that I do not know enough about statistics, I give them the opportunity to find another reviewer who does. Furthermore, the AC could also assign a subset of reviewers to individual aspects of a paper (e.g., related work, writing style, experimental setup, statistical tests, technical correctness, replicability). This could avoid duplicated work and simultaneously increase the quality of the reviews.

Author-Reviewer Collaboration

Reviewers point out weaknesses and suggest improvements in a paper. Given the significant amount of time some reviewers invest in reviewing a paper, and given that their suggestions may significantly improve a paper, it might be a good idea to offer a way of attributing their contributions. For example, reviewers could be mentioned by name in the Acknowledgements section (if they want) or they could become co-authors of the final submission. This would certainly change the character of the review process and introduce some problems. While the aforementioned changes could already be implemented for next year’s CHI conference, this final suggestion certainly needs more discussion and refinement to get it right. One option could be for reviewers to ‘fork’ a submission and submit ‘pull requests’ with proposed changes similar to software development processes on GitHub and similar platforms. (In general, a version-controlled approach to paper writing, including authorship attribution for each sentence sounds quite interesting.)

Given that the program chairs of recent CHI conferences have put much effort into evolving the conference series, I am optimistic that some of these proposed changes will be implemented in the near future. And frankly, it would be very fitting for the CHI community to be at the forefront of academic collaboration and publishing.

Protected: Tips for Student Volunteers Chairs at Medium-Sized Conferences

This content is password protected. To view it please enter your password below:

META: imported old blog posts and comments – images broken at the moment

I just found time to import my old blog formerly hosted on .

Posts and comments have been imported flawlessly.However, all images are currently broken until I fix the links.

DFG-Empfehlung Nr. 17 – eine Analyse und ein Vorschlag

Am 3. Juli 2013 hat die Mitgliederversammlung der Deutschen Forschungsgemeinschaft (DFG) mehrere Ergänzungen zu ihren Empfehlungen zur Sicherung guter wissenschaftlicher Praxis beschlossen. Diese wurden am 4. Juli veröffentlicht. Im Vorfeld hatten ich und andere Befürchtungen geäußert, dass insbesondere eine neue Empfehlung – Nr. 17 – den offenen wissenschaftlichen Diskurs einschränken könnte. Stefan Heßbrüggen hat am 2. Juli 2013 auch eine Petition gegen diese neue Empfehlung und die ihr zugrundeliegende Empfehlung der Hochschulrektorenkonferenz (HRK) gestartet.

Haben sich nun unsere Befürchtungen bewahrheitet? Darauf hat noch niemand eine Antwort.
Die DFG-Empfehlung ist weit weniger drastisch formuliert als es noch die HRK-Empfehlung war. Zumindest der zweite Teil der Empfehlung ist aber trotzdem unnötig, stiftet Unsicherheit, und beschädigt Vertrauen.

Im Folgenden versuche ich, zu klären, was Empfehlung Nr. 17 für den allgemeinen wissenschaftlichen Diskurs, für die Aufklärung von Fehlverhalten und für das Vertrauen in das Wissenschaftssystem bedeutet.

14.07.2013: Stefan Heßbrüggen diskutiert auf Carta intensiv den Konflikt zwischen DFG und HRK bezüglich Whistleblowern.
12.07.2013: Nachtrag: wie schon am 10.7. korrigiert (siehe unten), entspricht die verabschiedete DFG-Empfehlung der Beschlussvorlage des DFG-Senats.
12.07.2013: In einem merkwürdigen Editorial vom 10.07.2013, das mehrere faktische Fehler enthält, hat die Fachzeitschrift Nature die DFG-Empfehlungen kritisiert.
10.07.2013: Die DFG hat in einer Pressemitteilung noch einmal explizit erklärt, dass sich die Vertraulichkeit rein auf das Ombudsverfahren bezieht, nicht auf sonstigen wissenschaftlichen Diskurs.

Read the full post »

Vertrauen und Vertraulichkeit

Wissenschaft basiert nicht auf blindem Vertrauen, sondern auf immanentem Zweifel.
Wissenschaft braucht aber auch Vertrauen, intern und extern.
Intern erleichtert es uns Wissenschaftlern die Arbeit, wenn wir unseren Fachkollegen ein Grundvertrauen entgegenbringen können, und nicht dauernd nagende Zweifel an ihrer Redlichkeit hegen.
Extern braucht Wissenschaft das Vertrauen der Gesellschaft, die sich ja auf unsere Erkenntnisse verlassen und unsere Forschung finanzieren soll.

Dieses Vertrauen untereinander und der Öffentlichkeit in uns ist immens viel wichtiger als mögliche Reputationsverluste eines Einzelnen.

Transparenz  schafft Vertrauen – und nicht Kommissionen, die im Geheimen entscheiden, wer weiterhin vertrauenswürdig ist, und wer nicht.

Alleine die Tatsache, dass HRK und DFG geschlossen der Heimlichkeit den Vorzug geben, nimmt mir wieder ein Stück Vertrauen in unser Wissenschaftssystem.


Warum die neue DFG-Empfehlung Nr. 17 der Wissenschaft schadet

In Ihrer Mitgliederversammlung vom 1.-3. Juli 2013 will die Deutsche Forschungsgemeinschaft (DFG) ihre “Vorschläge zur Sicherung guter wissenschaftlicher Praxis” um einen weitere “Empfehlung” (Nr. 17) ergänzen: Vorwürfe wissenschaftlichen Fehlverhaltens dürfen nur noch den Ombudspersonen mitgeteilt werden und nicht mehr öffentlich diskutiert werden.
Dies wird anonyme Beschuldigungen nicht stoppen.
Wer aber unter seinem bürgerlichen Namen öffentlich wissenschaftliches Fehlverhalten anprangert, verstößt in Zukunft – nach Meinung der DFG – selbst gegen die Regeln der guten wissenschaftlichen Praxis. Whistleblower können dadurch von der DFG-Förderung ausgeschlossen werden – mit dramatischen Folgen gerade für junge Wissenschaftler.
Die neue Empfehlung Nr. 17 ist ein Gummiparagraph, der den wissenschaftlichen Diskurs in Deutschland lähmt und die Aufklärung von wissenschaftlichem Fehlverhalten verhindert.


10.07.2013: Die DFG hat in einer Pressemitteilung explizit erklärt, dass sich die Vertraulichkeit rein auf das Ombudsverfahren bezieht, nicht auf sonstigen wissenschaftlichen Diskurs.
08.07.2013: Ich habe meine Meinung zur Empfehlung 17 noch einmal als Blog-Eintrag dokumentiert.
04.07.2013: Die DFG hat die neue – gegenüber der HRK-Formulierung weniger scharfen – Empfehlung 17 heute veröffentlicht (PDF).
02.07.2013: Es gibt jetzt eine Petition an die DFG auf Ich warte noch ab, was morgen auf der DFG-Mitgliederversammlung überhaupt beschlossen wird.
01.07.2013: Erbloggtes erklärt eloquent, weshalb Ombudssystem und öffentlicher Diskurs zusammengehören.
01.07.2013: English blog post “Whistle blowing in the German University: A Regulatory Scandal in the Making” by Stefan Heßbrüggen-Walter.
29.06.2013: Link zu Blogeintrag von Benjamin Lahusen eingefügt.

Read the full post »

Schavan, schludrig

Read the full post »

How would you like to be remembered by the people who will live in 2200?

At TEI 2008, Hiroshi Ishii gave a very interesting keynote, presenting highlights of the incredible amount of research he had been involved over the last decades.

However, it was the last slide that has stayed in my mind since then:

Wiley’s Major Reference Works available for free (sort of)

Update (08. March 2012): Wiley has informed me that they just fixed the issue.

70 GB of digital content from Wiley’s Major Reference Works can be downloaded for free from Wiley’s web server. This is probably not intended, as Wiley still charges hundreds or even thousand of Euros for each one. I notified them of this fact several times during the last two months. They do not seem to mind. In this post I explain how I found out about this, how one can download the content, and why this is probably legal (at least in some jurisdictions).
Read the full post »