A precedent of subjectivity

Mike Montiero recently posted an essay to Medium “One person’s history of Twitter, from beginning to end“. It’s a good piece, but what really struck me was this referenced Tweet from Twitter co-founder Biz Stone:

Tweet by Twitter co-founder Biz Stone, reading: 'Mike, this is the hardest time to stick to principles. The easy way out would be to ban. That sets a precedence of subjectivity.'

This is one of the saddest tweets I’ve ever read.

Stone talks about setting “a precedent of subjectivity” as though subjectivity is something that can possibly be avoided. Everything is subjective; the relevant questions are how much and in what way.

Usually I give people a pass for over-simplifying things on Twitter, but if Stone has trouble expressing himself in 140 characters he has only himself to blame. More seriously, this over-simplification is the source of so many problems that sites like Twitter, Google, and Facebook have been having. They want to be content-neutral, a platform rather than a publisher, a (non-regulated!) utility – in other words, objective.

They know their platforms aren’t actually neutral or objective, not entirely. They know that their abuse teams are staffed by subjective humans and their algorithms are steered by subjective humans and learn from subjective, human-created data. It is impossible to design an open platform that does any kind of data filtering or prioritization without having to make judgment calls. Stone knows. They all know.

So why deny it? My guess is that it’s an avoidance mechanism. As long as we’re debating ‘Should social media sites make subjective decisions about content?’ we’re not getting to ‘What kinds of subjective decisions should social media sites make?’ and ‘To what degree should social media sites be subjective?’ And so the sites can continue to make their subjective decisions in private, without formal or informal oversight.

I want to be clear: there’s real value in the way Google and Facebook and Twitter have prioritized objectivity. I’m pretty sure platforms which didn’t even try for objectivity would be even more alienating and frustrating to use. But ‘better than terrible’ is not enough.  When your sites are changing the course of history you have a moral obligation to tackle the really hard stuff, and to do so openly and accountably.

The precedent of subjectivity was set a very long time ago. It’s about time we got into the details.

Science vs Software

Andrew Gelman has a brief post up on his blog comparing the way bug reports in open source software are received to the way many researchers respond to criticisms of their work.  The comments there are good, and cover my first reaction, which was, “Developers respond well to bug reports?”  But that’s a bit tongue in cheek.  I do think that, overall, developers are a bit more responsive to bug reports than scientists are to published criticisms of their work.  Here are my theories as to why that is:

  • Bug reports are not analogous to published criticisms. Bug reports are the primary way for people to give feedback to the maintainers of a project, while historically, much criticism of research has been through less formal mechanisms such as email, questions at conference or post talks, and face-to-face at lab meetings, conferences, etc.  I have never seen a scientist respond poorly to a critique at a lab meeting or a poster session, for instance.  If you factor in these other interactions, the average emotional response to criticism might be more equal.
  • Research careers are measured in papers.  The academy clings to its traditions, and that’s one of the big ones.  Papers are difficult to amend in significant ways, and they take a lot of work to produce.  If software developers could only push to production once every year or so, I bet we’d find receiving them much more stressful!
  • Most bug reports don’t existentially threaten software projects.  I mean, I’m sure it’s happened (and now I’m kind of curious when and how!).  But many critiques of published research are suggesting that reported effects may not exist, which is a much bigger blow than “hey your app keeps freezing”.  So long as researchers are lauded not for the quality of the work they do but the significance of the results they achieve, this kind of critique is going to feel like a threat, especially to non-tenured researchers and scientists in other types of precarious positions.

If I have one critique of the open science movement, which I otherwise endorse and consider myself a part of, it’s that we focus too much on the behavior of individual researchers and not enough on the systems which motivate that behavior.  It’s not that developers are better people than scientists.  It’s that the systems developers operate within are set up to reward and punish different things.