The yin and yang of questions and answers

In a previous post, I wrote that asking questions is harder than answering them, although I qualified that in a big way with “answering [questions] involves going back over and over again and updating our hypotheses, which makes answering questions feel hard”.  I want to revisit this claim.

Some of you may be familiar with the “reproducibility crisis” happening in the sciences, where many popular and well-known results have failed to replicate.  But what does failure to replicate mean?

Maybe it means that there was something wrong with the original study.   Maybe it means that there was something wrong with the replication.  But those aren’t the only options.  As nobel laureate psychologist Daniel Kahnamen wrote in an open letter to the scientific community:

In the myth of perfect science, the method section of a research report always includes enough detail to permit a direct replication. Unfortunately, this seemingly reasonable demand is rarely satisfied in psychology, because behavior is easily affected by seemingly irrelevant factors.

Note that underspecification of methods is an issue in all sciences.  Psychology just has a particularly rough time of it because psychology itself, like other soft sciences, is so underspecified.  Behavior is affected by seemingly irrelevant factors which are actually relevant previously unspecified factors.

In a better world, replication would be a collegial and common process involving many back-and-forths between originators and replicators.  Each replication could help identify new factors that turn out to be surprisingly relevant.  Eventually the hypothesis and methodology would be specified enough to permit consistent replication, at which point we’d have both our question and our answer.

This example makes clear that asking and answering questions are not two separate activities.  They are intertwined, at least when the questions and answers are new.  So it makes no sense to say, “Asking questions is harder than answering them” or vice versa, because you can’t do one meaningfully without also doing the other.

FYI: to read more about replication, try this article I wrote back in 2014 on the Open Science Collaboration blog: What we talk about when we talk about replication.


Hard and soft sciences

Back when I was a research scientist, I straddled the boundary between “hard” and “soft” sciences.  I did social psychology, which is a pretty soft science as sciences go, but I paired it with biology and physiology in general and endocrinology in particular, which meant getting a taste for some of the harder stuff.

I have never particularly liked the terms “hard” and “soft”, though, because it’s too easy to conflate them with “hard” and “easy”.  There’s a saying that goes: the soft sciences are easy to do poorly and hard to do well.  They are easier to do poorly than the hard sciences, and harder to do well than the hard sciences.  Here, have a chart:

What’s going on here?  The hard sciences are better developed than the soft sciences, so it’s clearer when someone’s making obvious mistakes, cutting corners, or making under-supported claims.  That makes it difficult to do poor work.  It’s also difficult to good work, of course.  The easiest thing to do in the hard sciences is to meet a minimum level of competency and do solid but uninspiring work.

Meanwhile in the soft sciences there’s questions even about the field’s basics.  There’s still a minimum level of competency, but it’s much less stringent than in the hard sciences.  So sloppy researchers tend to end up in the soft sciences.

Here’s another way to approach the hard/soft distinction. What’s easier, formulating questions or answering them?  It’s almost always easier to do the latter, provided you’ve very clearly and specifically formulated your question.  Of course we seldom do get our questions right on the first try, and so answering them involves going back over and over again and updating our hypotheses, which makes answering questions feel hard.  But the hardest parts of answering questions are really secretly still about asking them.

In the hard sciences, it’s easier to clearly and specifically formulate questions because so much knowledge has already been established.  Isaac Newton famously said (paraphrased) ‘If I have seen further it is only by standing on the shoulders of giants.’  The hard sciences are full of giants, with shoulders for modern researchers to stand on.  The soft sciences are by and large still on the ground.

For this reason, I prefer the terms “developed” vs “undeveloped” sciences.  I think it comes closer to the essential difference.

Note: this post has an update/correction post.

Shades of gray

What shade of gray would you say this is?

Considered in isolation, and without the help of a photo manipulation platform to tell you the precise CMYK values, it’s hard to say.  We reach naturally for points of comparison.  It’s lighter than a slate gray, we might say, but darker than a gainsboro gray.  Cooler than warm gray but warmer than cool gray.  Is it lighter than light gray?

Maybe?  Probably?  Let’s take a closer look:

That’s our custom gray on the left, and light gray on the right.  So it’s just a smidge lighter than light gray, but we can only tell when we put them right up close.  Put it up against a different gray, and it looks darker rather than lighter:

This is of course a metaphor.

When it came out last week that Al Franken had sexually harassed and assaulted Leeann Tweeden, people naturally reached for the most obvious point of comparison: the other Senator/Senate candidate recently accused of sexual assault, Roy Moore.

Some people have claimed that Moore and Franken are similar.  They both have used their power to violate the boundaries and consent of women.  Others have claimed that they’re dissimilar: that when you put Franken’s behavior up against Moore’s systematic stalking and molestation of girls as young as fourteen years old, he might as well be a saint.

But is Moore a useful point of comparison?  It’s not like Franken’s running against Moore.  And “the ideal public servant” is not all that useful either.  After all, no public servant is perfect.

So what is a useful comparison for Franken’s behavior?  Given that the question on everyone’s lips is whether Franken should resign, I submit the following point for comparison: the minimum acceptable standard for a US Senator.

Only 15 Senators have ever been expelled from Congress, the majority for making war against the United States by supporting the confederacy.  A larger number have resigned rather than face expulsion. Most of these cases were about bribery and corruption, but some were due to sexual misconduct.  Take for example Bob Packwood of Oregon, who resigned when the Senate Ethics Committee unanimously recommended his expulsion for gross sexual misconduct.  The chair of that committee, none other than Mitch McConnell, spoke about it four years later during the Clinton impeachment:

During the Packwood debate, we made the tough choice. And, I have to say, that decision was one of the most difficult things I have ever had to do in my career in public service. To recommend expelling from the United States Senate a colleague, a member of my own party, and most importantly, a friend with whom I had served in the Senate for over a decade.

We sent a clear message to the nation that no man is above the law. That no man is so important to the well-being of our strong and prosperous nation that we have to compromise the fundamental, founding principles of truth and justice. We chose to rise above, not sink below. Rather than change our standards, we changed our Senator.

Let me also make a political point, here. We Republicans were aware during the Packwood debate that we would likely lose that Senate seat if Senator Packwood was removed from office. So, we had a choice: Retain the Senate seat or retain our honor. We chose honor, and never looked back.

I think that the United States Senate has a clear choice today. Do we want to retain President Clinton in office, or do we want to retain our honor, our principle, and our moral authority?

For me, and for many members in my impeachment-fatigued party, I choose honor.

What happened?  Twenty years later, McConnell has chosen dishonor over and over again, not least by supporting the leader of his party, Donald Trump, who has been accused of assault and harassment by 15 different women and has been caught bragging about harassing women on tape.  Perhaps if you asked McConnell, he’d say that Democrats started the moral race to the bottom by refusing to hold Clinton to the same standard as Packwood.  But what is the “standard” to which Packwood was held?  One could likely articulate a standard for behavior that Clinton passed while Packwood fell short of.

Packwood and Clinton and Moore and Franken are all different people who committed different types of wrongs. The same is true of Trump, and Dennis Hastert, and Anthony Weiner, and the many other elected officials who have been accused or convicted of harassment, assault, or abuse.  The question is not how they compare with each other but how they measure up to our standards for public officials, and whether we are strong enough to enforce our standards when people fall short.

I believe that our elected officials should be held to high standards.  Specifically, I believe that any type of sexual assault, abuse, or harassment should be disqualifying.  I am open to the idea that those who have done these things in the past and have made reparations should be able to hold office, though I’m not aware of any official or candidate who meets that description. I’m also open to the idea that those accused deserve an impartial investigation, although the strength of the evidence against both Moore and Franken make that point moot.

This is all a very long-winded way of saying I think Franken should resign, but I think it’s worthwhile to sketch out the framework through which I came to that conclusion, so that I can hold myself to it in the future.

Compromise and its discontents

It’s easy to complain about “purity politics”.  It’s easy to complain about “neoliberals” and “sellouts”.  But we live in hard times, and the easy route’s not going to get us anywhere.

Here are two things that are both generally true: you need to compromise sometimes and sometimes you need to stick to your principles.  This question is, which times are which?

I thought about this recently when I read, back to back, two stories.  The first was about Cyrus Vance, the corrupt Manhattan DA who recently ran unopposed as a Democrat.  “Get him out of office!” I thought, angrily.  “Even if he gets replaced by a Republican!”  Then, I read about the DFA withdrawing their endorsement of Ralph Northam because he came out against sanctuary cities.  “That’s terrible,” I thought, about both Northam and the DFA.  “I disagree with him on this issue but we’ve still got to get him elected.”

Whenever a Democrat does something wrong (or a Republican does something right) you will find people on the left arguing over whether or not we should support them.  There’s always someone saying we need to take our allies where we can get them, and someone saying that this or that is a bridge too far.

This dynamic is especially heated right now, thanks to the Republican party.  They’re fueling both groups.  The compromisers say, “Look, look at the other option.  We need to do all we can to resist these hateful, violent, reckless people.”  And the principle-stickers say, “Look, look at what happened to Republicans when they embraced that mindset.”

So how can we figure out when to compromise, and when to stick to principle?  We need to get better at articulating the specific context that’s driving our arguments, rather than falling back on statements like “We can’t sacrifice our principles!” or “We need to compromise sometimes!”  Not only are those statements obviously true, they read like an attack.  No one wants to be told they’re unprincipled or impractical.  No one needs to be told that, either.

In that spirit, here is just a starter list of things to consider when making a specific judgment call:

  1. What good/bad things will happen if we don’t compromise?  What good/bad things will happen if we do?
  2. Of the things that might happen, how much of them will affect us personally?  If most of them will affect other people, what do those people say about the dilemma we’re in?
  3. How set-in-stone is this decision?  Are we electing someone for a one year term or a six year term?  How hard will it be to roll back this legislation?
  4. Will our actions change the fundamental structure within which we act?  If we support someone who is corrupt, or who won’t enforce constitutional checks and freedoms, will we have the institutional tools to take them down if we change our minds, or is now our best or only chance?

A perfect circle

Chalk Corridor by fdecomite 6/25/2013 CC BY 2.0

Objective truth exists in the way that a perfect circle exists.  They’re both useful constructs, helpful for comparison and for motivation, but we must be careful in how we apply them.

Statements about truth that are unhelpful:

  • “What I’m saying is objectively true.”  (“What I’m drawing is perfectly circular.”)
  • “What you’re saying is not objectively true.”  (“What you’re drawing is not entirely perfect.”)
  • “Nothing is ever entirely objectively true, so we might as well give up caring about truth.”  (“No circle will ever be perfect, so we might as well give up trying to make circles.”)

Statements about truth that are helpful:

  • “This statement is as close to objectively true as I can get it.”  (“This circle is as perfect as I can draw it.”)
  • “Your statement would be closer to objective truth if you X.”  (“Your circle would be rounder if you fixed X.”)
  • “This statement is more true than that statement.”  (“This circle is more round than that circle.”)
  • “That’s not even trying to be true, that’s a damn lie.”  (“That’s… not a circle.  That’s a square, I think.”)

Go forth and draw, my friends.


A flight out sideways

This is a short and subtle piece by Diana Senechal about sexual harassment claims and our response to them.  I by and large agree with it, but I like it mostly for the poem (or, well, stanza of a poem) it introduced me to, Robert Frost’s The Wood Pile:

A small bird flew before me. He was careful
To put a tree between us when he lighted,
And say no word to tell me who he was
Who was so foolish as to think what he thought.
He thought that I was after him for a feather—
The white one in his tail; like one who takes
Everything said as personal to himself.
One flight out sideways would have undeceived him.