Did a Change in Ballot Design Cost Obama NH?

The puzzle from New Hampshire is that the polls conducted in the last couple days were so far off the final result. People have offered lots of theories to explain this, but here's one that actually has experimental data to back it up, courtesy of Stanford professor Jon Krosnick, who has conducted substantial research on ballot design and voting behavior:

Without a doubt, a big source of the discrepancy between the pre-election surveys and the election outcome in New Hampshire is the order of candidates' names on the ballot and in the surveys. Our analysis of all recent primaries in New Hampshire showed that there was always a big primacy effect--big-name, big-vote-getting candidates got 3 percent or more votes more when listed first on the ballot than when listed last.

Until this year, New Hampshire rotated candidate name order from precinct to precinct, which allowed us to do that analysis. This year, the secretary of state changed the procedure so the names were alphabetical starting with a randomly selected letter, in all precincts.

The randomly selected letter this year was Z. As a result, Joe Biden was first on every ballot, Hillary Clinton was near the top of the list (and the first serious contender listed) and Barack Obama was close to last of the 21 candidates listed.

Thus, I'll bet that Clinton got at least 3 percent more votes than Obama simply because she was listed close to the top.

Most, if not all, of the pre-election telephone polls rotated name order from respondent to respondent, which meant name order did not distort their overall results. Failing to incorporate the name order effect that probably happened in the voting booth is therefore probably partly responsible for the polls' inaccuracy.

More importantly, if New Hampshire had rotated name order in the voting booth as it has always done in the past, the race would probably have been too close to call without a recount and might even have been an Obama victory.

See the ballot for yourself here. This sounds very convincing, although I would very much like to see the details of the New Hampshire analysis, which has apparently not been published. As a tenured Stanford professor, he gets the benefit of the doubt from me that his methodology is sound, and a quick look at some of his other work shows that he is a serious quantitative researcher.

Krosnick summarizes the literature, law, and his peer-reviewed research from other states on the "name order" phenomenon in this 2004 book chapter. The 3 percent advantage to the first big-name candidate appearing on the ballot appears consistent with his results from other states.

What's remarkable is that the earlier NH primaries, for which name order was randomized for each precinct, constitute the perfect true experiment to evaluate the effect of name order in NH primaries. There's no need to try to speculate what results from other places might mean for the state in question. This is the kind of thing that makes social scientists salivate.

The book chapter makes a number of interesting points. First, the importance of name order has been recognized for a long time. The chapter quotes Woodrow Wilson from 1910 complaining that when a voter is faced with a long, complicated ballot, "in nine out of ten cases, he will simply mark the first name under each office, and the candidates whose names come highest in the order will be elected." And a 1934 study noted the following:

Much more important than the order of offices on an election ballot is the order in which the names of the candidates appear in office group ballots. This is particularly true in direct primary and nonpartisan elections, and is of most importance in cases where several persons are to be elected to the same office. The position at the top of a list of candidates is of material help to the candidate thus favored....

Second, although psychologists haven't nailed down exactly why this happens, such "primacy effects" whereby the first option is favored have been found in many experiments:

For example, when students take multiple-choice knowledge tests, they are biased toward selecting answers offered early in a list so they tend to answer items correctly more often when the correct answer is listed first than when it is listed last .... When people are told that an experimenter will imagine a series of questions and they should guess which of a set of offered response choices is the correct answer, people tend to select the first ones listed.... When people are asked to taste a set of beverages or foods (e.g./ four brands of beer) and select their favorite, they are biased toward choosing the first one they consider.... Voters may well manifest the same sort of bias in elections.

The chapter suggests several cases in which primacy effects might be acute. The following appears most applicable to the NH case:

If a voter has paid close attention to a well-publicized race and learned both favorable and unfavorable information about all competing candidates, the voter may feel substantial ambivalence, finding it difficult to choose between the competitors. In this situation, people may settle for the first name they see because they cannot make a choice on substantive grounds, despite having plenty of information about the competitors.

It appears that Obama may have been unfortunate victim of a screw-up by a NH election administrator. Previous to this year, New Hampshire was one of the few states to get it right: because the order was rotated precinct-to-precinct, no candidate enjoyed a primacy effect in the overall statewide count. The switch to a single statewide ballot, presumably done to reduce ballot printing costs by a few dollars, may have ended up costing Obama victory.

One might note, however, that in terms of the delegate outcome--which is what the primary is supposed to be for--a few extra percentage points for Obama wouldn't have made a difference. Clinton and Obama each received NH six delegates, and it would have been the same result had Obama nudged past Clinton.

This all just confirms what we learned in 2000 and again in 2004: having elections administered at the state and local level is a total disaster. If elections were run by a national body, we wouldn't end up having the results determined by screw-ups like this one in NH.