The Wavering Middle
I’ve long thought that Labour’s switch to selecting its leader via a direct membership vote was a wrong turn, and that the next Tory leadership battle will be disastrous for the same reason.
If MPs select their leader from among themselves, they’ll be choosing the person they think is most likely to lead them to electoral victory. That means choosing someone who appeals not only to most of their members, but to the wavering middle of the general population. The far-left in Labour and far-right in the Tories can be ignored, because they’ll vote for you anyway.†
If the party members choose the leader, the balance shifts towards the middle of their population. The bell curve shifts left for Labour, or right for the Tories. You end up with a party leader further to the left or right than most of the population can stomach: supporters of the opposing party will hate them, and the wavering middle won’t be as tempted to vote for them either.
In the past few years we’ve seen the effect of this on general support for the Labour party: lots of party members still like Corbyn, but the general public overall don’t.
The Tories dodged this in 2016 when MPs agreed among themselves on May before it went to a members vote (the party introduced membership votes for leaders in 1998, but it’s been somewhat ameliorated by the ability of Tory MPs to decide the shortlist). For a while, May was someone who could appeal to the wavering middle. As historian Robert Saunders shows, she’s undermined that by trying to become the personification of the “will of the people”, and Tory MPs are now in the same relationship to her as leader as Labour MPs are to Corbyn. But even if May goes, they’ll be in the same boat if they end up putting two leadership candidates to a vote of their party members. We’ll end up with a Tory leader far too right-wing to appeal to the majority of the British electorate. We can imagine how it will go: a moderate will be up against a Brexiter, and the far more Brexity Tory membership will vote for the latter.
†There’s a caveat here about UKIP. After the 2014 European Parliament elections, it seemed clear to Cameron that they were stealing the far-right flank of the Tory vote, and there was a risk of splitting the votes on the right in the next general election such that Labour could win under first-past-the-post. So he tacked right to head them off, by promising an EU referendum. [Edit, 25 June: I was wrong to echo this received wisdom. Cameron promised a referendum before the 2014 EU elections, arguably helping to normalize UKIP in the process.]