After Labour lost power to David Cameron's coalition in 2010, the favourite to take over the party was not Ed Miliband. It was his brother David.

David, the former Labour foreign secretary, was four years older than Ed, and had reached a higher cabinet rank than his younger brother. David was less geeky: whereas Ed could solve a Rubik's cube in 90 seconds, David was considered "vibrant, vital, attractive and smart" by Hillary Clinton.

Ed had always followed in his older brother's footsteps; from the non-selective state school they both attended in north London to elite Oxford University, working as advisers to Labour politicians, entering parliament themselves and becoming cabinet ministers.

So it was a shock when he attempted to overtake his brother by throwing his hat in the ring for the 2010 Labour leadership race. Had Ed Miliband finally had done with playing second fiddle?

Britain's election: rise of Scottish and English nationalists threatens old order

The challenge fascinated the British political class. Many commentators described it in biblical terms, comparing the Milibands to Cain and Abel and talking of fratricide. When asked why he had challenged his brother, Ed simply asked: "So you believe in primogeniture?" – the archaic right of the first-born male to inherit the estate.

And many Labour loyalists were devastated – after the brutal rivalry between Labour PM Tony Blair and his successor Gordon Brown, the last thing Labour needed was another bitter party split.

Ed positioned himself to the left of New Labour, Blair's election-winning, business-friendly equivalent of Bill Clinton's New Democrats, by that time irreparably tarnished after 13 years in office and following Blair's decision to follow George Bush into Iraq. New Labour had not done enough to address inequality, Ed argued, calling for a healthy society – economically, culturally and socially – in which wealth was more evenly distributed. David, by contrast, was much more in the centrist, Blairite tradition.

The battle between the brothers could not have been more dramatic if scripted by Hollywood. After the first round of votes were counted at the Labour conference in Manchester in September 2010, David was winning, but he did not have a majority. After the second round – still David, still no majority. Third round – still David. On the fourth and final round, with only David and Ed Miliband left in the race, for the first time Ed took the lead – winning the leadership contest by 1.3%.

Ed and David embrace after Ed is elected leader of the Labour party in 2010. Photograph: Christopher Furlong/Getty Images

The result was announced on stage in Manchester. Both brothers looked like they were in shock. They hugged. Ed said he loved David. And then followed a mighty fall-out. They barely spoke for a long time afterwards. David stood down as a member of the shadow cabinet and in 2013 resigned as an MP and moved to New York to work as head of the International Rescue Committee refugee charity. Five years on, although they insist their relationship is improving, it is still strained.

The process would have destroyed most families, and almost did theirs. But the Milibands are a resilient bunch, and had a long history of political scrapping. Their parents Marion Kozac and Ralph Miliband – a respected Marxist academic who died in 1994 – were Jews who lost close family in the Holocaust and had fled to Britain to escape the Nazis (from Poland in her case, Belgium his).

Ralph was well to the left of his sons. Ed and David had grown up in a family where political debate and visceral disagreement were meat and drink. Ralph had little time for his sons' politics and told them so – but he loved them, and was proud of them. Their mother Marion was also to the left of Ed and David – but in a more practical, less abstract way than Ralph. Taking on your brother was not such a big thing in such a fiercely, and overtly, political family – or so Ed Miliband had thought until the reality dawned on him. It is only recently that he has admitted how much David was "bruised" by the experience.

Has Ed Miliband got what it takes to be prime minister?

After leaving Oxford in 1992, Ed Miliband worked briefly as a political researcher for TV before being headhunted by Labour to become a researcher and speechwriter. He moved upwards quickly, and by 1994 was adviser to Gordon Brown in the years leading up to Labour's landslide victory under Blair in 1997.

Once in government, Miliband advised Brown during a period when relations with Blair had become so aggressive and embattled that they could not communicate without resorting to expletives. Miliband, regarded as the human face of the Brown camp, became known as the "ambassador from Planet Fuck".

In 2002, Miliband took a year-long sabbatical from the Treasury and went to Harvard as a visiting scholar, teaching economics for two semesters at its Centre for European Studies. It was a welcome return to the US for Miliband - his family had lived in Boston for two periods when he was a child when his father had been teaching there – and he remains a keen student of American politics and a keen fan of American sports – particularly the Boston Red Sox.

Ed Miliband and his wife Justine Thornton at a conference speech in Manchester. Photograph: Stefan Rousseau/AFP/Getty Images

In 2003, he returned to the Treasury, and in 2005 stood as a Labour MP, returning to the cabinet as a minister rather than an adviser. He and David were the first brothers to serve in a British cabinet since Edward and Oliver Stanley in 1938 and were regarded as the golden boys of a new generation of Labour figures.

I believe that people would quite like somebody to stand up and say there is more to politics than the photo-op

Ed Miliband