Although Fred Ebb – who wrote the lyrics to John Kander's music on shows including Cabaret and Chicago – died in 2004, he had a UK premiere of a new show last week, with the opening of The Scottsboro Boys at the Young Vic in London, three years after its Broadway debut.
Even allowing for the period of mourning that Kander understandably allowed himself after the loss of a lifelong collaborator – and the fact that he worked first on another unfinished show with Ebb, Curtains (2007) – the long gestation period of The Scottsboro Boys, a dramatisation of a 1930s racist miscarriage of justice in Alabama, is further testimony to the notorious difficulty of producing a new musical.
Other evidence is around. The Light Princess, the Tori Amos-Samuel Adamson show at the National theatre, had five years of delays, revisions and workshops before it reached the stage. And The Leonard Bernstein Letters, just published, contains pages of agonised correspondence, during the creation of West Side Story in 1957, between composer Bernstein, lyricist Stephen Sondheim, choreographer Jerome Robbins and Arthur Laurents, who provided the "book".
As the men were working together almost day and night, the rows between them were probably acted out in person rather than committed to foolscap, but Bernstein unloads the horrors of the process to his wife in long letters written during rehearsals and "out of town tryouts" in Washington and elsewhere. "What's the use?" the composer moans, deep into pre-production. "The 24-hour schedule goes on – I am tired & nervous & apey … this is the last show I am going to do." Elsewhere, he complains: "The work is endless: I never sleep. Everything gets rewritten every day … and imagine, we open two weeks from Monday."
It is the crazed revision – there are entire albums of Stephen Sondheim songs that were dropped from shows – that is the constant and astonishing factor in accounts of the creation of musicals, regardless of whether they prove to be a hit or a flop. Samuel Adamson told me in an interview that he was startled by the rotating metamorphosis of The Light Princess in comparison with the speech dramas – including Southwark Fair and Mrs Affleck – that he had previously written. And most playwrights who moonlight in song and dance experience this culture shock.
Of all the many theatrical mishaps described by Simon Gray in his brilliant series of Diaries, one of the most severe was his involvement in an early version of the 1993 Broadway musical The Red Shoes, composed by the then octogenarian Broadway legend Jule Styne, who wrote Gypsy and Funny Girl. Gray was one of many people who left the project during its development, and took his revenge by depicting Styne as a capricious egomaniac surrounded by a chorus of adoring aides. Public taste may also have been a balm for Gray, as The Red Shoes, after belatedly opening, closed after 51 previews (a quantity often indicating a troubled show) and five performances.
Possibly corroborating Gray's view, Jack Rosenthal, another British playwright who worked with Styne – on a musical version of the TV play Bar Mitzvah Boy – responded in similar fashion by writing a stage play, Smash!, in which a diffident dramatist is bullied and patronised by a lunatically self-regarding Broadway songwriter.
So why are musicals so hard to get right? One clue is in the cathartic rage with which Gray and Rosenthal set down their own version of events in a medium – journa; theatre play – over which they had sole control.
Whereas a straight play merely demands collaboration between a writer and a director (with producers and actors sometimes also having a say), a musical usually requires co-operation between a minimum of a composer, lyricist, dramatist and choreographer and, given the huge budgets general involved, often several producers. So a big show is lucky to get away with fewer than seven or eight credits, egos and their agents in the room. As at a European Union or United Nations summit, the diversity of rooted interests almost guarantees disagreement.
But perhaps the fundamental reason for the prevalence of off-stage drama is that the musical is an inherently artificial form and therefore, as with attempts at non-mechanical flight, most projects are doomed to extreme difficulty and probably failure.
Artificiality also applies to spoken drama, but the convention that invented characters are speaking made-up dialogue is rapidly established and accepted. Why does the protagonist make a long speech about his mother in act two? Well, because his mum died in act one, or because someone has just asked how she is. But for a character suddenly to burst into song about his mother is a development of a different order. Can the transition between spoken dialogue and song be achieved without disrupting the narrative flow or breaking the concentration of the audience? And is this "mummy" number sufficiently different from all the maternal tribute music written in the past? Perhaps the composer and lyricist could be sent to a hotel all night to knock out an alternative.
Because the musical is an inherently illogical form – but with a greater craving for realism than its near-cousin, opera – there will always be someone suggesting a different logic. Some writers – for instance, middle-period Lloyd-Webber – tried to solve the speech-song shifts by through-composing shows, but this merely introduces the new artifice of people singing bald exposition ("I am his brother") or banality ("Will you want milk with that?") at each other.
The single consolation of the death of Fred Ebb is that at least – on Curtains and The Scottsboro Boys – he got to leave most of the hell of musical gestation to other people.
