Comedian Alex Horne and bassist Will Collier are discussing what to call the music in their show, The Horne Section. "The band are all jazz musicians," Horne says, "but they don't really play jazz." He wrinkles his nose. "I can't stand jazz."

How would they describe the music, then? "Eclectic," Collier offers. "It's jazz instruments," says Horne. "Not necessarily," Collier interjects. Horne looks discomfited. "We should really find a word to describe it."

They might struggle to define it, but Horne and the five musician friends who make up the Horne Section have hit on a winning formula: the show is a riotous splicing of live music and comedy, with the band improvising comic songs in a variety of styles, and Horne presiding as a winningly baffled-looking master of ceremonies. The show kicked off in Edinburgh in 2010 and has grown into a late-night festival institution: around 200 comedians, from Harry Hill to Jimmy Carr, have taken up guest spots and the Horne Section have recorded two series for Radio 4.

This summer, they're doing several sessions at London Wonderground on London's South Bank; and a four-week run in the Udderbelly, the inflatable purple cow that dominates Edinburgh's Bristo Square for the duration of the fringe. They will also be road-testing a family-friendly show. "Kids make the best audiences," Horne says. "They're like drunk people – they don't mind telling you exactly what they think."

The band grew out of the childhood friendship between Horne, drummer Ben Reynolds and trumpeter Joe Auckland; Horne admired his friends' musicianship, and wanted to work with them, but couldn't quite work out how. He came up with the idea for the Horne Section after performing standup at Ronnie Scott's. "It was the first time I'd seen jazz and comedy put together," he explains. "It just seemed to fit."

He sees the show on a continuum with old-time variety shows hosted by the likes of Ken Dodd. "I love his shows," Horne says, "because there's a band and you don't really know what's going to happen. Although," he frowns, "organising a band involves a lot more admin than just doing standup."

I can see what Horne means: the musicians, expected an hour ago, are arriving in dribs and drabs. I ask Auckland about the band's origins and he tells me about the time, as teenagers, that he, Horne and Reynolds accidentally set fire to a disused train carriage. "The embers of the Horne Section were in that fire," he jokes, while Horne mutters: "I was keen to keep that a bit secret."

I have no idea whether the story is true, but it contains all the elements that define the Horne Section: a show built on the gentle banter between friends having a good time on stage, and just enjoying playing music – however you describe it.