Today’s episode of BBC Radio 4’s Great Lives is my third attempt to use my limited comedy fame to foist the non-idiomatic music-making of the Sheffield-born guitarist Derek Bailey on an unsuspecting public. In 2009, I chose Derek as my specialist subject on Celebrity Mastermind, beating the comedian John Thomson, who chose James Bond villains. To be fair, I would also have won if I had done his round. I was getting questions like, “Which Japanese duo collaborated with Derek Bailey on the 1995 album Saisoro?” and John was getting, “What colour was Blofeld’s cat?”

Musical minds immeasurably superior to mine have grappled more succinctly with the enigma of Derek, who died in 2005 at the age of 75. Writing in the Quietus four years ago, Jennifer Lucy Allan explained: “Derek Bailey is one antidote for anyone who thinks they’ll never understand improvised music. His guitar playing is that which requires a surrendering to your own ears. It is what it is, and that’s exactly what he intended it to be.” I, in turn, listen to Derek and think: “This, whatever it is, is resolutely and implacably this.” And that is what I, as a comedian, have tried to steal from it.

I think I first saw Derek play at the ICA in the early 90s, as I started scouring pub attic evenings and Labour club Sunday afternoons for free improvised music, after I came to London to try standup in the winter of 1989. Voraciously curious about the capital’s concealed subcultures, I nonetheless rarely understood who or what it was I was watching. Seeing the band Morphogenesis burst a balloon and amplify pot plants as a lone cat crept audibly around them, upstairs in a squatted Hackney pub named after a Captain Beefheart song, I’m not sure I was aware that these musicians even made records. And there was no Google to tell me.

But I do remember the first time I met Derek. In 1996, I visited his humble Hackney home to interview him for a Sunday newspaper about his new album Guitar, Drums ’n’ Bass, which Derek, then in his late 60s, had created by jamming along to pirate radio broadcasts of the brutal new dance music phenomenon, which had forced their way into his frequencies from concealed aerials on nearby tower blocks.

I remember the room he practised in being sparse and white, empty and monastic, and that he affected a comical bemusement at the idea that anyone would buy his records: “Do you just sit still and listen to them, or do you potter about and make cups of tea while they’re on?” For Derek, music lived in the moment, perhaps as a reaction to the decades he spent as a successful session musician for hire – did he really play on Petula Clark’s Downtown? – before his Damascene conversion to the absolute abandonment of tyrannical composition.

Derek sent me away with his seminal book, Improvisation: Its Nature and Practice in Music, and a cassette of the Scottish music hall comedian Chic Murray, both of which influenced me enormously. Fresh from the encounter, I wrote: “Bailey comes over like a cross between Clegg from Last of the Summer Wine and a harsh-but-fair science teacher, who illuminates the Genesis myths of improvised music without being remotely patronising.”

For the next decade, Derek supplied me with CDs of outtakes and postcards he inscribed with inspiring mantras, my favourite reading simply: “The struggle continues.” Derek died on Christmas Day 2005, Jesus Christ in reverse. At his funeral, I performed a Chic Murray routine, while the wake afterwards, upstairs in a Hackney pub, saw the veteran American jazz hoofer Will Gaines, then 78, sit in a chair and tap out a fond and furious farewell. I wish this had been preserved for posterity, but its undocumented passing is quintessentially Derek.

I visited Derek’s Hackney Downs house for the fifth and final time earlier this year, ostensibly to hold a ladder for the archivist Tim Fletcher, as he scoured the dark attic for illuminating items before the property was sold. Shafts of sunlight shone through the lean-to extension Derek had improvised freely long ago, in simpler times, when marginal artists could still afford to live and create magic in the city’s neglected nether regions.

The presenter of Great Lives, the former Tory MP Matthew Parris, had clearly arrived for the programme having prepared little, relying on notes supplied to him, and was taken by surprise by Derek’s music, which he described as sounding “like a chimpanzee”. A fractious exchange followed of which I am not entirely proud, mediated by the improv expert Ian Greaves. I’m surprised the producer left it in. But I suspect Derek would have enjoyed it. The struggle continues.

• Stewart Lee comperes a two-day celebration of Derek Bailey at London’s Cafe Oto December 15/16. Stewart Lee vs the Man-Wulf tours until the end of 2026

Derek ‘The Chimpanzee’ Bailey v Matthew Parris: a primer by Stewart Lee

The definitive streamable Derek Bailey playlist doesn’t exist, as so much of his music is off limits, though Honest Jon’s records offer luxurious vinyl reissues. Nonetheless here’s half an hour of an exploded jazz standard, some graffitied drum’n’bass, the dubby space-funk of Arcana and a final recording routing Derek’s debilitating motor neurone disease into plangent and emotive improvisation.