Monday morning in Camden Town, north London. Rain slicks the market's cobbled yard, drenching the hemp sellers and noodle stalls. And Tracey Thorn and Ben Watt, the two halves of the couple once known as Everything But the Girl, sit upstairs in the attic office of Watt's record label, Buzzin' Fly, talking about the past.

"Listening back to our early records," Thorn says, cradling a mug of tea, "you do have a sense of, 'Gosh, well, I'm not really that person any more.' Some of that stuff came out nearly 30 years ago. When you think about all the things that have happened in the interim, you do feel like you're listening to the voice of a different person."

Thorn and her husband (they've been a couple since meeting at Hull University in 1981, have three children, and married in 2009) have spent the last few months listening to that voice. They've been revisiting the first four albums they made as Everything But the Girl – from 1984's Eden, which first announced the band's laidback, jazz-inspired sound, to 1988's soul-influenced Idlewild – to create a set of lovingly prepared reissues.

Out this month, the four two-CD sets place the original tracks alongside demos and B-sides. Each one also contains a booklet with old photographs – there, with Eden, is the source of the band's name: a slogan on Turner's furniture shop in Hull – with an introductory passage by Watt, describing the recording of that album. "I'd write them," he laughs, "and then flash them under Tracey's nose to check whether they were true."

The reissues are an unexpected move: Everything But the Girl disbanded in 2000, when Thorn decided she wanted to spend more time with their young children. The couple have revisited the band only twice since then: in 2001, to release an after-hours compilation (Back to Mine), and in 2003, to put out a "best of". So perhaps it's not so surprising that the idea of revisiting their early recordings wasn't their own.

"We don't control that part of our career any more," Watt says bluntly. "Warners own the rights to those albums, and our big fear was that one day we'd wake up and they'd have reissued them, without telling us. Then we got a phone call from someone who specialises in reissues, saying he wanted to put them out. And I thought, 'Well, if it's going to get done, we might as well do it now.'"

Listening to their early music for the first time in years, they put paid to some of their own preconceptions. "When you listened to [the 1986 album] Baby, the Stars Shine Bright," says Thorn, "it struck you as being much better than you thought, didn't it?"

Watt nods. "Hmm. But I had to swallow the clunkers as well." Clunkers? "Yeah. I'd probably take Sean off Love Not Money [from 1985]. It's just a bit ham-fisted; it's a politicised song about Northern Ireland and I didn't quite pull it off." He grins boyishly. "But hats off for trying is what I say."

On a more positive note, the couple have enjoyed rediscovering the carefree, lo-fi energy that underpins their early recordings, something they felt was lost as the band's fame grew through the 90s. "Eden was recorded with that fierce, adolescent spirit that everybody had at that time," Watt says. "Self-awareness is a dangerous thing: by about the third or fourth record, people were throwing comparisons at us and you have to be very tough to withstand it. And by the end of the 90s, we were playing to 5,000 people a night. I'd stand on stage, looking out, thinking, 'I don't want to be this big.'"

I was one of the many who discovered Everything But the Girl in the mid-1990s – the monochrome electronic beats of Missing, the band's remixed 1995 mega-hit, and Thorn's 1994 collaboration with Massive Attack, soundtracked every teenage party I can remember. So listening again to the evolution of the band's sound through their early albums is a revelation.

Thorn and Watt hope others from what they call their "second wave" of 90s fans will feel the same, but stress that the reissues are not a curtain-raiser for a new album or reunion tour. "We were very clear," she says, "that we didn't want to do a retro tour with this stuff. I know that would be an obvious thing to do to promote it, but the thought of strapping a guitar on again and playing all those songs from the past really does fill me with cold dread."

In the meantime, Watt is busy with his label and his DJing and Thorn with her solo albums. She's released two acclaimed records since 2007, and is just putting the finishing touches to a Christmas album, on which Watt has been relegated, he admits, to "session guitarist and teaboy".

They joke about being in competition to see who can publish their memoirs first. In fact, Watt already won the toss in 1997 with Patient, a book about his experience of being diagnosed with Churg-Strauss syndrome, a rare auto-immune condition; he's now writing a second book, about his parents. Thorn's first book – a memoir called Bedsit Disco Queen: How I Grew Up and Tried to Be a Pop Star – will be published by Virago next February.

Working separately has, they say, been no bad thing for marital harmony. "One of the things that contributed to the end of Everything But the Girl," says Thorn, "was the slight pressure of being a couple and a band together. We've been together a long time now and you don't necessarily need to increase the things that make your life stressful."

And with that, they both collapse into giggles, like a couple who already figured that out long ago.

The reissues of Eden, Love Not Money, Baby, The Stars Shine Bright and Idlewild are out now on Edsel Records