In the 1960s and 70s, no serious rock fan viewed the drummer Jim Gordon with anything but awe. By the 80s, none of them viewed him with anything but contempt, a 180-degree turn that led to his virtual erasure from the culture. Even four decades later, when the veteran music journalist Joel Selvin first tried to sell publishers on a book meant to tell Gordon’s story with nuance and depth, they balked. “They would debate it for months and then say, ‘Nope, can’t do it,’” Selvin said. “It was almost impossible for them because of what he had done.”

In 1983, he entered his mother’s house and began to attack her with a hammer, crashing it into her skull four times before grabbing a knife and stabbing her repeatedly, the final time with such force it pinned her to the floor. Soon after her resulting death, Gordon was arrested, charged and convicted of murder, and spent the next four decades in prison, before dying this past March at 77. Over the years, several prominent articles have been published that tried to trace the outlines of Gordon’s story, ascribing his heinous act to an undiagnosed case of schizophrenia that forced him to hear voices and experience hallucinations. Yet only in Selvin’s new book, Drums & Demons, does the reader get a feel for the full horror of his disease and the mess it made of his mind. “In one of his hallucinations, he thought he was in a jail cell that was on fire,” Selvin said. “To me, that was a metaphor for Jim’s whole life. For him, life was a jail cell that was always on fire.”

Despite the chaos that created, both for Gordon, and increasingly, for those around him, Selvin aimed to tell his story with empathy. Only after the drummer’s death was he able to finally convince a publisher to go along. “The guy got so little compassion,” he said. “I wanted readers to know just how impossible Jim’s life was and how brave he was in battling the disease.”

At the same time, the author meant to “restore Jim’s peerless legacy. Who has done more to put his mark on our music than Jim Gordon?” Selvin said. “What a playlist he was on!”

Just tracing the surface of Gordon’s contributions reveals more than 100 classic songs powered by his invention and finesse. In his early studio work, he appeared on an entire chart’s worth of pop hits, by acts like the Beach Boys, Ike & Tina Turner, the Byrds and Glen Campbell. By the 70s, he became a key member of pivotal rock bands, including Delaney & Bonnie, Mad Dogs & Englishmen, Derek and the Dominos and Traffic. Growing up in the San Fernando Valley of California, Gordon became entranced by the power of the beat from childhood. He played in bands by puberty and, by 17, helped flesh out demos for the publishing arm of Liberty Records. That same year, he joined the Everly Brothers on a tour of England and, afterwards, became part of the storied Wrecking Crew, a loose collection of studio musicians who played on a dizzying range of 60s hits. “Back then, there were loads of great studio drummers,” said Lenny Waronker, a legendary producer and record executive whose career started in the same west coast studio milieu of the 60s. “Jim was able to plow through that. All the other musicians were amazed by him.”

Gordon’s role on those storied sessions extended way beyond the simple task of keeping time. “He wasn’t just a backbeat guy,” Selvin said. “He was a fully musical drummer who embedded his playing into the core of the composition.”

For instance: in the 70s hit, Grazing in the Grass, by the Friends of Distinction, Gordon’s drum elaborated the song. “Even though there was a chart in which every note was written out for him, he added a Latin boogaloo feel that exploded the whole record,” Selvin said.

The fills and intonations he added to Carly Simon’s You’re So Vain contoured the melody and directed the listener’s ear to the record’s subtler touches. “Jim orchestrated that entire song from the drum stool,” Selvin said. In Maria Muldaur’s number one smash Midnight at the Oasis, he added a key samba groove, while in Steely Dan’s Rikki Don’t Lose That Number, the tricky beat he devised deepened the song’s debt to jazz. In doing so, “Jim became an important part of the hit-making process,” Selvin said.

Mark Lindsay, frontman of the hit group Paul Revere & the Raiders, immediately noticed Gordon’s gift after he was hired to drum on their song The Great Airplane Strike. “He was doing this polyrhythmic thing with a kick, a snare and a high hat, accented by tom-toms,” Lindsay said. “He changed the song up so much that I wound up rewriting half of my lyrics to fit was he was doing! Jim became the conductor of the track.”

Waronker recognized the same level of creativity on Sundown, a song he produced for Gordon Lightfoot that became a number one hit. “His drum part made the song move in its own way,” he said. “It’s a specific rhythm that Jimmy picked up from Gordon’s guitar. It became one of the most important parts of the song.”

In the 70s, Gordon expanded his range to work with rock’n’roll’s most cutting-edge bands on the road. “When you listen to his live work with Mad Dogs & Englishmen or Derek and the Dominos, he’s unleashed,” Selvin said. “The ideas just flow from him.”

At the same time, the voices that were roiling inside his head began to find disturbing external expression. In an infamous incident on the Mad Dogs tour, he hauled off and punched his then girlfriend, the singer Rita Coolidge, in the head. “Here was a guy who was noted for being gentle, smiling and laid back,” Selvin said. “But that was just the mask he wore.”

Some people were already beginning to see through it. “[The singer] Claudia Lennear said she always wondered about that smile,” Selvin said. “It was too simple. She felt he was hiding behind it.”

“Jim had such genius,” Lindsay said, “but I sensed there might be something lurking behind the curtain.”

To Selvin, Gordon’s talent can’t be separated from his torment. “The level of intuition that Jim displayed in his playing requires a certain electro-chemical makeup,” he said. “His highly personal style had to come from the same place in the brain that produced his schizophrenia.”

At the same time, the focus and power involved in playing drums gave Gordon a refuge from the cyclone of thoughts whipping through his head. “The combination of the resonance of the drums and the rhythmic entertainment of the groove produces a hypnotic feeling that can lift you out,” Selvin said. “Nothing calms a schizophrenic faster than a Walkman and a pair of headphones. For Jim, the drums provided a place where the voices couldn’t follow.”

Strangely enough, the herculean amount of recreational drugs Gordon took at the time also had a calming effect. “You would think that the massive amounts of cocaine he did would make things worse,” Selvin said. “But I talked to psychiatrists who said that it would normalize his dopamine levels. He was doing blow to feel normal.”

Similarly, the crazy rock’n’roll lifestyle of the 70s, which Gordon exemplified, served as a cover for his increasingly aberrant actions. “The rock scene of the time was nearly indistinguishable from psychotic behavior,” Selvin said with a rueful laugh. “Jim just blended into the background.”

It helped that, at the time, he was still soaring creatively. In 1973, Gordon devised a pair of drum patterns that proved crucial to the development of two separate genres. His work on the Hues Corporation’s smash Rock the Boat, with its high-hat syncopations and danceable beat, helped patent the rhythms of disco. Similarly, his extended break on the song Apache, paired with the congas of King Errisson, became a foundational pattern in hip-hop that was later sampled ad infinitum. “When Kool Herc found Jim’s long drum break on Apache, he discovered that he could make it bound from one turntable to another forever,” Selvin said. “He was driving crowds nuts with that sound.”

By late in 1973, however, Gordon’s beat, and sanity, were beginning to seriously waver. He viciously attacked his wife Renee Armand, cracking several ribs in the process, ending their marriage. His work with the would-be country-rock super group Souther-Hilman-Furay Band grew so erratic they had to sack him. While he managed to keep it together in the studio for a few more years, by 1978 Gordon proved too unreliable to be employed.

In a reporting coup, Selvin acquired research that helped fill in Gordon’s inner life during that pivotal time. He found two women who, in the late 80s, had gained the drummer’s cooperation for a book that never got off the ground. The notes they took gave Selvin access to jail house interviews with Gordon along with his medical records and related court documents. (Selvin sent several written requests to interview Gordon himself but they went answered.) Regardless, the research he acquired from the women allowed him to put the reader deep inside the musician’s roiling mind.

The voices Gordon heard shamed him so deeply, he rarely told anyone about them, which contributed to him never getting a proper diagnosis. His mother, one of his closest witnesses, believed that drinking and drugs were his problem rather than a symptom of something far more corrosive. While Gordon began to imagine that many people were torturing him at the time, the main voice in his head was his mother’s. “Because Jim’s father was a practicing alcoholic, his mother became the sub rosa leader of the household,” Selvin said. “That’s why she became the major figure in this panoply of voices hectoring him.”

As a result, it was her voice that he felt the most urgent need to silence. Once details of the subsequent murder came out, some observers who knew Gordon in his high functioning days were floored. “When I knew him, he was a tremendously nice person,” Waronker said. “He was the all-American boy.”

Selvin’s book describes what led up to the murder in granular detail, but he doesn’t write much about Gordon’s subsequent decades in prison because, he said, he found it undramatic. Often keeping to himself, Gordon became a virtual zombie due to the anti-psychotic drugs the prison pumped him with. Rare as Gordon’s particular case was, one key reason Selvin said he wrote his book was to let readers know how common various forms of schizophrenia are. “To me, the single most astonishing fact of the research I did was that schizophrenia affects one in 100 people,” he said. “Let that sink in: Multiple sclerosis affects one in 10,000! We see these people out in the street, hearing voices all the time. Their world is totally frightening. And I have nothing but compassion for them. Unfortunately, society doesn’t.”

The other key reason Selvin wrote Drums & Demons, he said, was to restore Jim Gordon to the popular music world. “He’s gone,” he said, “and he needs to come back.”

• Drums & Demons: The Tragic Journey of Jim Gordon is out on 27 February