Street musician Jack Morgan had rejigged his normal set. “I’m going to be playing some protest songs,” he told the crowd milling around in front of Bath Abbey – and launched into a (suitably quiet) version of The Sound of Silence.

The reason for his change of emphasis was an extraordinary row that has erupted over the buskers who play in the squares in front of and to the side of the magnificent abbey.

For the first time in living memory, perhaps in hundreds of years, a choral evensong service was halted because the buskers could be heard above the sound of the Bible reading and threatened to disrupt the choir.

The rector of Bath Abbey, the Rev Prebendary Edward Mason, went on the attack, saying buskers and their battery-powered amplifiers were infringing the rights not only of worshippers but of the whole city to a bit of peace and quiet. The council joined in, saying it would use new laws coming into force next month to try to ban amplified music around the abbey.

But the buskers are hitting back. They say to stop them would infringe their liberty to play – and the freedom of their audiences to hear live music in the open.

Morgan, one of those who was playing when the service was halted on Sunday, said he wasn’t playing loudly. “I was playing at a reasonable volume. The service was cancelled and a number of parishioners came out and were very abusive, saying it was my fault. A few minutes later some others approached who were shocked the service was stopped. They said they could barely hear the music.

“I think the rector has a personal issue with buskers. I think he feels that the only music in the square should be the fine choir of the abbey. I think he would like us to go away. I don’t think it’s a volume issue, I think it’s a more political issue. I think we’re seen as a rabble but we have a right to free expression and free speech and that’s what we’ll fight for. We’re part of a liberal British musical culture.”

On Tuesday just before 10am the buskers gathered as usual in front of the abbey to draw pegs out of a bag to decide on their running order. They are a disparate bunch – singer-songwriters such as Morgan, classical singers and musicians, Spanish guitarists.

They elected folk musician Amelia Orgill to read out a statement on their behalf which said they were shocked and upset. “We want the abbey to know we are not attacking the church. A lot of us are Christians and respect the abbey.”

But the buskers said they were upset that the abbey had gone to the media – Mason wrote an impassioned article for the Western Daily Press – rather than speaking to the musicians or council first.

The statement concluded: “I think we need constructive dialogue so services in the abbey can go undisturbed and buskers can go on enjoying playing their music and make our living.”

The buskers are plotting their next move, which may involve a silent protest in Bath on Sunday. They will take their instruments out but not play them and explain to visitors and parishioners why the squares are so quiet. “I’d probably make more money if I stayed quiet,” said one.

Most were keen to tell their stories and explain why busking was so important. Ben, for example, described how he started playing the penny whistle, saving up until he could afford a flute. He said he had played with Van Morrison and Mary Black. “Music is my life – we have some excellent musicians who play here,” he said.

And most passersby seemed to back the buskers. Paul Guest, from nearby Keynsham, said he enjoyed visiting Bath partly because the music was so good. “It’s a disgrace if they try to stop it,” he said. Joseph Vincent, a member of the folk club at Bath University, said: “It’s quite shocking.

“There should be room for all sorts of music.”

In his office the rector admitted he was passionate about the issue. He had been reading an Old Testament passage about leadership when he felt he had to show just that quality and say the service had to stop.

Mason said: “I’ve been working with the street musicians for a long time but I’m really anxious about this city. I feel like weeping for it really.

“We’re surrounded by noise quite a lot of the time. I feel like weeping for those who try to sort out issues in the city and how we live together. I weep for a beautiful place that is subject to this noise. I weep that we human beings just cannot resolve conflict.”

Mason said the local authority, Bath and North East Somerset council, had little power to solve the problem – but he thought the Antisocial Behaviour, Crime and Policing Act, which comes into force next month, might help.

The council said it would look to use the new rules to clamp down on the buskers.

It said its officers had been “proactively monitoring” the vicinity of the abbey after concerns were raised and had held a number of meetings to try to resolve the situation.

It said: “The council is able to take action under the environmental protection act 1990 if the busker is causing a statutory noise nuisance. To do this, the council has to collect evidence of an ongoing pattern of serious noise disturbances which are materially interfering with the enjoyment of someone’s property. Noise abatement notices have been served on buskers by the council in the past when all other interventions have not been complied with.

“The council is in the process of contacting the abbey after receiving a complaint yesterday afternoon and will be investigating further.”

David Dixon, the council’s cabinet member for neighbourhoods, said: “Clearly the powers previously offered to local authorities have not been effective enough to deal with this type of nuisance.

“Therefore, under new powers provided by the Antisocial Behaviour, Crime and Policing Act 2014, we will be looking to ban all amplification within the immediate vicinity of Bath Abbey.”