An organiser of the Reclaim Australia rallies fears extremists and fringe political parties are hijacking the movement.
Landscape gardener John Oliver, 39, told the Herald Sun he and four others were the committee behind Reclaim Australia, which is opposed to Islamic sharia law and halal food certification.
A major rally held in Melbourne last Saturday led to violent clashes between Reclaim Australia supporters and Left-wing groups.
The group is planning more rallies later in the year.
Mr Oliver said opponents had claimed his group contained “racists, bigots and neo-Nazis”, but this “couldn’t be further from the truth”.
He admitted prominent supporter Shermon Burgess had previous links to the extremist anti-Muslim group the Australian Defence League.
“He did have a couple of videos surface of him before the rallies a couple of years ago where he made some racist remarks. That has since been apologised for,” said Mr Oliver from the NSW Central Coast.
“The fact that he had friends in the ADL was the reason why everyone thinks we are connected in some way, and we’re not.
“We basically said we don’t want to be affiliated with any groups that have any ties to extremists because they’re the people we are trying to fight. We don’t want to be seen as extremists ourselves.”
Mr Oliver warned other unsavoury groups threatened to sully the image of Reclaim Australia.
“We’ve had skinhead groups, even from America, listing our rallies on their websites, and that’s giving us a bad name,” he said.
“These people are obviously trying to make our rally into something it’s not. They can see there’s an element of it that may work in their favour, and they are trying to capitalise.”
Images of the Reclaim Australia stoush at Federation Square went international, after police struggled to separate opposing protesters.
Some of the photos taken showed people with swastika tattoos, and others who burned Australian flags.
Speakers invited to address the crowd included controversial evangelist and founder of the right-wing Rise Up Australia Party, Daniel Nalliah.
Mr Nalliah, who has been heavily criticised in the past for linking the Black Saturday bushfires to Victorian abortion laws, told the Herald Sun he was a supporter of the intentions of the group, but was not an organiser. He said the group had been set up by “ordinary mums and dads” and that they were not racist.
Mr Oliver said since the group formed, other political parties and Right-wing groups had also been in contact.
“We have had Restore Australia, which is more an activist group. The Secure Australia Party is interested in talking to us now, Australian Liberty Alliance, and the Q Society has shown a bit (of interest).
“We have got these people interested but in no way do they speak for us or we speak for them. They are just supportive of the movement.
“We don’t want the lines to be blurred, we are not a political party. We are just a grassroots movement of ordinary families that are concerned.”
But Debbie Robinson, who spoke to the Herald Sun yesterday from the Q Society office in NSW, said the group had “nothing to do with Reclaim Australia”.
Reclaim Australia is planning more events in July and August. There could be an earlier second rally in Melbourne.
Mr Oliver insisted that one man photographed in Fed Square who had a swastika tattooed on his head, an image that went around the world, was not a member of his group.