LILBURN, Ga -- The Dar-E-Abbas Islamic center currently sits on a small plot of land on Highway 29 and Hood Road. They want to raze the current building and build a two-story mosque.

It’s that expansion that has residents living in the area up in arms.

“I am prejudice, I just don’t like Muslims and I don’t want them taking over our neighborhood,” said one resident who lives near the mosque. He did not want to be identified for fear of retaliation.

“I don’t want someone coming to my house and burning it down.”

An attorney for the mosque, Woody Galloway, told CBS Atlanta he hopes the issues residents have isn’t about religion, but, knows it is a strong possibility.

“The mosque just wants to be a part of this community and make it better,” Galloway said.

In order for the mosque to expand they must rezone from residential to commercial. That rezoning is what residents are fighting against and followers of the mosque are fighting for.

The plans include building a 10,000 square foot worship building. The building is modeled after a shrine in Karbala Iraq, for the Shrine of Abbas. The mosque also wants to be an 8,400 square foot indoor sports facility. The site has been occupied by the mosque for the past 11 years. In that time membership has grown from 10 to 70 with attendance of 250 or more during special religious ceremonies. A cemetery is also in the plans.

“I don’t have a problem with it being a place of faith I have a problem with it being a cemetery,” Resident Rod Oaks said. According to the letter filed with the rezoning office the cemetery would only take up 1 acre of the 8 acre site.

A rezoning meeting with members of the community has been rescheduled for Nov. 12 at 7:30 p.m. in the Gwinnett County Justice Administration Center.