Farmers trying to graze sheep on their land claim they have been abused and threatened by dog walkers.
Members of farming partnership WD Carr said they could be fined if they do not maintain the field at the conservation area at the top of Coombe Rise in Saltdean.
But when one farmer started to put up a sheep-proof fence around the open-access land a dog-walker threatened to “knock his lights out” if sheep were put in the field.
Another pet owner allegedly threatened to cut down and vandalise the fence.
Michael Carr, 55, of WD Carr Partnership said: “The dog walkers' problem is that if there are animals in the field they would have to keep their dogs under control, on a leash, while they pass through it.
“Our man who tried to put the fence up was very upset about it all. We can't do anything if we're being threatened.
"It would only be for a few weeks while the sheep graze the field to encourage flowers to grow and the field is only about a quarter of a mile long.
“We have no problem with dog walkers being on there and they would only need to have their dogs on leashes for about ten minutes.”
He said the conservation area has been designated by Natural England.
Mr Carr added: “We are required to keep it in check, which includes grazing. If we don't keep it maintained we could lose the conservation status and even get fined.
“Plus we would lose all the money we've received so far from Natural England to keep it a conservation site.
“It is an awkward situation for us.”
Although the field is owned by the farming partnership, it is designated as open access under the Countryside and Rights of Way Act 2000 (CROW).This means the public can walk freely on it without having to stick to paths.
Since CROW came into effect in 2005, people across England now have more than 865,000 hectares of land to freely explore without having to stay on paths.
Mr Carr said the incidents were not reported to Sussex Police and the partnership would try to erect a fence again soon.