Campaigners fear planning delays could mean Royal Alexandra Children's Hospital could collapse before it is redeveloped.
Brighton and Hove City Council approved multimillion pound plans to redevelop the site in Dyke Road, Brighton, five months ago.
But work still has not begun as the local authority negotiates a legal agreement over how much funding developer Taylor Wimpey must provide for transport and education work in the local area.
During the delay, despite round-the-clock security, groups have repeatedly broken through the hoardings to enter the site.
Others said the Victorian building's turrets were being stripped of lead by thieves.
Dan Wilson, of St Michael's Place, Brighton, said: “The Alex is decaying every day before our eyes. I've seen people entering the site without any problem.
“My fear is that the Alex is Brighton's next West Pier and will burn down before it can be developed.”
The developer, who bought the site in 2006 for more than £10 million, will convert the distinctive main 1880s building into 20 flats.
The rest of the site will be demolished and replaced by 99 new homes.
A section 106 agreement of about £350,000 between Taylor Wimpey and the local authority is at the centre of the delay.
A council spokeswoman said they expected an agreement to be signed in the coming days.
Mick Hamer, of the Montpelier and Clifton Hill Society, said: “Any dilapidated site is always vulnerable meaning it is a likely target for vandals.
“I'm pleased the red tape has been sorted out and that work is going to begin.”
David Brown, of Taylor Wimpey, said: “We are keen to begin work on the redevelopment as soon as possible.
“All being well, we still hope to be in a position to begin selling the first selection of new homes at the development off-plan later in the year.
“In the meantime, we are satisfied that every reasonable step is being taken to prevent unauthorised access to the site.
“We are also ever grateful for the watchful eyes of residents in the local neighbourhood.”
A council spokeswoman said: “There have been no unnecessary delays from the council which has been negotiating the legal agreement with the developer.
“It is important for the council and residents that all the agreed contributions are secured for the benefit of the community.
“Our planning enforcement team is also monitoring the site and making regular visits.”