The site for Rotherfield District Scout Council has been removed while redevelopment takes place.