Deriving diffeomorphism symmetry