On extremal (almost) edge-girth-regular graphs