The delay on drawing annotations was fixed and, together with some poppler patches by jaliste, we managed to get it working properly. Check it out:
There are still glitches here and there, but we are working on it so that the feature is available on evince’s next release (fingers crossed!). After we get highlighting done, the other text markup annotations should be fairly straightforward.