Weirdly, this has resolved itself, but I'm not entirely sure why. I removed "Appraiser" from the list of roles that can answer some of the questions, added the role back in again and it started working. I had previously checked, and none of the questions required the Appraiser to answer, so very odd. Guessing this won't be easy to reproduce since it's now fixed itself, so maybe just one of those things unless it happens to someone else.
Dan