Hi Richard,
Unfortunately, it's not possible to restrict access further by activity (it's either all data related to that subject-user, or none). It is, however, possible to filter down to just data related to a specific activity.
Useful feedback for Phil Williscroft (Perform product manager, also on this thread) though – I think we may have heard similar input from others.
Jennifer