Hi Tomas

Yes that will unassign all users in the group - I can't think of a way just to unassign a few users from the group. 

This isn't one of the HR Import sources so this isn't possible through this either unfortunately.

Regards