Hi Sam,

This setting doesn't control whether non-members can see inside of the group. It controls whether everyone who is even not a user on the web site can access the group. The help information next to the checkbox describes it: "Allow people who are not logged in to view this group including the forums". Unchecking this setting doesn't seem to be what you are looking for. You should use "Hide group" setting to control visibility of content to non-members.

Yuliya