Hi Sabrina

I don't think you are doing anything wrong - once a field is aggregated (count by audience) there isn't an option to unaggregate this afterwards (count of all users)

There is some work being planned in the next few months to improve report aggregation and graphing 

regards