There has been a feature request raised that will likely cover this type of scenario.
See https://totara.canny.io/feature-requests/p/allow-admins-to-assign-reports-to-users-or-audiences-without-needing-to-use-role and add your 'vote' and scenario.
In the meantime If you're a tenant admin, you might need to work with the site admin to create some additional roles to limit access to the report or even use global user report restrictions: "are designed to restrict the content visible in report builder reports. Turning this feature on allows for fine grained control over what records are visible to users viewing a report builder report but can have a significant impact on performance."
You should already be able to set the content to only display records for users in a specific audience:
Within the report settings click on 'Content'
Scroll to the section named 'Show records by audience'
Enable this setting
Use the drop down menu to select the required audience
Save changes