Hi Harry,
I think the obvious concern is a performance hit. Its hard to say if there would be problems unless you thoroughly test, but if you are also using dynamic audiences with a massive amount of orgs/jobs then there's probably a higher chance of performance issues. Id test to see what it takes to slow the system.
cheers,
George.