I have an idea that might resolve some of these issues, to answer your questions first:
How will it behave once this setting is enabled? Will everything by default be visible?
No, the proposal is that everything will be hidden by default. However see my idea below.
Will this also lock down search to limit visibility to catalog/programs?
Yes, search will be restricted with the same criteria.
will doing assigned learning to an audience automatically make that program/course visible to the audience even if the audience has not yet been assigned visibility? or will the drop down be restricted to only audiences that have been assigned to those programs/courses?
Yes, visibility will be the sum of visible learning + enrolled/assigned learning. If you are assigned to a program you will get to see it automatically.
Ok, on to my idea.
To resolve the requirement mentioned in 3 and 4, and to prevent the poor usability mentioned in 5, what if we put some extra options on the course/program editing page:
1. Course visible to all
2. Course hidden to all
3. Course visible only to the following audiences
[normal audience selector here]
This would allow admins to make a course visible to all without creating a custom audience using 1. Amir would be able to hide his courses to edit using 2. Option 3 would be the normal functionality.
The advantage of this approach is that we wouldn't need to do a per course capability check like we do with the current view hidden courses.
The only disadvantage I can see is that a course could be added to an audience but if 1 or 2 was set it would have no effect. The best solution to that would probably be to include an indication of each courses' status on the 'visible learning' tab.
We could also have a site wide setting that determines the "default" value for this new setting (when you create a new course). That would allow you to make new courses visible to all by default or controlled by audience by default.
I've attached a simple mockup to describe what I mean - let me know what you think.
Simon