I looked in code and elsewhere for global settings for this block and I don't see this as being configurable. In looking at the query that is used to get the records for this block, it appears that a simple "LIMIT" on the query could limit the number of courses that this block displays. That of course would be a custom coding change. The query currently appears to sort the courses by most recently enrolled courses, so only adding/using LIMIT in the query may or may not produce the expected results you are wanting.

I would recommend an enhancement request on this one which implements some configurable settings for this block allowing you to set a max number of records to display.