The main reason for visiting this museum was Bosch's piece, which is indeed the main attraction and already worth the entry fee. The museum is quite big, so take your time. The permanent collection itself is not so impressive, although the highlights are (they point them out on their flyer- that's really cool). Their focus is on sacral art and pieces from Flemish painters. For children it's really boring, as well for people who do not possess a certain knowledge about and passion for Flemish and religious paintings, and antique furniture.
We did not encounter a lot of other visitors (January), so it's quite quiet (something which I really like). For all Bosch admirers I recommend you to visit the museum, it's worth it!
Best thing would be for you to take the bus (as it stops right in front of the museum).