Museumplein is actually a big open area (or a square) that's a short distance from most of the major museums in Amsterdam and Vondelpark. The main attraction is to take a photo with the letters- and I think that yes, you should do that if it's on your way! After all, when in Rome do as the Romans do.
After taking cutesy photos with the letters then there'll be no end of museums around to fill your time or to simply go on a cycling excursion in Vondelpark.
During winter when it's cold enough, there'll be an ice-skating rink just in front of the letters,
If you own or manage Museumplein, register now for free tools to enhance your listing, attract new reviews, and respond to reviewers.