Milford Sound is absolutely worth the visit and certainly a stellar attraction of the South Island. No matter the weather, it is compelling, dramatic, astonishingly beautiful and intimidating all at once. In few places on the planet, will you be privileged to see such gorgeous pristine wilderness. Just soak it up because your stay will be all too short.
Of equal merit is the drive from Te Anau to Milford, best done with your own transport if you can manage it - a minimum 3 hours to allow for a couple of brief stops en route. It is breathtakingly gorgeous and you will get tired of saying 'wow' at every turn in the road. Great in spring when there is still plenty of snow on the peaks but beware road closures for avalanches. Check the weather forecasts and go when it's clear - no hesitations!
But just go. More