If you are visiting Iceland and want to get a true Icelandic experience, it doesn't come any better than this. The trail is breathtaking (both for its views and hills) and takes you past beautiful pools of scalding hot water and steaming vents.
When we finally got to the stream, there were a fair number of people right near the "changing wall" (provided to block the wind more than for privacy). I'd recommend walking a couple hundred yards down where there are fewer people but the water is still nice and warm.
For full disclosure: yes, you do need to get undressed in the open. Just wrap a towel around the bits you don't want to show. No one cares anyway. :) Getting into the water on a cold, windy day is an experience and one that I highly recommend. We soaked for about an hour, but could have gone longer.
Some tips that we took away:
-Bring beer (we didn't)
-Plan on getting up there with a picnic lunch and stay for a while
-Bring something to stand on while you are getting dried off and dressed (in case the ground is muddy)More