I'm not a museum lover even though I'll always go to one in every City. But the National Museum of Iceland is small, manageable in a few hours, has quite a few nice videos (with chairs nearby), and is educational but not overwhelming. My favorites were the videos, the model hut, and the old church doors inscribed with runes. It is about a 15 minute walk from the City Center toward the University. If you are interested in the sagas, this museum does not focus on them, although they are certainly mentioned.
If you own or manage National Museum of Iceland, register now for free tools to enhance your listing, attract new reviews, and respond to reviewers.