Armory Square is a busy entertainment, restaurant, and bar area, I think you will find it comfortable and safe to walk around in day or night.
The place is a zoo in the evening, especially on the weekend when there is a festival going on in Clinton Square. Extremely safe.
I would have no hesitation booking a hotel room in a hotel in Armory Square. Yes, it's the place where there is the most activity in the evenings, because there is a concentration of higher-end condos, restaurants and bars. Police presence is usually in line with that, too and they will close off streets during peak hours.
It's probably the area downtown where I would feel most safe at night.