Baltimore is a friendly, welcoming city with many attractions, restaurants, shops, and hotels just a short just a short walk from historic neighborhoods and the world-famous, Inner Harbor. Like any large city in the USA, Baltimore has areas best left unexplored by tourists. The main tourist areas of the city are perfectly safe in the context of big cities anywhere. In other words, you're surrounded by thousands of other people during the day and into the evening hours, and thus it's extrememely unlikely that you'll be the victim of any type of crime. Safety issues in the tourist areas during the day and in the early evening would mainly involve the possibility of pickpockets or pandhandlers. Although the pandhandlers are rarely aggressive and a simple "no" works fine.

Areas like the Inner Harbor/Downtown, Harbor East,  Fell's Point, Federal Hill, and Canton and Mount Vernon, are the parts of the city most likely to be visited by tourists. That's where the attractions, hotels, restaurants, bars and cultural actvities are mainly centered. These areas would be busy during the day with foot traffic, and well into the evening in areas like the Inner Harbor, Federal Hill, and Fell's Point especially.

Traveling to and around the baseball and football stadiums is also not a problem. Realize that you'll be arriving and departing with thousands of other fans and unlikely to encounter any problems, along with the fact that there are many police in the area directing traffic.

Use "big city smarts". Take a taxi if it's late. Be aware of your surroundings, and if by unlikely chance you've wandered somewhere where you feel uncomfortable - leave! There are some areas (Let's use Lexington Market as one example) where it's perfectly fine during the day but not the place to wander in the evening as streets are deserted. Don't accept rides from unmarked taxis. These do exist in Baltimore, mainly to get locals around, they're not likely to be cruising the tourist areas. 

And don't leave anything of value in your car, regardless of where it is parked.