Walking back from the Golden Temple where I had stayed on the previous night (in the Dharamshala/public accommodation they have for pilgrims), I found the Lucky Guest House, which jumped out at me because I'd read about it in the Lonely Planet previously. For Rs 400 a day, I found their non-AC room pretty comfortable and at the end of March, surprisingly cool. Other similar non-AC rooms (with a double-bed) located on the lower floors cost more - Rs 500. Apparently, the cost decreases as you go up a couple of floors (there's no lift/elevator obviously). My room was on the top-floor, which didn't show any signs of the roof-top cafe that Lonely Planet mentions - the owner seems to have converted the top-floor to make more rooms. The cleanliness was okay - it could be better. But I guess you get what you pay for.
This is a good place if you want to stay close to the Golden Temple - it's just a 2-minute walk away. You can get to the temple early morning and late at night and experience how it looks and feels at different times. It also seemed to be very popular - courtesy Lonely Planet, I'm guessing. There were loads of backpackers as well as Indian families staying here. It's location - bang in the middle of the bazaar, proximity to the temple, and popularity makes it a safe choice for a woman traveling on her own - like I was. Also, do try the great sweet lassi (buttermilk) at the stall right next to the guest house if you are in Amritsar - it's amazing!