If you want a great place by the Castle District, this is where you should stay. I just spent 5 nights at Hotel Castle Garden and I have nothing but good things to say. Nice rooms, well kept, great breakfast (included in the room fee) and a quality restaurant. It's literally a one minute walk from the District. Best of all, they have an awesome staff who are kind, interesting people. They speak fine English and were fully prepared for all of my questions (I had a lot) and my extremely limited Hungarian language skills.
To hotel: Please tell Laura (Lora?) at reception that the flight home was pretty smooth and that I think she should visit America. It's especially nice in the spring, summer, fall or winter.