We live in Swanage and have been trying many of the local pubs now we have moved there. The Castle inn is definitely our new favorite. The pub looks great as you walk in, decorated with twinkly lights. We actually phoned ahead as it was very late lunchtime on a Saturday and the landlord was very friendly over the phone and said we would all be fed no matter what, which actually turned out to be after when they would normally stop serving food.
The specials board was filled with locally sourced produce and the meal we both had was excellent (sea bream). Also they have a toy box which our 4 year old loved as he took toys to the table.
All in all excellent food, very friendly staff, good local ales and a great location. Can't recommend highly enough.