Columbus is a very quiet small town. It has a few good hotel rooms and at least one B&B that I know of. Its town center looks very much like it did in the early 20th Century, so much so that it was recently used in the movie about John Dillinger starring Johnny Depp.
Basic restaurant options are available, but nothing fancy. If you need a down day before you hop on the train, it's a good place for that. There is a giant antique mall to browse through, and good local people to talk with on the street who will be fascinated by a Scottish accent, no doubt.
So if you want slow and quiet, it will be great for you. If you prefer lots of options and stimulation galore, you may rather stay in Madison.
Good luck with the planning, and keep coming back with the questions!