Providence is connected to the rest of the East Coast via Amtrak and Bonanza Bus Lines.  Both have stations downtown.  There is a local bus system as well.

Rhode Island Public Transit Authority offers routes and schedules. RIPTA ,as it is called, is a great public transportation system.  You can literally ride all over Rhode Island for  a few dollars. There are buses going to every conceivable spot in the state and they run frequently.  The fare is $1.50.   In the summer they have special buses going directly to the beaches and back.  There are frequent buses going to Newport from Providence, at that low price. Also in Newport, RIPTA has incredible buses which tour the famous cliffs and other scenic areas.  You can get a day pass you all the buses for $6.00, truly amazing.  And what tops if off is that RIPTA buses are all equipped with bike racks so you can take any bus, put your bike on the bike rack in front of the bus, ride as the bus for as many stops as you want, then just grab your bike off the rack when you get off the bus. So you can take the bus to Newport and bring your bike - All for $1.50.