Here is some info + train times from the Man in Seat 61's highly informative website:
If you have a private driver it should be possible since it can be arranged for him/her to pick you up at Wang Po. However, you will have to take the 10.25 train from Kanchanaburi to make it worthwhile. This train is scheduled to arrive there at 12.04. Since you plan on visiting the Hellfire Pass I suggest that you drive directly there. However, trains in Thailand do run late and this is the problem.
Once you arrive in Kanchanaburi ask at the railway station as to the arrival/departure time of the train. You may have time to visit the nearby WW 2 cemetery and/or the Thai-Burma (WW 2) Railway Museum.