Phnom Chisor is located on a hill somewhere 40 kms to the south-east of Phnom Penh. We went there by car and combined with a visit to the Ta Phrom temple at Tonle Bati. Total cost was 60$ for a oundtrip from Phnom Penh in a Toyota monovolume, we were five people (plus the driver). We think it is rather too far to go by tuktuk, even if passing through some rural villages is nicer by tuktuk.
You'll have to climb some 400+ steps. Even in the 'cold' season, you may find it quite hot by the time you reach the top. Take a hat and good shoes (no hiking boots, though: too hot).
The temple on Phnom Chisor suffered from the 'secret' bombing on Cambodia against Vietcong trails in the early 1970s. As a result, the two basins in the east entrance hall have stopped being filled by a natural water source. The temple is indeed heavily damaged, presumably not only caused by shells. Dated a bit earlier than Angkor Wat, it is rather deceiving if you have already visited the Angkor Archaeological park. I'd recommend to go here before you see Angkor.
What makes worth your visit is the breathtaking view on the countryside, reaching tens of kms wide from the south-east to the north-east. You can clearly follow the old access route from the east that reaches the baray and then passes two (ruined) gopuras. Apparently, king and priests had to climb up the steep stairs from the east side to visit the temple.
A plus (we found) was the total absence of children trying to sell incense or lotus flowers etc. or otherwise plainly asking for money. There are some fruit seller stands if you go up the hill from the north-west.
This review is the subjective opinion of a TripAdvisor member and not of TripAdvisor LLC.