You have to be willing to do a lot of commuting if you live in Idyllwild. If you sneeze - while driving through the 'town' you'll miss it. Its very very small.
Shopping, groceries, etc., are quite a drive into town. You'll have to be good at stocking up, taking inventory and managing supplies, otherwise, you'll be up and down the hill every other day. Schools are not the best quality there either and the place can get overrun with tourists during the summer.
Yes, the road is dangerous even if you are a careful driver because there's always the fool who's taking the turns too fast and yakking on the phone while doing it or the tourist not watching the road. I certainly wouldn't want my teenager driving that road on a regular basis - I'd be a basket case of worry.