Ever since I spent my first full year there, I've thought that my two favorite combinations of time and place were Durham in October and Durham in May.  Temperatures for Durham in October average out somewhere between 65 and 70 degrees.  And, while it does rain in Durham in October, I only seem to remember sun drenched days and foliage akin to New England's.  May in Durham is warmer than most Julys in northern cities, so I would consider it Durham's best "summer" month, even though it's spring according to the calendar.  It's not hot to the point that you would trade a pound of gold for some central AC (like you might consider in June, July or August in Durham), but it is plenty warm enough so that you could sit out on the lawn and while away the day, just catching some rays from that famous Carolina blue sky, accompanied or solo.

As is mentioned above, the middle summer months in Durham can be devastatingly hot---and humid.  This is not to say that you won't be able to enjoy yourself in Durham during these months, it just not be ideal for those who are used to cooler weather.

Despite its southern location, Durham can get quite cold in the winter months.  It's nothing compared to the cold experienced by midwesterners and northeasterners, but it's not the most comfortable time to be outside either.  It does not snow very often in Durham, but when it does, the scene can be apocolyptic.  There are only one or two plows in the entire city, and people can be extremely touchy and tentative when driving in the snow (understanably, as it occurs so infrequently).  All this is to say, don't avoid Durham in the winter, but don't make it your vacation spot.  Save that for March through May or September through mid November.