What is the weather like in May?
www.wunderground.com/travelplanner/index.asp
is a great travel planner and will give you historic temperatures for any location for any period of time. It's great golf weather at that time of year.
Where should we base ourselves?
Since you'll have a car, you can base yourselves anywhere you find a hotel that fits your budget and your tastes and be able to drive to the nearby towns. The heart of the Hamptons is considered from Southampton to Amaganssett, including Sag Harbor. The very nicest places (imo) are the inns in East Hampton like 1770 House, c/o the Maidstone and Hedges Inn but those will likely cost $500/night or more. For a lot less money you can stay at the Hampton Maid in Hampton Bays, across from the water and still be quite comfortable. You're not in walking distance of restaurants though at the Hampton Maid.
If you're willing to stay in a b & b, A Butler's Manor in Southampton gets rave reviews and in early May, the prices aren't too bad.
tripadvisor.com/Hotel_Review-g48658-d255231-…
Here's a map that shows those places
http://goo.gl/maps/34s4P
How long to drive from NYC?
Totally depends on day of week, time of day you go. Anywhere from 2-4 hours depending on traffic. If you go midweek and avoid rush hours, it's possible to do in 2 hours. Midweek will also be less expensive in hotels.
We are happy to pay top dollar if necessary, but what sort of costs should we anticipate?
A room can cost $200-700/night at that time of year. Dinner at a nice restaurant can easily cost $150+ for 2 with drinks but you can also eat less expensively and casually.
What is there to see and do?
Most first time visitors enjoy walking the various villages (Southampton, Sag Harbor, East Hampton, even Bridgehampton) looking at the shops. The beaches are gorgeous and while it may not be warm enough to swim, you could find it nice enough for sitting on the beach or at least walking. If you're into any sports, there are public golf courses including a 9 hole course with driving range called Poxabogue. You can rent kayaks, you can drive out to Montauk and go on a fishing boat or explore there. Visiting the lighthouse is popular. (Lots of Irish kids come to work in Montauk for the summer).
There are 3 wineries in the Hamptons that you can visit (and do tastings). Channing Daughters and Wolffers are both quite good. You can also drive or take ferries to visit Shelter Island (charming with lots of Victorian architecture) and can even go to the North Fork where there are many more vineyards. There's also a big discount shopping mall called Tanger in Riverhead where you can shop til you drop. That would be an easy stop on the way out or on the way back to Manhattan.