I think Greenport offers you everything you asked for. It was mainly a fishing village in the past but has become a haven for artists and tourists. There are no chain hotels in the area, but there are a few boutique hotels in downtown and quite a few B&B's near by. Google staying in Greenport or North Fork B&B's for listings. The area has over 40 wineries if you like taste testing and a short ferry ride from downtown Greenport will put you on Shelter Island. A beautiful drive across SI and another short ferry ride and you are in the Hamptons. Do not wait to make reservations, many of the better B&B's were booked up at the end of last season
Well you've posted this on the Greenport forum so I'm assuming you're leaning toward that? The Hamptons imo are nicer with better beaches, but also pricier. Your budget may determine your options.
I definitely agree with Nywhiz. From what you say you are looking for, you would be happiest staying in South Hampton or East Hampton. Beautiful ocean beaches and more shopping and restaurants there. The North Fork definitely has it's charms (personally, I prefer it.) but there is more "action" on the South Fork and the beaches are world class. You might want to consider 3 nights in the Hamptons and 1 night on the North Fork because they are both great and quite different from each other. Or you could do the North Fork as a day trip using the ferries on either side of Shelter Island - they run from Sag Harbor on the South Fork and Greenport on the North Fork.