Your best bet for Thai culture is to hit Phuket Town, especially the old part. Rawai and beaches along the east are probably the place to head for fishing activity and more cultural aspects as it's less touristy on the east coast (beaches aren't so nice). Plenty of temples to visit - Big Buddha.
Here's a list of some quieter beaches you could look at too:
YOU are a tourist going to the biggest tourist area in thailand..along with thousands that are looking for the same thing.; you will find more tourists than locals andtmany of the thai's are transplants from other areas comming here to service the hoards of tourists
If you get off the island you have lots of choices but depends just how far you want to travel , you vcan head up to Phang Nga area or go down the Krabi area away from touristy places, or, depending on time of year go over to Khanom area on the Gulf of Thailand , there are lots of choices but it comes down to time of year and how far from Phuket you are prepared to travel
I stayed in Phuket for 3 weeks and went to a lot of the islands. If you want a less crowded trip to the islands, go to ones that takes longer to get to like Surin or Similan Islands. They are more expensive to get to, but far less people sign up for these (especially Surin) so you get a smaller group and the diving/snorkelling is just incredible. Closer "popular" islands, although still nice, will likely be packed with tourists:
As for temples, if you just drive around town, local temples are often very quiet since they are not the main "attractions" of Phuket. Phuket Town has more locals, but if you want to go to the beach, you will need to get a taxi or arrange transportation, which could be a bit pricey in Phuket.
Some beaches will have more tourists than others - stay away for Patong Beach for sure if you hate crowds. Rawai Beach, Nai Harn Beach, and Nai Yang Beach are not as nice sand-wise but generally much quieter. You can see photos I took here:
Ko Yao Noi is a really nice quiet island only 1 hour from Phuket ,but feels like years apart. Rent a scooter and drive around the island . There are several small villages with fisherfolk and rubber farmers. Also a great base for day trips to other islands . Resorts for all budgets . Including the very up scale sixth sense resort. The beaches are not white and the water is not aqua blue ,but thats what has spared this island from over development. Go before it's to late.
Agrees with sawadeeka....I can only recommend booking a (tourist)trip to Koh Pan Yi fishing village....
You will most likely avoid large crowds on KPY(we did).....And the lunch on KPY is just ok...However,the highlight to me seems a little invasive tho' extremely interesting....And this is discovering the village on stilts behind the tourist restaurants.....Fascinating how a village exists for generations on stilts virtually suspended almost completely over water, with their school and community and families....
A tip:Avoid having your photo taken with a gibbon or any wildlife on KPY.....
As for Temples...We went to see Wat Chalong Temple, interesting tho' chockers full of tourists....
We loved the temples in Khao Lak which are not on the tourist trail....in low season anyway....
If I were you I'd head to Lanta, that was the most chilled place we went anyway
But as others have said... whilst quieter places do exist... sadly Phuket is not a place to experience real Thailand culture away from the crowds... in fact it is just the opposite!