The Palace is about 2km from Central Ulaanbaatar, 12000MNT by taxi or 500mnt by bus (taxis easier because unless you speak Mongolian catching a bus is not easy at all!) and is definitely worth heading out to.
Friendly staff in the ticket office- entrance will set you back 8000MNT so hardly a great expense even if on a budget- however it's worth noting you cannot take any photos inside (well you can if you pay 50000 for a permit).
The Palace itself seems to be in a bit of a state of disrepair, but the main attractions are the temples which are an amazing feat of architecture. Especially at this time of year in the snow, they look truly magical. They contain a wide array of weird and wonderful artifacts and art.
A great insight to Mongolian traditions which can so easily be missed in bustling UB.