Be sure to visit this library, which is free. If you can, time it so you can go on one of the free hour-long guided tours, which explain the significance of all the beautiful art. The courtyard is pleasant and would be a good place to have lunch. Either bring your own, or there is a cafe.
The library is located in Copley Square, which is an attraction in itself, with Trinity Church nearby. Boylston Street is worth a wander, Newbury Street is a block away, and the Public Garden is down Boylston. This is close to everything you want to see. Don't miss!