I stayed at La Boulangerie for 3 nights with my parents (who are getting on a bit) and my sister. We thought it was great. The stairs up to our room were very narrow and twisty (as you would expect in an old building), which wasn't great for my mother, but the rooms were very comfortable and the breakfast provided each morning (fruit, orange juice, a selection of breads and jams, and your choice of tea, coffee or hot chocolate) was very nice. It was a short walk from the centre of rouen, in a quiet street which looks sort of dingy and deserted but at no point did any of us feel unsafe there.
One potential problem that we didn't face but others might - the woman who runs the boulangerie and the B&B sems to speak really limited english. We spoke french with her and everything was fine, but possibly if you have anything out of the ordinary to discuss with her, it might prove difficult if you don't speak french.