Henry Lyte, who wrote the famous hymn, was vicar of the church of All Saints in Brixham. The present vicar of All Saints is vicar of two other churches too, and All Saints isn't always open, so if you want to visit check beforehand. The tourist office should be able to help, or see http://www.allsaintsbrixham.co.uk