I would drive. It's easy. Take motorway A4 then A14 from Dresden direction Leipzig. Exit Döbeln-Nord, major road B169 to Döbeln, B175 to Hartha, B176 to Colditz. It's doable in less than 1 hour.
The train would be close to 3 hours, if not more. You would probably want to avoid the expensive ICE trains and stick to regional trains plus buses via Chemnitz.
Another option: Take the bus from Dresden to Nossen (Dep. 9.50 am, arr. 10.33 am) and from there the train to Tanndorf, which is the closest stop to Colditz (Dep. 10.54 am, Arr. 11.45). You could make a reservation for a taxi in advance to pick you up there - it's only 4 miles or so to Colditz.
Here is the website of a taxi guy in Colditz: