Quickest way will be by MTR. Fortress Hill MTR Station (the entrances are on Kings Road) on the Island Line is just a couple of minutes walk from the City Garden. Take a train heading for Sheung Wan and go 3 stops to Admiralty Station. Disembark and walk to the opposite platform to change to the Tsuen Wan Line (to cross the harbour) and go one stop to Tsim Sha Tsui Station on Kowloon. Should take no more than 20 mins. The harbourside is about 5 mins walk (or if you intend visiting the Temple Street Night Market stay on the MTR a further one stop and disembark at Jordan Station, take exit A, turn right on reaching the street and walk about 200yds until you reach Temple Street on the right with barriers across).
Trains run until about midnight at frequent intervals.
You can return by the same route or instead of MTR under the harbour you can take the Star Ferry across to either Central Pier 7 and get the MTR from Central Station or take the Star Ferry to Wan Chai Ferry Pier and take the MTR from Wan Chai Station. However in either case it's about a 10 minute walk from the ferry Pier to the MTR station.
There are also frequent buses up Kings Road from Central and Wan Chai.
You can see the Symphony of Lights from outside the Hong Kong Convention and Exhibition Centre (next to Wan Chai Star Ferry Pier) on HK Island or from anywhere on the Tsim Sha Tsui waterfront on Kowloon side (Tsim Sha Tsui Promenade/Avenue of Stars).
Personally I think it's best seen from the Kowloon side but it's easily seen from both sides. There's plenty of viewing space and you don't need to worry about getting there early to get a good place to see or anything.
