The Victoria Conference Centre is located right in the downtown core of the city, and is attached to the historical Empress Hotel. There is a large number of hotels and bed and breakfast in this area, offering a wide-range of accommodations ranging from modest to luxurious. The Conference Centre maintains a list here.
Tourism Victoria has also a useful accommodation listing / booking site here.
Some hotels offer discounted rate for AO4ELT2 conference delegates:
The Executive House (5 minute walk to the Conference Centre): Single (1 King size bed) $79.00 + tax; Double (2 double beds) $89.00 + tax; To book, call the hotel and request the "UVIC Rates"
The Inn at Laurel Point (15 minutes walk to the Conference Centre): Laurel Wing Guestroom $149 + tax single/double occupancy; Erickson Wing Studio Suites $199 + tax single/double occupancy; To book, call the hotel and ask for the #866701 or the "Adaptive Optics 4 Extremely Large Telescopes" rates.