Kazakhstan is not a destination with a lot of architecture, its population having only recently been moved out of their yurts. If you like modern iconic landmark architecture, Astana will be your first stop, with a slew of spaceship-style buildings to put the new capital on the map.
Soviet architecture can of course be found all over the country, but its most interesting and beautiful expressions have been left mainly in the former capital Almaty. Many Soviet mosaics have however been destroyed in Almaty, these are easier to be found in poorer cities with less money for renovation.
Kazakhstan is poor in medieval architecture, except for 2 gems in the south: the mausoleum of Aisha Bibi near Taraz and the Yasawi mausoleum in Turkestan. Finally, there are 3 remarkable mosques, in Pavlodar, Saumalkol and Zharkent worth seeking out for die-hards.