We have split up West Kazakhstan into 2 itineraries. 1 starts from Atyrau, the other from Aqtau. Both are about harsh landscapes, geology and paleontology, as you explore the bottom of the ancient Paratethys Sea. Fossils, depressions, gorges and weirdly coloured mountain ranges, oil fields, asphalt lakes, mysterious balls and underground mosques.
From Atyrau, we head towards Dossor, site of the first oil exploration in Kazakhstan a century ago. Past a little mausoleum of a Sufi saint and into the wild. First comes the colourful Akkergeshen plateau, with the man-made mountain-cave Imankari, then, past another abandoned oil field and bitumen mine, we head to the out-of-this-world shapes of Aktolagai plateau, with a thin pyramid-shaped stone as the cherry on the cake.
There are a lot of interesting places in the Mangistau area. A further exploration by this publication is needed. In the meantime, check out the names on the map and start googling. An excellent trip report with pictures of an itinerary taking in many of these places can be found on this Ukrainian blog. Be sure to click part 2 and 3 as well.
Another possibility is this gps route. More pictures can be found in this photo gallery of the area; the regional tourism site has some nice 360 pictures. Fanatics will be happy to dive into more out-of-the-way geographical features described on Wikimapia. Not mentioned anywhere (could not find gps coordinates) is this archaeological site of a ruined 10th-13th century city, as well as this recently discovered site.