There are no trains between Turkey and either Georgia or Armenia. There is a line between Tblisi and Yerevan, but it is quite a lot slower than going by road.
The border between Turkey and both Georgia and Armenia are in pretty remote east end of Turkey. So don't underestimate how long it will take to get there. There are direct buses between Istanbul and Tblisi but 1, it takes a very long time, and 2, the border can be painful. I have never heard anyone express concerns about the reliability of the bus, but do remember that bus drivers in Turkey can be quite 'excitable' and often buses don't have toilets - so you have to plan around service station stops, which can also be quite stimulating - and not in a good way.
Most people fly.