Depending on where you're travelling from, it is probably a better idea to do the Cardiff Valleys on a separate adventure. It is doable in one day, pretty much, but that's not going to be all that much fun. Certainly so if you cram in all the other bits in Wales into a 4 day adventure, you'll be in for some long days without much rest or time for enjoying some of the places you'll go to.
There's not much to see apart from the ferry terminal really in Fishguard, from the station at Goodwick there are probably decent bus connections to other parts of Wales like Cardigan and St David's, but it doesn't sound like you're overly interested in the non-train parts of Wales. Which is fair game, I was like that for many years! So it's definitely easy to just pop up the line and back.
The Pembroke Dock line is nice, and I'd look at bus links from Pembroke to Milford Haven/Haverfordwest to cut out a lot of hanging around with doubling back to Whitland first. I can't remember what the buses are like in that area, it's been too many years since I did buses around there, but it's a beautiful part of the world and worth exploring fully.
The Cambrian, another rather nice line and I think if you get good weather you'll be in for really beautiful views along the coast. I haven't been fully up the Pwllheli line in many, many years (I'm told I did the whole line when I was a small child, I have no memory of it of course) but the bit I did do was incredibly nice. It's on my list for a full-on explore some time, preferably before the 158s are replaced (so I have a couple of years yet)!
Speaking of the Cambrian, I don't rate Borth much at all so I'd not be rushing to go for a walk on the beach. Aberystwyth is much better for that. There are useful bus links both northwards to places like Barmouth but more useful would be the bus from Newtown to Llandrindod Wells. Said bus also goes to Brecon (I am basing this off approximate memory, and this was all pre-Covid, so it would need checking), and from there you can head to either Cardiff or Hereford, there may also be links to Llandovery and Abergavenny still. I don't know if they're still accepting it for sure, but Sargeants certainly used to accept the passes on their Hereford-Kington-Llandrindod Wells routes. Other bus routes of potential use will be the X13 from Llandeilo to Swansea and the X4 from Abergavenny to Cardiff via Ebbw Vale (drops off near Town station), Merthyr Tydfil and Pontypridd.
All these buses have the potential to save you a lot of time doubling back, but it obviously depends how you want to do the lines. They may not be terribly useful, but it's worth bearing them in mind. They'll also get you some more of the Welsh scenery you won't get from the train too.
The Heart of Wales line is nice and scenic in plenty of places, and is home to some slightly unusual/interesting bits. Such as Llanwrtyd Wells, the smallest town in Wales, the triangular station at Pantyfynnon (only the Heart of Wales line is in regular use these days), semaphore signalling and Cynghordy with its viaduct. It also has Sugar Loaf, one of the most remote stations in the country and Sugar Loaf itself is one of those places worth visiting on a walking holiday. The line used to be relatively local to me, not so these days though. It's firmly on my list to go back to one day, and indeed talking of it there's also Sugar Loaf Tunnel which is of a significant length.
If you decide to change trains in Whitland, there's not a whole lot to do there, but there is vastly more there than at Clarbeston Road! Llanelli isn't the most exciting place either, I haven't been through there in years but it wasn't the most interesting station or town back then, and I doubt it's got much/any better! Do keep an eye out for Loughor Viaduct between Llanelli and Gowerton, depending on the weather there can be some decent views from it if I remember correctly.
In terms of pure planning of the journeys you'll need to make, I'm not really the most helpful there but there is plenty to see and do in Wales. If you get the chance, it'll be worth stopping off in some places and having a look around. Even if just for an hour between trains, Ferryside particularly comes to mind here. Similar for places like Barry Island, there's plenty of railway history there but it's not a bad place to spend a bit of time enjoying a walkabout by the seaside and having a short break between trains.
Whatever you choose to do, even if it's just jumping off one train and onto another just to do as much track as possible in the 4 days, then I hope you have an amazing time. You've got plenty of options around that's for sure!
