Diesel and electric locomotives that pull passenger trains seem to be typically the length of a short carriage - so a pretty large space, which I assume is devoted entirely to the engine plus two driving cabs.
But DMUs have been around since the 1950s and EMUs since the 1890s, and they rather obviously pack the engine into a far smaller space - usually underneath the train. Their existence appears to amply prove that engines don't need to be the size of a whole railway carriage! So why are locomotives that big?
Your assumption that the space is devoted entirely to the engine and driving cabs is wrong. Have you ever looked inside a large locomotive or sought out some drawings of the interior layout?
There are some basic concepts in relation to locomotive and train design that are necessary to comprehend. Firstly that the adhesion between the wheels and the rails is limited, typically to about a third (or slightly more) of the weight on the wheel/rail interface. Only this proportion can be converted to tractive effort at the wheel and therefore to pull at the drawbar. A train's acceleration is directly proportional to this force (see Isaac Newton in his
Philosophiæ Naturalis Principia Mathematica 1687) and for a given force this is divided among all the driven axles in a multiple unit. A modern four or five coach long multiple unit could have any number of driven axles from a proportion up to all of them. Modern locomotives have either four or six driven axles so the locomotive has to have sufficient mass to permit the available adhesion to generate sufficient pull. Taking the Class 68 as a modern example some 38% the locomotive's mass of 85 tonnes on four axles gives sufficient drawbar pull (317 kilonewtons) to enable a six or eight coach train to accelerate at a rate similar to an emu with many more driven axles.
Basically, a locomotive has to be this heavy in order to obtain a performance similar to modern multiple units.
Secondly, in order to develop the power necessary to match multiple units' performance quite chunky prime movers are needed. A five coach long Class 80X unit has three one thousand horsepower underfloor engines and a 5 coach Voyager has five 750bhp engines for a total of 3000bhp and 3750bhp respectively. All this power has to be concentrated in one prime mover, in the case of the Caterpillar C175 used in the Class 68 this weighs in at just over ten tonnes yielding 3,800bhp. To which has to be added the mass of the alternator, another three or four tonnes.
Modern diesel engines can be up to about 40% efficient in extracting usable work from the fuel which means that the cooling system has to reject some 5,700bhp through the radiators when the prime mover is running at full chat. This needs a lot of space to work reliably, especially in hot summer days.
In addition there are all the other things which a railway locomotive needs. Things such as the exhaust gas treatment kit and exhaust pipes; the traction motor cooling blowers and ducting; the brake gear and compressed air cylinders for the whole train, including the rheostatic brake grids and fans; the power electronics; the signalling electronics; the engine control and monitoring equipment; fuel tanks; batteries and so on. There also needs to be space for the maintenance crews to be able to get to the kit...
To sum up: I am in awe of the designers who can pack all this into a 20 metre long and 2.7 metre wide box which only weighs 85 tonnes.