My understanding is that before WW1/electrification, the majority of SER (later SECR) services to Charing Cross did run via Cannon Street, hence the direct line being only double track: it did not need quadrupling when the rest of the line was done. And post electrification, its usage diminished over the years, as others have stated. A query though: when did it, officially or actually, cease to be a 'passenger' line, in regular use for passenger trains? (It remained in periodic use for diversions if engineering work prevented direct London Bridge - Waterloo E trains until fairly recently.
While there was an intention, or at least a proposal, that there would be a regular service from the LSWR through to Cannon Street, my understanding is that the connection from Waterloo (LSWR) to Waterloo Junction/East (SER) was only ever used occaisionally, for special workings. I believe its last use was to convey a vanload of drawings from Ashford Works to the Southern Railway's CME offices at Waterloo shortly after grouping.