It's complicated. The problem (I think) is twofold:
1. What I think #metoo etc has revealed is that almost every woman has at least a couple of experiences of men pushing it too far - sometimes to the point of illegal sexual harassment, but perhaps more often to the point of being a jerk.
So while an innocuous remark is an innocuous remark, in that first moment, a woman doesn't know if this bloke is making a quick light comment, or if this is going to be another case of someone coming to chat to her all journey, or each time she travels. If she is a regular traveller, will she have to smile politely and laugh with him each journey?
2. Women get chatted at more than men. So when they're out and about, their day will be interrupted more frequently. What if a woman just doesn't feel like it that day? Or is trying to focus on something else? I'm sure both men and women have experiences of having to humour someone with small talk when they don't feel like it; but it's very different if that's a daily, a weekly, or a monthly experience.
Don't get me wrong - this is not crime of the century. I sort of agree with the argument that in public space, we will interact with people who irritate us a little and that's a feature of life. BUT I also think that a man interacting with a stranger in this way - particularly in the context of their own workplace - could better reflect on the position that he's putting her in. It is possible to smile to and make small talk with a stranger without referring to their looks and this is probably a better approach.