"people are just more productive" is not the case for everyone. That relies on a lot of variables; type of work being done, as you mention the office environment, whether the people you're working with are in that office etc. My home office is far superior to any environment I've ever been given by an employer. I have multiple large screens, control over the heating, a door I can close and most importantly a cat to stroke whilst thinking.

Anecdotally I find extroverts prefer the office and introverts prefer home. I find that seamless access to people can be really distracting especially when I'm doing focus work.

I think you're correct if people are given a good work environment, are in on the same days and are all in the same office this makes sense. The reality is that in many large organizations this isn't the case.