The biggest thing I miss were the daily walks with close coworkers, whether to grab coffee, lunch or just to get outside and enjoy the fresh air.
I'm blessed in that I have two roommates I'm also best friends with, so that definitely helps, but I do agree with you that there is a certain privilege, if you will, to the spontaneity that comes with office talk.