This is really my favorite feature in OpenSSH. It doesn’t seem that too many know about it however. I guess we could blame that on ssh-agent. ;)
Host *
ControlMaster auto
ControlPath /tmp/ssh-%r@%h:%p
Amazingly enough – this is the entire content of my ~/.ssh/config.