Why is a team of four horses in two pairs called a four-in-hand?

Because correctly, when you drive a team, you hold all of the reins (yes, all four sets) in one hand.

So, you have four horses in one hand, which rapidly contracts to "four-in-hand." (And no, I'm imagining it's not easy, although I've only ever driven a single myself).

The driver in this shot has the reins in the left hand but is "assisting" with his right hand - this is top British driver George Bowman. Image source, Vickusin via Wikimedia commons.