I've mentioned this before - but driving, and even riding in a carriage, is actually more dangerous than riding a horse.
If a saddle horse panics, you may well fall off, or the horse may run you into something (I'm apparently not alone in having been clotheslined by a horse on an obstacle...ow). The horse may keep running and run into something or, worse, somebody.
But when a harness horse panics, they run off with a huge great vehicle attached to them. Youtube is full of videos of nasty carriage collisions, carriages being pulled into crowds, etc. If the carriage is what spooked the horse, they will keep running because oh my it's following me I need to get away. And if one tips, it's a lot worse than a horse falling on you (which I've also experienced).