The vast majority of working male equines are gelded (this includes all male mules).
Horses can either be clean cut when castrated or proud cut. Proud cut means that the epididymis - the place sperm is stored - is left in the body. Traditionally, this has been associated with retaining stallion like behavior after gelding. In fact, circus people used to do it on purpose because they believed a proud cut horse would "sparkle" more in the ring.
Stallion like behavior in geldings is actually more often caused by a retained testicle (a "ridgling"). It is also often seen in geldings that were not castrated until after puberty (one of the reason ex racehorses have a reputation for being difficult is that they are often castrated late enough to have developed stallion behavior).
Leaving the epididymis in has no effect on hormone levels and is now almost never done intentionally. (However, if you're setting a story in the past, the "proud cut" thing may still be believed).