This came up in a Youtube video I was watching. Horses have considerably better night vision than humans. They can see in what we would consider pitch darkness, and I've misjudged things on the trail before and had to let my horse find the way home.
The price horses pay is that their eyes do not adjust well to sudden changes of light. The modern horse is a plains animal for much more of its evolution than we are. We still have a lot of features of a diurnal forest dweller, so our eyes are designed to handle light changes very quickly. That 30 to 60 seconds it takes for your eyes to adjust to a rapid light change is pretty quick.
A horse's eyes don't adjust nearly that quickly. In fact, a horse's eyes can take anything from 15 to 30 minutes to adjust. As plains dwellers, they aren't naturally adapted to patterns of light and shade the way we are. Their much better night vision also requires more time.
For writing purposes?
Horses will tend to spook when going from light to dark and vice versa. To a horse, a dimly lit barn door is a black cavern and even if they go in and out of that barn all the time...they will at the least hesitate, especially if they don't properly trust their rider or handler. Cross country course designers often intentionally put fences in the shade as a test of trust between horse and rider. Riding home after dark tests this trust in the other direction.
There have been incidents of horses running into objects, or each other, because lights in their pasture compromised their night vision.
Horses are going to be more vulnerable than human riders to light-based attacks - flares, light spells, etc. If you are fighting a mounted person on foot, tossing a simple globe of light spell at the head of their horse could be incredibly effective.
(And what if elven eyes work like this too...)