What it really means is that color isn't really that important. A good horse is a good horse - some of the best horses I've known have been pretty much plain brown.
That aside, though, humans have bred horses for color from pretty much the start of domestication. The Roma prefer black and white horses (although there's some association between pinto markings and quiet temperaments). Many draft breeds have been bred to come in only two or three colors, or even just one - all Suffolk Punches, for example, are chestnut - perhaps in part to make matching teams visibly easier. The Queen has her Windsor Grays.
And, of course, America has the beautiful Paints and Appaloosas and many breeders breed Quarter Horses for dilute colors such as palomino or buckskin.
What the saying really reminds us, though, is that color has no impact on performance - and every so often horsemen need to think about that, especially if breeding for color.
Maybe in your world there are color breeds too, and arguments about whether certain colors are better than others. I'll talk about color myths tomorrow - there are some doozies.