Yup. It's common belief that horses don't understand delayed consequences and therefore won't stay "mad" with their handlers, but this is not entirely true. (Actually, it's that given the communication barrier between the species, horses tend to assume that they are in trouble for the most recent thing they did).
I've had a horse grump at me for an entire ride because they were annoyed that I made them walk out of the barn on their own, or otherwise do something they didn't want to do. And they definitely remember people who didn't treat them well (and people who did). So, yup, they can hold a grudge. How long, well...that depends.