Oh, absolutely, says she who got taken off with by a horse that, well, doesn't normally do that to her. Oops.
When the weather is cold, especially in fall when a horse's winter coat hasn't come in yet, they do notice - although they don't feel the cold as much as we do. Horses in the wild do a number of things to stay warm - they eat a little bit more, they huddle with other horses, and they roll in mud to get a nice insulating layer.
And, of course, they move around more. Which means that if you pull them out of a stall, in which they haven't been able to move, they're likely to be just a little bit cold - and then they want to move around to get warm...and might well do things like not standing to be mounted or just plain trying to go for a run regardless of what their rider has to say about it.