Almost right away! In the wild, foals need to be able to keep up with the herd from the start, which is why their legs are often not much shorter than their mother's. It's considered a likely problem if a foal is not standing within an hour of birth (although they do sometimes fall over a couple of times).
They can walk within two hours and they should be running within four hours of birth. Pretty incredible, right? (Again, they do sometimes fall over a few times while working it out).