Honestly, we don't know. Likely not long after the first person sat on a horse's back. Humans have raced everything else - including themselves - to the point where it appears to be a natural human desire.
The Romans raced horses both under saddle and in harness. Chariot racing was a particularly popular (and dangerous) sport - its spiritual descendants include modern harness racing and the rodeo chuck wagon race (which is also quite dangerous). Hippodromes were present in most major cities. We also know that chariot and bareback racing were included in the original Olympics (horse racing is not part of the modern Olympic games, but some equestrian events are).
It's likely that organized horse racing also took place in China and Japan, and likely in North Africa and the Middle East - certainly horse racing is a popular sport in the modern Middle East, where Arabians and Thoroughbreds are both raced.
We also know racing took place in Medieval England and that professional jockeys were working as early as the 11th or 12th century - with the first recorded race for a purse happening somewhere between 1189 and 1199.
Modern racing started in the 17th century with the King's Plates in England and racing for betting in France. In 1664 the first racehorse in the United States was set up on Long Island. As importance increased, the modern Thoroughbred was developed.
Harness racing, on the other hand, developed out of match races between farmers, with the Dales Cob being the oldest known breed bred specifically for harness racing at the trot (it has long since been eclipsed by the modern Standardbred). These races took place at the trot or pace because they were originally conducted using the same vehicles farmers took their goods to market in. The modern Standardbred was refined in the US, but is now regularly seen in Europe.