Do horses have milk teeth?

Yup, just like we do. Horses have 24 milk teeth - 12 incisors and 12 premolars. They get their first permanent teeth at about 2.5 years and may not completely stop teething until 5 years old. (I've certainly observed a 5 year old gelding in discomfort from the late-growing canine teeth).