Modern riders use stirrups almost all of the time. Why?
The first reason is that stirrups do make riding more comfortable. They help provide a bit of extra balance - although a competent rider does not rely on them. (A good rider can do everything without stirrups they can with and carry on as if nothing has happened if one of them breaks). The exception is racing - jockeys use stirrups to support themselves above the horse's back. This doesn't literally make them lighter, but it allows the horse to have more spinal motion and thus run faster.
The stirrup was invented in China, with the earliest recorded depiction being in a Jin Dynasty tomb dating to about 322 AD. In order to have stirrups you also have to have a solid treed saddle (which were invented in 200 BC, again in Asia). Stirrups did not find their way into Europe until the 6th or 7th century.
Prior to the stirrup, riders rode without, although "toe loops" that provided some support appeared in India in the second century BC. The Sarmatians used one stirrup, but only to make mounting easier (another good use of stirrups).
So, if a good rider can manage without stirrups, are they just for comfort?
No. Stirrups spread across the world for a reason - they made mounted combat much more practical. A mounted archer with stirrups is far more accurate than one without. Melee combat on horseback is also much harder without stirrups.
The Medieval joust requires stirrups...the rider stands up in them as he takes the shock from the lance. Without them he would have no chance of staying on. (This, of course, makes the idea of mounted lance tourneys being conducted by King Arthur's round table in the 4th or 5th century a major anachronism - King Arthur, if he existed, did not have stirrups!). It's even argued that Medieval feudalism could not have evolved without the stirrup.
Prior to the stirrup, chariots were used instead of cavalry. The chariot eventually went out of use - chariots are only useful on very flat, open battlefields, whilst a mounted warrior can handle a much greater variety of terrain.
Stirrups are also vital to the cowboy for the same reason they're vital to the knight - they provide a stable platform for roping.
This trail horse is hitched to a trailer fully tacked, clearly showing the western-style "wooden loop" stirrups.