There are a lot of people who have thought about learning to ride but then are too nervous to try. Like any sports there are risks on the other hand the number of people who have hurt themselves just tripping over etc.! Over the years I have had people having to cancel their lesson because they have fallen off the balcony, down the stairs, taken part in a dancing show (yes!), playing football and caught their hand in the door. I once had somebody who badly strained their ankle while trying the trainers with the little back wheel in the actual shop!
So, when you put it in context considering you are dealing with a live animal the risks are not that high.
It is becomes a more risky sport when being given instruction by somebody who themselves have not been taught correctly. When learning a sport like horse riding it is important to learn from somebody who is actually qualified so that they can help create a clearer picture of the hows and whys. Like with anything (like learning maths etc.) someone who is successful at something doesn't necessarily mean that they can train somebody else successfully!
I'm sure there are a number of occasional riders who don't particularly feel that comfortable when they do ride e.g. their back feels uncomfortable (when they previously didn't have back problems). This is apart from the feelings of apprehension because they don't feel confident in their abilities having had a few falls or scares.
If they have not been taught the basics correctly this is more likely going to happen. If they have been given horses that are not suitable (not trained or not suitable for a novice rider) then they will feel nervous.
A qualified instructor will know how to get a new rider upto a level that they are able to ride with certain amount of confidence and competence a quiet trained horse.
A qualified and experienced trainer will know when to graduate an ambitious rider who wishes to get to competition levels knowing when they are ready psychologically to be pushed and how far the rider can be encouraged or not! Many previously naturally brave novice riders have been pushed too fast and start showjumping when they are not at all ready and have been over faced (asked to jump too big) then lose their confidence. With a qualified and experienced instructor this would not happen.