It does require phisical skill. You have to aim, perfect hand eye coordination, muscle memory, breath control. A sport doesn’t require a full body workout, if it did than bob sleighing or dressage or table tennis wouldn’t be sports either. Sport is about physical fitness but also about teamwork, competition, perseverance and much more.

It’s like saying deathmetal isn’t music, because you can’t understand the words they sing.