Well, here's a true story! When I was in elementary school, 4th grade was the first year you could play a band instrument. You could rent from the school for $1 a month, or from a music store for $7 a month. (Yes, this was a long time ago.) I wanted to play trombone, but they ran out of school trombones. I didn't want to subject my parents to the astronomical $7 rental from a store, so I said never mind the whole music thing.
About a month later, the band director came around to my classroom to tell us some more school instruments had come in. There was an oboe, some other stuff, and something called a tenor saxophone. Now I knew what a saxophone was, but no idea what this tenor saxophone thing was supposed to be. Anyway, he looks around the classroom and says, "Yeah, I guess you're all big enough to play a tenor saxophone."
I got ahold of the band director and said I was interested in this whatever-saxophone thing. I wasn't at all a big kid back then. (Grew a lot later on!) But I took to this tenor sax thing, and never really gave it up.
If a puny, skinny little 4th grader can play a tenor sax, you can too. So if you have the opportunity, don't let it slip away.
Also, there are videos floating around on Facebook of some 8 year old kid prodigy playing a tenor sax. It's almost as big as he is, but he can play it! He plays Coltrane solos by rote. It's amazing to look at. He's just a little kid!
Good luck. You can do it!