Same goes for any style. 1st innovators then copiers and finally the scholars. By that time the music is archaic.Jazz “theory” is a mess. After a lot of study I’m starting to realize how many things are related and I have realized a lot of stuff is made more complicated than it needs to be. The back door progression (IV- bVII7 I) is really just “play in the minor and resolve it to the major”. Tritone substitution is “move keys chromatically downward”.
I realized from learning and analyzing Miles Davis’ solo on Doxy that bebop is fundamentally about playing clever resolutions, but a lot of the other stuff is pentatonic or blues (Miles used both). You don’t have to play every chord, in fact outlining the chords sometimes sounds bad! You don’t even always have to resolve the chords… And if you resolve the chords, you can play almost anything before the resolution.
Jazz was initially not a written set of rules, people would learn by imitating or making up their own lines and techniques or borrowing stuff from wherever (the blues, classical music, etc). All the seemingly obtuse academic stuff can be thought of as basically shredding techniques. They’re ear catching things. They don’t have a reason for being right other than “they sound cool” and “we recognize them because they are part of our musical culture”.