Not at all! Start with the blues. The slower and more concentrated you go, the better the effect of finding the notes that you're hearing. And you'll have the time to hear how the notes relate to the chords. Once you can comfortably navigate what you're hearing on a blues in all twelve keys the process onward will be that much easier. But you have to really go slow. That may seem at odds with the idea of how much time it will take to achieve your goals but trust me, the slower you go the faster you'll progress. The things that I practice currently (having been playing now for over 40 years) are the very same basic things that beginners do. Thing is, over time you begin to see the universe in a twelve bar blues.
Speaking of goals, the goal is not to be an encyclopedia of tunes in all keys. The goal (for me) is to transpose on the horn as easily as when singing. That's a long term process, for sure. But don't let that intimidate you. The progress along the way is more than stimulating enough to keep things going!
Additional thought...just in case it's not clear…the reason this goal is important to me is not necessarily to be good at transposing (although that's an important skill)…but because I find that when improvising I can more easily play what I hear at any given time, thereby opening up the channels to freer expression. No matter what kind of music…
