I've got a long ways to go as an improvisor. I'm still working out primarily on the blues, which has more possibilities than many people realize. But I have found it more useful to focus on the chromatic tones, chord tones, and diatonic passing tones for each chord, rather than a specific scale. Of course, the chromatics, chord tones, and diatonic tones ARE the scale, but I think it helps to recognize the sound and function of each note in the scale, not just the scale itself. I realize that sounds a bit strange, but it seems to work for me. And Pete's chart is useful in that it points out the chromatic tones.
To explain this a bit further, it helps to know, for example, the actual sound of the b5, the 6th (13th), the b6th, the 2nd (9th), 3rd, b7th, the #9th, and so on. Both in relation to a specific chord and in relation to the key center. So you need to know how the 6th of the key sounds (for ex an "A" in the key of C), and how the 6th in a specific chord sounds (for ex, the "E" in a G7 chord, still in the key of C). Then you can make note choices based on sound, rather than running a scale. In general, I find that the edgier chromatic tones are best placed on upbeats or as the second note in a triplet, or as passing tones (which kind of amounts to the same thing). There are plenty of exceptions, for example if you want to emphasize the tension created by a chromatic note. The b5 of the key in a blues is a good example.
Anyway, sorry for the convoluted explanation. This is simpler to do than to explain. But you really have to tune in to the relative sound of each note: each chord tone, and all the related notes in the "chord scale."