It's all about timing and rhythm...
I was told by my teachers that the most fundamental thing about playing jazz is the underlying rhythm. If it don't got that swing, it don't mean a thing. Sometimes, the note choices aren't that important, you can get away with playing almost anything, if it's in time. When I was disappointed in my jazz playing I ended up spending a lot of time with a metronome, and this seemed to make a difference in my playing.
I was also told to put more air through the instrument (was playing tenor at the time) and that helped to get the notes moving too. You've got to abandon your fear, or get used to it and learn to enjoy it, because you can't make split-second decisions during an improv session unless you are totally concentrated on the music.
I find it easier to play by ear, to let my fingers reach for the notes before I tell them to, just by thinking of the music when the sax is in my hands. That way there is nothing between me and the music, during improv. But when there are technical passages I want my fingers to get the hang of, I will practice it until I don't have to think about it.
That way hopefully I will feel prepared when I am trying to 'seriously' improv and attempt to sound good. But I spend/spent plenty of time doodling and playing garbage too, just to learn the feel of the instrument and playing by ear without written music. After a while, getting the notes out that I want, more or less, became more natural and something I didn't struggle with as much. Now, the challenge for me is to be able to do it in every key on the sax, as I have trained myself better in certain keys.
btw - I have been playing for about 10 years on tenor and just switched to alto recently.