For the record, Don Byas and Lucky Thompson were both recording long before 1954.
Benny is from my hometown and I can remember working a concert with him there at the old black musician's local, AFM Local 274, Philadelphia, Pa. The union was segregated back in the day, hence, the legendary 274, which was a who's who of jazz greats. This was a few years after he had returned from Hollywood where he got burned out from the deadlines associated with writing movie and tv scores. He told me that he had not touched his horn for 8 years while he was writing, and getting his chops back in order was not easy.
His sound was incredible that night, Selmer had given him 2 brand new Serie 2 tenors, one for his house in NYC, and one for his house in Paris, and his sound was huge on that shiny new horn. He talked to the audience about his composing and his time in Hollywood, he discussed the tragic element about having to compose "I Remember Clifford," and he also included a couple of little stories about mistakes he had made while composing. He told of being asleep and dreaming the greatest melody he had come up with yet and he rushed downstairs to pencil and paper and wrote it out. But when he got up in the morning, he found that had written the melody to a Horace Silver tune in his sleep! Benny was not above laughing at himself.
So he wasn't in any way controversial, and I think that helped keep the writers and the hype people away from him. He is just a saxophone virtuoso who composes music of the highest order, melodies that will live forever. I'm glad I got to know him and work with him a little bit.