I personally feel like it isn't about "forgetting" the scales as much as it is them becoming ingrained in your brain as sounds that your ear recognizes and can pull out on command, either consciously or unconsciously. Most people do not analyze the sentence structure they are utilizing in real time as they speak, but someone well versed in a language is able to express themselves and follow the "rules" with quite a bit of freedom. They would also be able to go back and analyze the written word without a problem. I don't think learning a musical language is a whole lot different.
If you are practicing scales without learning the aural relationships that help you identify the sound, then I agree that practicing scales and modes is a bit of a waste, but scales and modes are an excellent way to identify and categorize sounds so that you have a tangible word to describe a certain set of notes. If you are concerned consciously with using a certain scale/pattern when you are improvising then that is when you start to get into the trouble of not sounding like yourself and not really improvising in a way that leads to fresh creation.
It is difficult to improvise freely and attempt to think about heavy theoretical concepts at the same time. That is why it is beneficial to have a very disciplined practice routine and then when you go play a gig or whatever you can just let loose and play. It is all about learning new relationships between sounds and practicing them until they become second nature. You practice to gain control of the material so that you can then be free to make music when you are performing. You can call a collection of notes whatever you want to in your brain in order for you to be able to catalog that sound and know it when you hear it, as well as be able to get that sound out on your instrument.
As long as you are training your ears, what you choose to call the sounds is irrelevant assuming you don't have to relay that information verbally to another person. Scales and modes are a common written language that complement reading the musical notation that most musicians understand, but not knowing the information in that way isn't going to hinder your musicality or understanding of how to create the sounds you want. The beautiful thing about improvisation is that everyone pursues it from a different angle, yet you can understand what others are creating through your lens, even though you might call it something entirely different in your head.
As long as the "scales and modes" don't rule your creative thought process when you are in the act of improvising, then I think it is a worthy endeavor to learn them and have command of them. But if you are stuck in a rut of playing things in a "dorian" position or some train of thought such as that then I'd say dropping the scales for a while and focusing purely on intervallic ear training would be useful. As I said before, it all depends on how you work as an individual, and that is a beautiful thing.
