Hi Reeder49,
I agree with Rootman, I don't consider gypsum or some of the other basics to be 'fertilizer' and therefore in my opinion (as Rootman suggests) it should not change the classification. I would be much more concerned about the high PH (nearing neutral) that you have. Ginseng typically prefers acidic soil. While it should grow ok for you, a range of 6.0 -6.7 would be better and I would prefer to see it slightly below that even.
The problem is that it might take years to adjust the PH in a way that will last.