Justin,
Welcome to the forum! We hope you will stay and share your knowledge, experience and stories about Ginseng!
I live in East Tennessee and the Ginseng has died off here until next year. I am sure that it has there in Vermont as well, unless you all are having some kind of Global Warming episode. While Ginseng is a pretty tough plant, most of the plants begin to die off (leaves turn yellow) within about a week of a killing frost or several less severe frosts or even several good cold spells with temperatures in the 30's and light frosts. However, some plants with the protection of a heavy canopy of tree leaves or thick bushes, weeds, etc. may hang on for several weeks up to a month longer. Once the leaves turn yellow, they will gradually turn a yellowish-brown and fall off within a couple of weeks. Some stems with the forks on plants such as 2 prongs, 3 prongs and 4 prongs can sometimes still be spotted for quite some time after the leaves fall off providing that there is not a heavy leaf fall or a snowfall which knocks the stems over and separates any forks from the main stems. Once the stems are knocked over by heavy leaf fall or snowfall, finding a stem of a Ginseng plant is nearly impossible.
Frank