I think it's safe to suggest that not wearing white after Labor Day is a pretty silly concept, so please don't ditch your whites until next year.
If white is way too bold for you to wear in the fall think of pairing your favorite pieces with fall fabrics and colors. Like a white blazer with a navy skirt or a wool fedora with white jeans. Also, when purchasing new items choose ivory pieces which are less jarring.
Either way don't stop wearing white just because Labor Day has come and gone. You paid good money for what's in your closet and it deserves to be worn.
P.S. Fun MADE IN AMERICA post coming this weekend!