I'm not sure what, but there is. I grew up with the understanding that the Stars and Bars represented rebellious, stick-it-to-the-man, drinking/fighting/fishing Southern folk that live just a bit outside the law.
I never associated it with white supremacy.