I've lived 55 years in the South and I grew up liking the Confederate flag. I haven't flown one for many decades — but for a reason that might surprise you.I know the South well. We lived wherever the Marine Corps stationed my father: Georgia, Virginia, the Carolinas. My favorite uncle wasn't in the military, but he did pack a .45-caliber Thompson submachine gun in his trunk. He was a leader in the Ku Klux Klan. Despite my role models, I was an inept racist as a kid. I got into trouble once in the first grade for calling a classmate the n-word. He was Hispanic.As I grew up and acquired empathy, I learned that for black folks, the flutter of the Confederate flag felt like a poke in the eye with a sharp stick. And for the most prideful flag-wavers, clearly that response was the point. I mean, come on. It's a battle flag.What the flag symbolizes for blacks is enough reason to take it down. But there's another reason white Southerners shouldn't fly it. Or sport it on our state-issued license plates, as some do here in North Carolina. The Confederacy — and the slavery that spawned it — was also one big con job on the Southern white working class. A con job funded by some of the antebellum one-percenters, and one that continues today in a similar form.You don't have to be an economist to see that forcing blacks — a third of the South's laborers — to work without pay drove down wages for everyone else. And not just in agriculture. A quarter of enslaved blacks worked in the construction, manufacturing and lumbering trades, cutting wages even for skilled white workers.Thanks to the profitability of this no-wage/low-wage combination, a majority of American one-percenters were Southerners. Slavery made Southern states the richest in the country. The South was richer than any other country except England. But that vast wealth was invisible outside the plantation ballrooms. With low wages and few schools, Southern whites suffered a much lower land ownership rate and a far lower literacy rate than Northern whites.My ancestor, Canna Hyman, and his two sons did own land and fought under that flag. A note from our family history says: "Someone came for them while they were plowing one day. They put their horses up and all three went away to the war and only one son, William, came back."Like Canna, most Southerners didn't own slaves. But they were persuaded to risk their lives and limbs for the right of a few to get rich as Croesus from slavery. For their sacrifices and their votes, they earned two things before and after the Civil War. First, a very skinny slice of the immense Southern pie. And second, the thing that made those slim rations palatable then and now: the shallow satisfaction of knowing blacks had no slice at all.How did the plantation-owning one-percenters mislead so many Southern whites?They managed this con job partly with a propaganda technique that will be familiar to modern Americans. Starting in the 1840s, wealthy Southerners supported more than 30 regional pro-slavery magazines, along with many pamphlets and novels that falsely touted slave ownership as having benefits that would — in today's lingo — trickle down to benefit non-slave-owning whites and even blacks. The flip side of the coin of this propaganda is the mistaken notion that any gain by blacks comes at the expense of the white working class.Today's version of this con job no longer supports slavery, but still works in the South and thrives in pro-trickle-down think tanks, magazines, newspapers, talk radio and TV news shows.For example, a map of states that didn't expand Medicaid — which would actually be a boon mostly to poor whites — resembles a map of the old Confederacy with a few other poor, rural states thrown in. Another indication that this divisive propaganda works on Southern whites came in 2012. Mitt Romney and Barack Obama evenly split the white working class in the West, Midwest and Northeast. But in the South, we went 2-1 for Romney.Lowering the flag because of the harm done to blacks is the right thing to do. We also need to lower it because it symbolizes the material harm that the ideology of the Confederacy did to Southern whites, which lasts even to this day.One can love the South without flying the battle flag. But it won't help to get rid of an old symbol if we can't also rid ourselves of the self-destructive beliefs that go with it. Only by shedding those, too, will whites in the South finally catch up to the rest of the country in wages, health and education.Frank Hyman lives in Durham, N.C. He's a carpenter and stonemason and policy analyst for Southern Working Class Political Consulting. This is reprinted with permission of the Richmond Times-Dispatch, where it first appeared.