The lyrics of "Southern Man" are vivid, describing the racism towards blacks in the American South. In the song, Young tells the story of a Southern man (symbolically the entire South) and how he mistreated his slaves. Young pleadingly asks when the South will make amends for the fortunes built through slavery when he sings, "I saw cotton and I saw black, tall white mansions and little shacks. Southern Man, when will you pay them back?" The song also mentions the practice of cross burning.
Southern rock band Lynyrd Skynyrd wrote their song "Sweet Home Alabama" in response to "Southern Man" and "Alabama" from Neil's 1972 album Harvest. Young has said that he is a fan of both "Sweet Home Alabama" and Ronnie Van Zant, the lead vocalist for Lynyrd Skynyrd. "They play like they mean it," Young said in 1976. "I'm proud to have my name in a song like theirs." Young has also been known to play "Sweet Home Alabama" in concert occasionally. To demonstrate this camaraderie, Van Zant frequently wore a Neil Young Tonight's the Night T-shirt while performing "Sweet Home Alabama." Crazy Horse bassist Billy Talbot can often be seen reciprocating by wearing a Jack Daniels-styled Lynyrd Skynyrd T-shirt (including at the Live Rust concert).
Lynyrd Skynyrd tried to arrange to have Neil Young come on stage during a performance of "Sweet Home Alabama," where he would have sung "a Southern man don't need me around anyhow," but the performers were never able to arrange this performance due to conflicting touring schedules.