I blundered into that morass when I moved from Ohio to North Carolina. (Ohio, of course, was the birthplace of such luminaries as U.S. Grant, William T. Sherman, and Philip Sheridan.)
Actually there is some justification for the distinction. If we define "civil war" as a war to determine what faction is to be in control of the country's government, the unpleasantness between 1861 and 1865 was, indeed, not a civil war. The Confederacy had no intention of taking over the United States government; it was trying to break loose of (or secede from) it.
The English Civil War, the Russian Civil War, and the Spanish Civil War were honest-to-goodness civil wars.
On the other hand, no less a distinguished Southerner than Shelby Foote titled his three-volume masterpiece The Civil War: A Narrative. That's a pretty strong Southern endorsement for the term. (I still think they should have gotten him to play Lee in the movie "Gettysburg.")
The depth of emotions about the conflict in question in this part of the country is sometimes downright mind-boggling. I strongly recommend a book called Confederates in the Attic: Dispatches From an Unfinished Civil War, by Tony Horwitz. One of the funniest and, simultaneously, scariest books I've ever read.
Another little bit of historical trivia: legally it wasn't a war. The Congress didn't declare war on anybody - because there was no enemy nation to declare war on. (Neither the U.S. nor any other country ever officially recognized the Confederate States of America as a nation.) What happened between 1861 and 1865 was, in official legal terms, an unsuccessful rebellion.
The students in my American military history courses are sometimes surprised to hear that the United States, in its entire history, has actually declared war five times: against Britain in the War of 1812, against Mexico in the Mexican-American War of 1846-1848, against Spain in the Spanish-American War of 1898, against Germany and its allies in World War I, and against Germany, Italy, Japan, and their allies in World War II.