From looking at Family Search, his name appears to have become muddled, as John Quincy Addison Payne, which conflicts with the older brother, John Chilton Payne.
The lineage seems correct, but as husband of Elizabeth Givens, his name seems to correctly be Quincy Adams E. Payne. Adam indeed appears in the census, in the proper age range. So I am going to adjust that, and also the WkiTree file. https://www.wikitree.com/wiki/Payne-6820