Drury May was one of the earliest settlers in Fayette County, arriving before 1824, when he was first listed in the Fayette County Tax Digest. He had removed from Pulaski County, where he was listed in the 1820 census. He had resided in Pulaski County from at least 1813, at which time he enlisted for War of 1812 service. Drury May was born in North Carolina, and likely came to Georgia with his parents as an infant, perhaps around 1787. He was the youngest of seven children thought to have been born to James May (c.1735-1789) and Lydia Bishop of Wilkes County.

In 1809, Drury May (1783-1840) married Anna Moses (1785-1839), widow of John Moses of Randolph (Jasper) County. In 1812, he was named guardian for David, Nancy and Hiram Moses, the three children from his wife's prior marriage. At the time of the 1824 Tax Digest, Drury May held over 405 acres in Fayette County, as well as over 1,400 acres in Henry, Pulaski and Twiggs Counties. His Fayette County land holdings continued to grow, and, by the time of the 1834 Tax Digest, he held 1,619 acres in Fayette County, primarily in District 6, as well as 810 acres in Pulaski County.

During this time in history, the South, particularly Georgia, was made up of many slave holding plantations. Drury May's was no exception. He would have been considered a prosperous planter, with a large part of his wealth made up of the forty slaves he eventually owned. By the time of his death, his plantation in Fayette County comprised over 2,000 acres, on the west side of Whitewater Creek, along present-day Redwine Road, about one-half mile south of today's Peachtree City on Highway 74. His early 1830s spacious antebellum home has been restored, and has been featured in various publications, known in later years as the Swain Peeples home.