Looking for Information on Walker family from Alabama that moved to Texas in 1850s

Hi,
I haven been desperately searching for more information on my Walker ancestors. My maiden name is Walker and it has been very frustrating to not find information on my family. My great grandfather was Adair Darling Walker born in 1897 in McClennan County Texas. His parents were William M. Walker and Alice Virginia Smith. I have been able to determine that William Walker's father was a Wylie or Wiley Walker born about 1825 in Alabama. I have been able to find Wylie Walker on census records from 1870-1910. I cannot find anything on him before 1870 nor can I find anything about where or when he may have died. If anyone has any information that would help me find more of my Walker family I would be so happy!