I think Harvey was the child of Ida Taylor, not MaryAnn Buzby Taylor. In the 1900 US Census for Upper Township, Cape May, NJ Harvey is seen in the household of Walter Taylor (Maryann's son). Walter lists Harvey as his Nephew, not Brother.
New Jersey, Births and Christenings Index, 1660-1931, found on Accestry.com lists his mother as Ida Taylor (Walter's sister).