In 1861, the family were at 181 Trongate St, St. Enoch’s, Glasgow. The record for the family is as follows:
 Agnes Gilmour, head, wid, 47, b. Kilmarnock, AYR
 Jane Gilmour, dau, unm, 20, Herald Newspaper paper. b. Kilmarnock, AYR
 Agnes Gilmour, dau, unm, 15, Machine Worker b. Kilmarnock, AYR
 Robert Gilmour, son, unm, 13, Jeweler b. Kilmarnock, AYR
 Mary Gilmour, dau, 6, b. Glasgow, LNK (at school)
 George Gilmour, son, 4, b. Glasgow, LNK
 Alexander McMillan, Boarder, unm, 21, Blacksmith b. Glasgow, LNK
The number of rooms with one or more windows in this household was 2.

In 1851, the family including James were living in St Enoch’s, Glasgow. James was a Workman Shoemaker age 40 and born in New Cumnock, AYR. Apart from his wife Agnes, also in the household were Jane (10) and Agnes Jnr (8) and Robert (2).

In 1881, the widow Agnes was living with another daughter and her husband and their family at 166 Hospital St, Govan as follows:
 George Fleming, head, 42, Silver Engraver b. Glasgow
 Jane Fleming, wife, 44, b. Kilmarnock, AYR.
 George Fleming, son, 17, unm, Apprentice Engraver b. Glasgow
 Jane Fleming, dau, 16, unm, Message girl b. Glasgow
 James Fleming, son, 14, scholar b. Glasgow
 Robert Fleming, son, 12, scholar b. Glasgow
 John Fleming, son, 10, scholar b. Glasgow
 Agnes Fleming, dau, 8, scholar b. Glasgow
 William Fleming, son, 6, scholar b. Glasgow
 Louisa Fleming, dau, 4 b. Glasgow
 Alexander Fleming, son, 2, b. Glasgow
 Agnes Gilmour, mother-in-law, 67 b. Kilmarnock, AYR.

James and Agnes were still in Ayrshire at Kirkton Holm, Kilmarnock. There was also Jean Gilmour age 6 months and William Gilmour age 4. Presumeably, William died before the 1851 census or else was living away from home at age 14 as he does not appear in the household in 1851.

Married after Banns according to the forms of the Established Church of Scotland at 18 South Coburg Street, Glasgow (home address) Minister David Brown of St Enoch's. Witnesses George Fleming and Richard Barnes
Brides occupation given as Heddle caster and Groom is a silver engraver