The Colvin family had a long history of service to Queen and country, both in the military and administration. John Horace Ragnar Colvin was the son of Admiral Sir Ragnar Colvin; the grandson of Clement Sneyd Colvin;^[1] and the great-grandson of John Russell Colvin, lieutenant-governor of the North-West Provinces of British India during the mutiny of 1857, who had ten children and founded a dynasty of Empire-builders. Relatives included Walter Mytton and Auckland, also lieutenant-governor of the North-West Provinces and Oudh. Brenda Colvin (1897–1981)^[2] was an important landscape architect, author of standard works in the field and a force behind its professionalisation. Sidney Colvin was a critic, curator, and great friend of Robert Louis Stevenson.^{[citation needed]}

Colvin married twice. His first marriage was to Anne Manifold in 1948, which ended in 1963.^{[citation needed]} His second was to Moranna Cazenove in 1967.^{[citation needed]} Each marriage produced a son and a daughter.^{[citation needed]}.

His former wife Anne later married Admiral Sir Anthony Synnot, a senior officer in the Royal Australian Navy, and became Lady Synnot.

One of Colvin's children (with Anne) was Mark Colvin, a hugely influential and much loved Australian journalist.^[3] Mark wrote of his father's career in his 2016 book Light and Shadow: Memoirs of a Spy's Son.