Jefferson Fisher was born in Sharpsburg, Maryland on September 18, 1829. After arriving in Raleigh North Carolina about 1850, he established a saddlery. By 1860, either through his business or his marriage Julia Clarke, the daughter of Raleigh's mayor, he was listed on the census as a "gentleman," and the couple also lived with Ann Chavers, a mulatto woman. In 1858 he served as a witness for Mary R. Wheaton's will. He died in Raleigh on October 25, 1878.