When he was twenty, Fitch moved from Montville to Norwich and became an apprentice wagon maker and he remained occupied at that calling for a number of years, doing business on his own account at Norwich Town. Later he also turned to cabinet making.

A good looking young man in his early twenties, Fitch was out in Norwich one day in 1882 when he noticed some teenage girls standing on a bridge and talking. He was mightily impressed by one, and said to himself "that's the girl I'm going to marry". Mary Williams, or Mamie as she was known, was an attractive, independent, fourteen year old at the time. Fitch followed his instinct and became acquainted with her family. When she was sixteen he proposed marriage and they became engaged. Then, with remarkable patience, they waited until she was eighteen before being wed. Two years later, daughter Natalie was born after a difficult delivery witnessed by Fitch. He then vowed never to put his wife through that ordeal again, thus making Natalie their only child.

Fitch bought a house in Norwich and had it moved to another lot at 24 Town Street, where Natalie grew up. Unfortunately the house burned in the late 1900s. He was a member of the First Congrgational Church in Norwich Town where he was a deacon and sang second bass in the choir.

In 1900 he went to work for the Gas and Electric Company when it was privately owned and was assistant superintendent. When it was made a municipal plant he became a master mechanic. The 1910 census shows him as a meter tester for the Gas & Electric Co. The story passed down is that in that job he was exposed to gas or chemicals that led to his early death in 1917, at age 57, His obituary states that he died in the middle of the night of a heart attack.