According to editor Jamie Richardson and researcher Jim Richardson at the Leather Sleeves newsletter, "It seems with the advent of organized sports, there was a need for uniforms. There was an additional need for identifications which was satisfying by the use of emblems or letters.

In 1865, the Harvard baseball team added an old English 'H'. The 'H' was embroidered on the gray flannel shirt. The football team started to use the 'H' in 1875. It is interesting to note that for 25 years following the introduction in 1865 of the letter, it was the practice for the team captain to allow certain players who played in the most important games (Yale or Princeton) to keep the 'H' jerseys as an award. If a player did not play in an important game, the player had to return the jersey at the end of the season. Awarding the 'H' jersey may have been the birth of the varsity letter as an award. The letterman sweater was first regularly used by the 1891 "Nine" (baseball) and was black with a small Crimson 'H' on the left breast.

It is not known when the letterman sweater came to high schools. The earliest example that VLAS has come across is in the 1911 yearbook of Phoenix Union High School, Arizona Territory. Pictured, not in football uniform, wearing a V-net sweater with the letter 'P' on the left side is a student in a group photo."