Smith was, along with Stéphane Grappelli, Svend Asmussen and Joe Venuti, one of jazz music's preeminent violinists of the swing era. He was born in Portsmouth, Ohio in 1909 and studied violin with his father. Smith cited Louis Armstrong as his primary influence and inspiration to play jazz, and like Armstrong, was a vocalist as well as instrumentalist. In the 1920s, he played in Texas as a member of Alphonse Trent's band. After moving to New York he had a regular gig with his sextet at the Onyx Club starting in 1935 and also performed with Coleman Hawkins as well as with younger musicians such as Charlie Parker and Dizzy Gillespie, and later, Sun Ra.
After being signed to Vocalion in 1936, he had a big hit with "I'se A Muggin'" and was billed as Stuff Smith and his Onyx Club Boys. He recorded for Vocalion in 1936, Decca in 1937 and Varsity in 1939-1940.
He is featured in several numbers on the Nat King Cole Trio album, After Midnight.
Part of Smith's performance at what is considered the first outdoor jazz festival, the 1938 Carnival of Swing on Randall's Island, turned up unexpectedly on audio engineer William Savory’s discs, which were self-recorded off the radio at the time, then long-sequestered. Some newsreel footage had survived, but no audio of the festival had been believed to have, until the discs were acquired and studied by the National Jazz Museum in Harlem, in the person of its executive director Loren Schoenberg, in 2010.
Smith was critical of the bebop movement, although his own style represented a transition between swing and bebop. He is credited as being the first violinist to use electric amplification techniques on a violin. He contributed to the song "It's Wonderful" (1938) often performed by Louis Armstrong and Ella Fitzgerald throughout their careers. Smith moved to Copenhagen in 1965, performed actively in Europe, and died in Munich in 1967. He is buried at Klakring Cemetery in Jutland, Denmark.
Stuff Smith is one of the 57 jazz musicians photographed in the 1958 portrait A Great Day in Harlem.