Torsmo's origins lie in the 1700s, when it functioned as a fäbod — a summer pasture settlement used by farmers from a home village to graze livestock away from the main fields. Over time it developed into a more permanent village.
When the writer and folklorist Karl-Erik Forsslund visited in 1916, he recorded 26 farms in Torsmo. The oldest of these was reportedly "Bilar Jugas", with a dwelling house dated to 1746 and a storehouse (härbre) dated even earlier, to 1691. Other named farms included Staffasgården, Snällgården and Ollargården, suggesting a settlement of some size and continuity by the early twentieth century.
Between 1920 and 1937 the village supported a small general store, run by Anna-Stina Andersson, indicating that Torsmo remained an active community into the twentieth century rather than being abandoned after its fäbod origins.
The surrounding landscape is notable in its own right: Torsmo borders Orsa Besparingskog, an extensive area of forest known for its natural and cultural heritage. Scattered around the village are archaeological traces of earlier activity, including old kilns and similar remains, hinting at a longer history of small-scale industry or land use than the surviving buildings alone suggest.