The church was built of local sandstone in the first half of the 1100s, with the tower added slightly later. Dendrochronological analysis of the roof timbers suggests the pine used for the trusses was felled in Västergötland around 1142–1172, possibly as narrowly as 1145–1155, while a beam in the tower comes from a tree felled in the winter of 1274/1275. The church is first mentioned in documents in 1310. During the 1400s the building was given vaulted ceilings, with the roof trusses adapted accordingly.
The tower originally carried a tall, pointed spire. In the 1600s, under the direction of Per Brahe the Younger, the upper half of the spire was removed and the tower given a flat platform surrounded by an iron railing. From 1692 to 1811 the church served as a schoolroom, known as Schola Brahea, and pupils are said to have used the tower platform for astronomical observation. Afterwards the building was mostly used for storage and stood empty for long periods, apart from a brief spell as a mission chapel in the early 1880s, until it was restored to church use in 1922, which gave it much of its present appearance.
The interior is notable for its wall paintings, with layers dating from the medieval period and the 1600s. The medieval images depict legends of saints, while the seventeenth-century paintings, attributed to Johan Werner the Younger, reflect the building's schoolroom era. Werner is also credited with relief figures on the pulpit.
Among the furnishings are wooden figures of the four evangelists and an altar front, both attributed to Werner and thought to date from around the mid-1600s; these are believed to have originally stood in the nearby Brahekyrkan before being moved to Kumlaby at an unknown date. A small organ built in 1953 was later moved to the Brahekyrkan in 1970, and a harmonium has stood in the church since.
Near the church stands the so-called Brahestenen, a memorial to Per Brahe the Younger created in 1653, a reminder of the family's long association with the site and its unusual schoolroom chapter.