The southern part of Kållerstad village once held a stave church built of oak, though little is recorded of its history. A bishop's visitation in 1746 prompted the parish to build a new church, and a cruciform timber church was consecrated in 1764, reportedly already complete by 1759. That building was demolished in 1861, its timber reused in the construction of the parish school. Notably, its painted ceiling of 1770, decorated with naive figurative scenes by the district painter Sven Niclas Berg of Ölmestad, survives and is stored in the attic of Ås church.
The present church was built between 1856 and 1858 as a masonry hall church of granite, under master builder Johannes Bengtsson of Bökhult, who later built the similar Jälluntofta church nearby. It was consecrated in 1861 by Bishop Henrik Gustaf Hultman. The nave has a broad, rounded chancel to the east with a sacristy to its north, while a tower with a lantern rises in the west. The design reflects the restrained Empire-style architectural ideals of the period, with simplified forms and round-arched openings; the exterior is rendered and limewashed white, and the chancel roof, with its rounded hip, is clad in copper.
Inside, the church largely retains its original Empire-style character, with much of its historic fittings surviving alongside a sense of spacious simplicity. A gallery with an organ stands at the west end. A renovation in 2001–02 replaced the electric heating system and gave the interior a wholly new colour scheme.
Among the church's furnishings is an altarpiece from 1867 depicting the Transfiguration of Christ, painted by Sven Johansson of Karaby after an original by Fredric Westin, itself inspired by a version seen in Svenljunga church. The altarpiece's frame, the numeral boards, and probably the pulpit are attributed to the joiner Joel Spets of Sjöbo, who also worked on Jälluntofta church. A disused pulpit, dated to 1642 and made by a master Jacob, is preserved in the gallery; it was purchased in 1722 from Södra Unnaryd church, along with its canopy, both conserved in 2010.
The church organ, built in 1875–77 by Carl Johannes Carlsson of Virestad and Johannes Andersson of Långaryd, is a mechanical instrument with eleven stops, retaining its historic character.