Next to the church is a shrine built into a Karst Window, known as the Schnurbusch Karst Window, allowing the view of an underground spring. This Karst Window is the result of the collapse of a cave roof, allowing the view of a small portion of a hidden underground stream. Karst topography is common in Perry County, Missouri, which is known to have at least 650 caves.
In 1857, Rev. Joseph Becker installed stone steps to the cave spring so drinking water could be carried to the rectory via buckets and neck yoke. Later, a water line was laid so water could be pumped to the rectory. In 1883, Rev. Francis Krieger had a dam built and installed a hydraulic ram, which brought the water to the rectory by its own power. This provided the water supply until 1927 when the present deep well was drilled.
In the 1950s, Father Michell Deck began development of the Shrine, which was further expanded by Father George Schrammel. Most of the stonework was laid by local Parishioners who used the foundations of old farm homes and barns in the community. The outdoor Way of the Cross or Via Doloroso along with a rosary laid out in the stepping stones were added in the late 1990s by Father Mark Bozada.
The via doloroso leads down a steep path to the bottom of a sink-hole known as the Schnurbusch Karst Window. The shrine features a statue of the Blessed Virgin Mary, under her title of Our Lady of Grace, an altar, and amphitheater. A small waterfall flows out of the cave, and its water flows before the altar, under the stone bridge, and back into the cave.