Thien Ton Pagoda and Cave were discovered during the era of the Hung Kings. The cave is dedicated to Thien Ton, a deity from 10th-century Hoa Lu legends. According to village legends from Bich Dao and Dai Phong in Ninh Binh, Thien Ton was originally a celestial god, born as a prince. His mother gave birth to him on March 3rd, during the 25th year of the Khai Hoang reign The prince, named Huyen Nguyen, grew up to be exceptionally strong and ventured across the world. He trained for 42 years in Dung Duong Mountain (in Hoa Lu, Ninh Binh), where he attained spiritual powers, enabling him to fly, transform, and vanquish evil spirits.

In 938, Cao Do, an influential court minister, ordered the construction of a temple at the entrance of Thien Ton Cave. A statue was carved, depicting the god with a sword in hand, standing on a turtle and serpent. He was conferred the title "Trấn Vũ An Quốc Đại Vương" . Later, when Emperor Dinh Tien Hoang led his uprising at Hoa Lu, he was aided by the god's turtle and serpent generals. As a result, Thien Ton was further honored with the title of "An Quốc Emperor," becoming a guardian deity of the eastern gate of Hoa Lu's capital.

Thien Ton Cave is a complex of caves, pagodas, and temples located at the eastern entrance to the ancient capital of Hoa Lu. The cave consists of two parts: the Outer Cave and the Inner Cave. At the entrance of the Inner Cave stands a stone Long Đĩnh (dragon throne), intricately carved with clouds and dragons. Inside the Long Đĩnh is a bronze statue of Thien Ton standing on a turtle. The statue is vibrantly decorated, with the god holding his sword before his chest, the tip resting on the turtle’s back. Hence, the cave is named Thien Ton Cave.

Thien Ton Pagoda and Cave face south and are part of the four key structures built to guard the cardinal points of Hoa Lu. These include: Thien Ton Temple in the east, Cao Son Temple in the west, Duc Thanh Nguyen Temple in the north, and Quy Minh Temple in the south. The pagoda and cave are situated in Dung Duong Mountain, Thien Ton Town, Hoa Lu District, about 7 kilometers north of Ninh Binh city. "Dung Duong Mountain" or "Vu Duong Mountain" means a mountain that blocks strong water currents flowing down. The three characters for "Dung Duong Mountain" are inscribed at the entrance to the cave dedicated to Trấn Vũ Thien Ton. Trấn Vũ Thien Ton, also known as Chân Vũ Emperor, is an incarnation of the Celestial Emperor Huyền Thiên Thượng Đế Kim Thuyết, commander of the heavenly armies, sent to earth to quell evil and restore order.

Thien Ton is a land of "majestic water and strange mountains," with its harmonized geography of mountains and rivers providing a strong defensive position. Both King Dinh Tien Hoang and King Le Dai Hanh used this cave as an outpost and lookout for Hoa Lu and as a reception point for foreign envoys before entering the capital. Today, the cave still houses many valuable cultural and historical artifacts, such as 18 Arhat statues, intricately carved incense altars, and stone thrones. Notably, there is a large bell with four knobs that produce four distinct sounds, dating back to the reign of Cảnh Hưng (1740 - 1786).

Due to its unique cultural, historical, and spiritual significance, the Thien Ton Pagoda and Cave complex was recognized as a national historical and spiritual site in 1962. The annual festival of Thien Ton Pagoda and Cave takes place from the 6th to the 8th day of the third lunar month.