The oldest known record of Panga village dates back to 1554. In the late 19th and early 20th centuries, Panga village was one of the largest areas in the region, where fishing and shipbuilding were carried out.
Sick eyes were washed in the Silmaallikad springs on the Panga cliff. To receive the healing effects of the spring water, people would throw coins into the spring.
On the high ledge of Panga cliff, there was a sacred grove, a historic place for prayer and making fires, and a sacrificial site called "mari kakk" in the sea. Every year, a person or an animal was taken there as an offering to the sea god. Later on, beer or vodka was poured as an offering to the sea god. On Christmas Eve and New Year's Eve, people went there to find out whether they would have good luck or bad luck. The weather was also predicted at the same spot. The lighthouse on Panga was built during the Soviet era, probably in the 1960s. There was no lighthouse on Panga before that.