IS took control of Palmyra in May, sparking fears the group might demolish the Unesco World Heritage site.
The group has destroyed several ancient sites in Iraq.
IS "placed a large quantity of explosives in the temple of Baalshamin today [Sunday] and then blew it up causing much damage to the temple," Syrian antiquities chief Maamoun Abdulkarim told AFP news agency.
"The cella (inner area of the temple) was destroyed and the columns around collapsed," he said.
Residents who had fled from Palmyra also said IS had planted explosives at the temple, although they had done it about one month ago, according to the Syrian Observatory for Human Rights.
Palmyra is famed for its well-preserved Greco-Roman ruins.
The BBC World Service arts correspondent Vincent Dowd says the temple built nearly 2,000 years ago was primarily a Roman era artefact. It was dedicated to the Phoenician god of storms and fertilising rains.
He said it was "extremely important" and had been "almost totally intact".
Last month, IS published photos of militants destroying what it said were artefacts looted at Palmyra.
A week ago, it emerged that the archaeologist who had looked after Palmyra`s ruins for four decades had been beheaded by the militant group.
Mr Abdulkarim said the 81-year-old had refused to tell IS where some treasures had been hidden to try and save them.
The group has also published photos of what they said was the destruction of two Islamic shrines near Palmyra, which they described as "manifestations of polytheism".
The modern city of Palmyra - known locally as Tadmur - is situated in a strategically important area on the road between the Syrian capital, Damascus, and the eastern city of Deir al-Zour.
Nearby, the monumental ruins of the ancient city rise out of the desert. Unesco and others consider the site one of the most important cultural centres of the ancient world.
More about:
