Mr Trump made the threats amid fallout from the US assassination of Iranian commander Qasem Soleimani.
The president said cultural sites were among 52 identified Iranian targets that could be attacked if Iranians “torture, maim and blow up our people”.
But the UN’s cultural organisation and UK foreign secretary were among those to note that such sites were protected.
The US and Iran have signed conventions to protect cultural heritage, including during conflict. Military attacks targeting cultural sites are considered war crimes under international law.
Qasem Soleimani was killed in a US drone strike in Baghdad on Friday on the orders of Mr Trump. The killing has sharply increased regional tensions, with Iran threatening “severe revenge”.
What were the president’s threats?
The first came in a series of tweets on Saturday.
Mr Trump said the US had identified 52 Iranian sites, some “at a very high level and important to Iran and the Iranian culture”, and warned they would be “hit very fast and hard” if Tehran carried out revenge attacks on US interests or personnel.
Secretary of State Mike Pompeo appeared to try to soften the threat by saying the US would act within international law.
But the president later repeated his threat, saying: “They’re allowed to kill our people, they’re allowed to torture and maim our people, they’re allowed to use roadside bombs and blow up our people – and we’re not allowed to touch their cultural site? It doesn’t work that way.”
On Monday, White House adviser Kellyanne Conway defended the president, saying he had not said he was targeting cultural sites, only “asking the question”.
She also said: “Iran has many strategic military sites that you may cite are also cultural sites”, before later clarifying her remark to say she was not suggesting Iran had camouflaged military targets as cultural sites.
Defence Secretary Mark Esper was later asked if the US would target cultural sites, and said: “We will follow the laws of armed conflict.”
When asked if that meant no, “because targeting a cultural site is a war crime?”, he responded: “That’s the laws of armed conflict.”
What criticism did his comments draw?
The director general of the UN’s cultural organisation, Unesco, Audrey Azoulay, said both Iran and the US had signed a 1972 convention to protect the world’s natural and cultural heritage.
They have also both signed a 1954 convention protecting cultural property in the event of armed conflict. Mr Trump withdrew the US from Unesco in 2018, citing alleged anti-Israeli bias.
US Democratic senators Elizabeth Warren and Chris Murphy said Mr Trump was “threatening to commit war crimes”, echoing similar statements by Iran’s Foreign Minister Mohammad Javad Zarif.
On Monday, UK Foreign Secretary Dominic Raab said cultural sites were protected by international law, and Britain expected that to be respected.
The wider region has suffered many cultural attacks carried out by the Islamic State group, which targeted mosques, shrines, churches and famous sites such as Palmyra in Syria. The Taliban in Afghanistan destroyed the world’s tallest Buddha statues, in Bamiyan province.
Iran’s top cultural sites
Iran is home to two dozen Unesco World Heritage sites. These are landmarks the UN body believes need preserving for their cultural, historic or scientific significance. They include:
Persepolis, the capital of the ancient Persian Achaemenid Empire, whose earliest remains date back to the 6th Century BC.
Naqsh-e Jahan Square in the city of Isfahan, which was built in the early 17th Century and is one of the largest city squares in the world.
Golestan Palace in Tehran, the residence and seat of power for the Qajar dynasty which ruled Iran from 1785 to 1925.
There are also a number of sites which – while not listed by Unesco – still retain huge cultural importance.