Justice S Muralidhar, who was hearing a petition into the religious riots, had sharply condemned both the government and police on Wednesday.
Orders for his immediate transfer came late at night the same day.
More than 30 people have been killed so far in the deadliest violence the Indian capital has seen in decades.
The clashes first broke out on Sunday between protesters for and against a controversial citizenship law in north-east Delhi.
But they have since taken on communal overtones, with reports of many Muslims being attacked.
Even though the violence largely abated on Wednesday, there were reports of sporadic clashes in affected areas overnight and the city remains tense.
On Thursday, focus had shifted to Justice Muralidhar’s transfer from the Delhi high court. His move was first announced nearly two weeks before the violence broke out, but BBC correspondents say that his biting comments in court may have hastened his transfer.
However, the government has insisted that the move was not politically motivated.
The case was taken up again on Thursday and a new bench gave the government a month to respond to the petitions against them.
While hearing petitions about the violence, the judge said that the court could not let “another 1984” happen on its “watch”. In 1984, more than 3,000 Sikhs were killed in riots against the community in Delhi.
Videos showing leaders from India’s ruling Bharatiya Janata Party (BJP) inciting Hindu crowds against largely Muslim protesters were played during the cases he was hearing.
Justice Muralidhar then questioned how police were registering complaints and directed the government to ensure that any displaced victims were given temporary shelter as well as medical treatment.
His comments made headlines on Wednesday, with many praising his “courageous stand”.
So the news that he had been removed saw many Indians express concern and outrage.
But the government has maintained that the transfer was done with the judge’s consent and a “well-settled process” was followed.
Justice Muralidhar has not commented on the development.
The law minister, Ravi Shankar Prasad, took to Twitter, where most of the outrage was being expressed, to say it was only a “routine transfer”.
But, a former Delhi High Court judge questioned the timing of the move, asking, “what was the hurry?”
In an interview with news channel NDTV, Kailash Gambhir called the timing of the order “disturbing”.
“The language of the transfer order is not routine,” he added.
Opposition parties also remained unconvinced – and continued to blame the government for the violence.
Opposition leader Sonia Gandhi called for the resignation of Home Minister Amit Shah, saying he was “responsible”.
Delhi Chief Minister Arvind Kejriwal has also been criticised for not coming out when the violence first broke out.