Australia's government announced the decision on Saturday, reversing decades of Middle East policy, but said it would not immediately move its embassy there.
The United States in May opened its embassy in Jerusalem amid fervent opposition from the region as well as the European Union. Only Guatemala and Paraguay followed suit, but Asuncion later reversed this decision.
The Arab League had issued a statement criticizing the Australian decision as "blatantly biased towards the positions and policies of the Israeli occupation".
But Bahraini Minister Sheikh Khalid bin Ahmed al-Khalifa described the statement as "mere rhetoric and irresponsible".
"Australia's stance does not impact the legitimate Palestinian demands, first among them being East Jerusalem as the capital of Palestine, and it does not contradict the Arab Peace Initiative," he tweeted on Saturday.
The post received angry responses from social media users accusing Sheikh Khalid of supporting Israel.
Sheikh Khalid has previously said Israel had the right to defend itself against Shiite Muslim Iran, which Bahrain blames for stoking unrest in the Sunni-ruled island state. Iran denies interfering in Bahrain.
The status of Jerusalem, home to sites holy to the Muslim, Jewish and Christian faiths, is one of the biggest obstacles to a peace agreement between Israel and Palestinians who want East Jerusalem recognized as the capital of an independent Palestinian state.
Israel regards all of Jerusalem as its capital, including the eastern sector that it annexed in a move not recognized internationally, after the 1967 Middle East war. The United Nations says the status of Jerusalem can be resolved only by negotiations.
Israel Prime Minister Benjamin Netanyahu has on several occasions hinted at warmer relations with Gulf Arab states and made a surprise visit to Oman in October to meet with its ruler Sultan Qaboos bin Said.
Israel has diplomatic relations with only two Arab states, Egypt and Jordan.