Speaking flanked by top general as troops paraded in a show of might, Rouhani also sought to reassure the region that the weaponry on display was for defensive purposes and not a threat.
"The region's nations have lived alongside each other for centuries and never had a problem ... If there is a problem, it is caused by others," he said in the speech broadcast live on state television.
"Let us stand together, be together and rid the region of the aggressor's presence."
Rouhani assured neighbouring countries that Iran's armed forces are "never against you or your national interests" but are "standing against the aggressors."
"The power of our armed forces is the power of the region?s countries, the Islamic world", he said.
"If we have a problem in the region today, its roots are either with Zionism or America's arrogance."
Rouhani called on Muslim nations to band together and "restore the historical right of the nation of Palestine," saying that "Zionism ... has been committing crimes in the region for the past 70 years",
"The final victory will surely be with the righteous," he said.
Iran's annual Army Day celebrations are an opportunity for the military to show off its latest weapons but also for its political leaders to try to reassure the region and the international community that they are for defensive purposes only.
The military parade was held next to the south Tehran mausoleum of the Islamic Republic's founder Ayatollah Ruhollah Khomeini.
It showcased Iran's latest weaponry, including the domestically designed and manufactured Kowsar fighter jet, which was first unveiled last year.
Diatribes against Israel are standard fare of the official speeches, although some, such as a call by Rouhani's firebrand predecessor Mahmoud Ahmadinejad for Israel to be "wiped off the map", have triggered international condemnation.
Rouhani has previously called Israel a "cancerous tumour", and called on Muslim governments to unite against it and its US ally.
Iran does not recognise Israel and opposition to the Jewish state has been a central tenet of official policy since the 1979 revolution.
Iran has supported Palestinian radical groups that lay claim to all of historic Palestine and has vociferously opposed the now moribund Middle East peace process under which the Palestinians were offered limited autonomy in the territories captured in the Six-Day War of 1967.
Iranian officials have warned repeatedly that Israel will soon cease to exist, but have usually been careful to underline that that will come about not through a direct attack by Iran.
"In 25 years' time, with the grace of God, no such thing as the Zionist regime will exist in the region," supreme leader Ayatollah Ali Khamenei said in 2015.
The presence of Iran's Revolutionary Guards in Syria supporting President Bashar al-Assad's forces in the eight-year civil war has sharply increased tensions between the two countries.
Israel has said publicly that it has carried out hundreds of air and missile strikes targeting the forces of Iran and its Lebanese ally Hezbollah in Syria.
Prime Minister Benjamin Netanyahu has vowed that he will never allow Iran to establish a long-term military presence in Israel's northeastern neighbour.
Netanyahu has been an outspoken opponent of a landmark nuclear deal Iran signed with major powers in 2015 and was the leading supporter of US President Donald Trump's decision to withdraw from it and reimpose crippling economic sanctions last year.
Washington has sought to forge an anti-Iran axis in the Middle East bringing together Israel and the Gulf Arab states to make common cause against what they see as Iranian "meddling" in the region.
Last week, Washington placed Iran's Revolutionary Guards on its blacklist of "foreign terrorist organisations," the first time it had imposed the sanction on a military arm of a foreign government.
More about: Rouhani