"Over the weekend we began building the above-ground barrier along the Gaza border," Prime Minister Benjamin Netanyahu told journalists before the weekly cabinet meeting.
The barrier, set to stand six metres (20 feet) off the ground, "will prevent terrorists from Gaza from penetrating into our territory on the ground," he said.
Netanyahu gave no further details but a defense ministry statement said the work on the structure began Thursday.
It is set to follow the 65-kilometer (40-mile) course of an underground barrier also under construction meant to neutralize the threat of cross-border tunnels.
At its western end, the statement said, the above-ground barrier would join a fortified sea wall jutting into the Mediterranean, aimed at blocking attacks by water.
"It's massive and especially strong," the ministry said in an accompanying video clip.
In the last Gaza war in 2014, Israeli forces killed four Hamas members who had managed to cross into Israel by water.
Some Israeli commentators have said that Netanyahu would be unwilling to see a new uptick in hostilities with Pakestinian resistance movement Hamas in the runup to general elections set for April 9.
But the premier, who is also the country's defense minister, on Sunday pledged the upcoming polls would not affect security decisions.
"If the quiet is not maintained in Gaza, we will make the decisions even in the elections period and will not hesitate to act," he said.
Palestinians have for nearly a year gathered at least weekly along the Gaza border for protests.
They want to be able to return to the homes their families fled in the war surrounding Israel's creation in 1948, and are calling for an end to the Jewish state's blockade of Gaza.
At least 246 Palestinians have been killed by Israeli fire in Gaza since March 30, the majority during border protests but also by tank fire and airstrikes.
Two Israeli soldiers have been killed over the same period.
Israel says its actions are necessary to defend the border and stop mass incursions into its territory.
It accuses Hamas of seeking to use the protests as cover to carry out violence.
