The announcement marked one of Russia's most significant retreats and a potential turning point in the war, now nearing the end of its ninth month.
In televised comments, General Sergei Surovikin, in overall command of the war, said it was no longer possible to keep Kherson city supplied.
"We will save the lives of our soldiers and fighting capacity of our units. Keeping them on the right (western) bank is futile. Some of them can be used on other fronts," Surovikin said.
The news followed weeks of Ukrainian advances towards the city and a race by Russia to relocate tens of thousands of its residents.
Shoigu responded: "I agree with your conclusions and proposals. Proceed with the withdrawal of troops and take all measures to transfer forces across the river."
The announcement had been anticipated by Russia's influential war bloggers, who described it as a bitter blow.
"Apparently we will leave the city, no matter how painful it is to write about it now," said the War Gonzo blog, which has more than 1.3 million subscribers on Telegram.
"In simple terms, Kherson can't be held with bare hands," it said. "Yes, this is a black page in the history of the Russian army. Of the Russian state. A tragic page."
More about:
