He conceded before official results were announced as exit polls showed he was heading for a heavy defeat.
"My experience of government finishes here," Mr Renzi told a press conference after the No campaign won what he described as an "extraordinarily clear" victory in the referendum on which he had staked his future.
The outcome energised the anti-immigrant Northern League party, an ally of French far-right leader Marine Le Pen. The party called for an early general election in Italy.
The risk of political instability in Italy, Europe`s fourth largest economy, led to stocks and the euro falling in early trading in Asia. The euro fell to a 20-month low against the dollar at $1.0505 in after opening at around $1.0685, but later pulled back up to $1.0562.
Italian PM Matteo Renzi resigns after referendum defeat
05 December 2016
Read: 1769
Matteo Renzi, the Italian prime minister, resigned late on Sunday night after losing a constitutional referendum.
