Mr Rajoy has been replaced by his socialist rival Pedro Sanchez after deputies voted 180 votes to 169 to make him the new PM, with one abstention.
The outgoing centre-right prime minister, who has been in office since 2011, was kicked out of office following a series of corruption scandals that rocked his government and led to ministerial resignations.
Mr Sanchez’s centre-left PSOE group was backed by the anti-austerity Podemos group and a constellation of small regional parties in the no confidence vote that brought down Mr Rajoy’s government, which has hobbled on as a minority administration since an election in 2016.
Under the Spanish system, a no confidence vote must also propose a new prime minister to replace the one being voted out – with Mr Sanchez now set to take the reins.
If Mr Rajoy had survived today’s vote he was expected to face further challenges from other opposition parties hoping to oust him on their own terms.
Centre-right liberals Ciudadanos, who are riding high in the polls, wanted fresh elections to take place as soon as possible. The liberals, who have gained political ground by staunching opposing Catalan independence, refused to vote for the PSOE motion and were the only major party to back the Partido Popular.
Speaking ahead of the vote, with the numbers against him, Mr Rajoy told the congress of deputies: “Pedro Sánchez will be the prime minister of the government and I want to be the first to congratulate him.
“It has been an honour to be the prime minister of the government of Spain. It has been an honour to leave a better Spain than the one I found. Hopefully my replacement can say the same on his day, I wish it for the good of Spain.”
Incoming PM Mr Sanchez said: “Today, democracy has won.” He added: “A new era in Spanish politics is beginning. I am reaching out to all the parliamentary groups to open these new times and I hope that we are all up to the responsibilities that we have ahead of us.”
Speaking after the vote, Pablo Iglesias, the leader of Podemos, opened the door to the left-wingers participating in a PSOE government led by Mr Sanchez.
He said he wanted to join forces to “leave behind a time of corruption, inequality and confrontation” and build “a Spain that no one wants to leave” with “a plural and stable government”.
Such an arrangement could take a similar form to the situation across the border in Portugal, where the centre-left socialist party has run an anti-austerity government with the backing of the radical left and communists. Even with the support of the left-wingers Mr Sanchez will find it hard to marshal a majority in the congress of deputies, however, making early elections likely.
The situation in Spain is being watched cautiously from Brussels, which wants to avoid political instability breaking out across the south of the eurozone. A new populist government was agreed in Italy on Thursday night led by the Five Star Movement politician Giuseppe Conte, with the backing of the far-right League group.
More about: Spain
