It is the first time since the Franco dictatorship and Spain's transition to democracy in 1975 that a Spanish prime minister has lost a no-confidence vote.
The vote by MPs in the Congress of Deputies - the lower house - ends seven years of rule by Mr Rajoy and his Popular Party (PP).
What support does Mr Sánchez have?
It is a dramatic turn of events, as the Socialists (PSOE) have just 84 seats in the 350-seat Congress.
Mr Sánchez managed to hammer together an odd mixture of parties to defeat Mr Rajoy, whose party was discredited by a massive corruption scandal. The parties do not appear to be very comfortable bedfellows.
Senior Popular Party officials were jailed over bribes and money-laundering
As a minority government, it could prove to be unstable.
Mr Sánchez is expected to be sworn in at the weekend and to name a cabinet next week.
He suffered two humbling election defeats, in 2015 and 2016, but bounced back months later, winning a PSOE leadership race.
In the June 2016 election the PSOE's support was eroded by radical leftist newcomers Podemos.
In return for having backed Mr Sánchez in the parliament vote, Podemos is likely to demand significant policy concessions from the PSOE, and perhaps some key cabinet posts.
Mr Sánchez is likely to be challenged strongly over his plan to stick to the Rajoy budget. It is not yet clear how soon - if at all - Mr Sánchez will call an early election.
Mr Rajoy's opponents have been vociferous in denouncing the PP's spending cuts, and now they will want to see a major change of direction under Mr Sánchez.
What are the economic challenges?
Podemos condemned the bitter austerity medicine prescribed by the EU - and implemented by the PP - to address Spain's economic crisis after the 2008 financial crash.
Spain now has a trade surplus, but very high levels of debt and its unemployment rate is among the highest in the EU, at 17.4%. The youth unemployment rate is running at 35% - the highest after Greece.
Public debt is close to 100% of total economic output (GDP) - uncomfortably high, though lower than the debts of Greece and Italy.
On the plus side, the European Commission has forecast GDP growth in Spain to reach 2.6% this year.
For key players in the EU, the Sánchez government will be less of a worry than the unprecedented new populist government in Italy.
Prof Xavier Torrens, a politics lecturer at the University of Barcelona, says the new Spanish government "will probably focus its efforts on economic redistribution, given its social-democratic tendencies, rather than solely on economic growth, as the Rajoy government did".
What about Catalonia and the nationalists?
The pressure for Catalan independence remains strong
Mr Sánchez is firmly against Catalan independence and opposed the Catalan separatists' controversial referendum last October.
This could be a thorny issue for his government, after Catalan parties backed his no-confidence motion on Friday. They said it was more a case of "rejecting the PP" than of backing Mr Sánchez.
But before the vote, Mr Rajoy's government formally accepted a list of cabinet ministers proposed by the separatist politician who will head Catalonia's regional administration, Quim Torra. The cabinet is expected to be sworn in on Saturday.
Mr Torra replaced four jailed and self-exiled candidates whom Madrid had refused to accept.
Mr Sánchez could now speed up the return of Catalan autonomy, which was suspended by Madrid after the separatists declared independence last October.
Mr Sánchez has recognised Catalonia and the Basque Country to be nations within Spain, not just regions. Prof Torrens told the BBC that "this is symbolically important, because Mr Rajoy never did so".
There remains much bitterness between the Madrid political establishment and the separatists. Spanish prosecutors want to put the separatist leaders on trial, including Carles Puigdemont and others who fled abroad.
Basque nationalists also backed Mr Sánchez, and they too could now push their own agenda.
The decision of the separatists Eta last month to disband, after decades of violence, was a victory for Madrid. But there is Basque pressure to get Eta militants, currently in Spanish jails, moved closer to their Basque Country families.
More about: Spain