Mr Turnbull had called the surprise vote himself in a bid to resolve speculation that his leadership was under threat.
In a party room ballot in Canberra on Tuesday, he defeated Home Affairs Minister Peter Dutton.
The prime minister won the vote 48-35, Liberal Party MPs said.
Following the result, he told colleagues: "It's really important that we put our differences behind us and get on with the job of looking after Australians."
Why was there a challenge?
Speculation over Mr Turnbull's future as party leader had escalated in the past few days.
He faced a revolt from conservative MPs over a key climate change policy. That led to him abandoning the plans on Monday in an attempt to blunt conflict within his government.
The PM has also been under long-term pressure due to a series of bad polls and a by-election loss in Queensland.
Polling has suggested the government will suffer a heavy loss in the next general election, which will be held before May next year.
So what happened on Tuesday?
In a bid to clear the air, Mr Turnbull called for a snap vote on the leadership in a party meeting.
Mr Dutton, a leading figure from the party's conservative wing, was nominated as challenger, confirming media reports that he had been gathering support.
Mr Turnbull said he had invited him to remain in his role, but Mr Dutton resigned his cabinet position and will return to the back bench.
More about: Australia
