Mr O'Reilly has been on holiday since 12 April, and was photographed shaking hands with Pope Francis at the Vatican earlier on Wednesday.
More than 50 sponsors have withdrawn ads from his show, The O'Reilly Factor.
Pressure on Fox has mounted since recent reports that five women received $13m in payouts because of Mr O'Reilly's behaviour.
The prime-time presenter, who has been with the US network almost since its inception, has denied all of the allegations.
The decision comes after an emergency meeting of the 21st Century Fox board, New York Magazine reported.
Fox broadcaster Tucker Carlson will be taking over Mr O'Reilly's 20:00 time slot, the company said in a follow-up statement.
In an internal memo sent to 21st Century Fox employees that has been shared by US media, the company writes the decision was made after "extensive review done in collaboration with outside counsel".
The letter - which is signed by Rupert, Lachlan, and James Murdoch - praised Mr O'Reilly as "one of the the most accomplished TV personalities in the history of cable news".
"We have full confidence that the network will continue to be a powerhouse in cable news," the statement continues, adding that they are committed to "fostering a work environment built on the values of trust and respect".
The sexual harassment charges against Bill O'Reilly aren't new. They've bubbled up before and have been quietly ushered from view, thanks to multimillion dollar legal settlements.
Now, however, they have brought down the top-rated conservative talk show on cable news.
So what changed? Was it lurid details of his alleged behaviour? The protests? Commercial sponsors abandoning his show? The impact the scandal could have on 21st Century Fox's attempts to purchase European broadcaster Sky?
All probably contributed to O'Reilly's exit. The crack in the proverbial dam, however, came when Fox News head Roger Ailes was sacked for his own sexual harassment scandal.
If Mr Ailes, who built the conservative television empire, could be laid low by his tawdry behaviour, no one was untouchable - even Mr O'Reilly.
Image caption Mr Ailes was sacked over a harassment lawsuit filed by anchor Gretchen Carlson (L)
Welcome to Fox News in the Donald Trump era. Mr Ailes is out. Mr O'Reilly is gone.
Presenter Megyn Kelly has bolted to NBC. The only giant left standing is Trump super-fan Sean Hannity.
A network that defined itself by blanket opposition to Barack Obama now appears adrift, challenged by more aggressive conservative media outlets and toxic to the left.
It's a tricky position to occupy - a victim of its own success… and hubris.
More about: #BillOReilly
