The academy - which runs the Oscars - said this was done in accordance with its standards of conduct.
TV star Cosby was convicted of sexual assault last month. Oscar-winning director Polanski admitted statutory rape of a 13-year-old girl in 1977.
Producer Harvey Weinstein was kicked out last year, following numerous allegations of sexual assault.
Less than a year after the downfall of the producer the #MeToo movement is catching up with other men who abused their power, the BBC's James Cook in Los Angeles reports.
Neither Cosby nor Polanski have publicly reacted to the academy's decision.
Cosby's wife, Camille, described his conviction as "mob justice, not real justice".
"This tragedy must be undone not just for Bill Cosby, but for the country," she said.
What did the academy say?
The prestigious organisation made the announcement on Thursday - two days after its board members voted on the issue.
In a statement, it said its board "has voted to expel actor Bill Cosby and director Roman Polanski from its membership in accordance with the organisation's Standards of Conduct".
"The Board continues to encourage ethical standards that require members to uphold the Academy's values of respect for human dignity," it added.
Only four people are known to have been expelled in its 91-year history.
The first was actor Carmine Caridi, who had his membership revoked in 2004 after he allegedly sent confidential film preview videos to a friend which ended up online.
On social media, many people have been asking what took the academy so long to take action against Polanski, who has been honoured in the decades since he admitted to statutory rape.
More about: Oscars
