Cambridge Analytica, which denies any wrongdoing, is accused of acquiring data from 87 million Facebook profiles for use in political campaigns.
In opening remarks, shared ahead of the hearing, Mr Zuckerberg apologised for Facebook's tools being used "for harm".
"That was a mistake," he added.
Mr Zuckerberg has been criticised for not taking enough care over how third-parties such as the political consultancy Cambridge Analytica accessed the information of Facebook users.
The social network also faces criticism for failure to tackle fake news.
Several MEPs asked the Facebook chief if he felt that the firm was a monopoly. He was also questioned about whether the company could be trusted to implement necessary changes.
In addition, Guy Verhofstadt MEP asked Mr Zuckerberg if he wanted to be remembered as "the genius who created a digital monster."
Meanwhile, British MEP and leading Brexiteer Nigel Farage expressed his view that Facebook was not a politically neutral platform, asking whether the social network "wilfully discriminated" against right-of-centre commentators.
The meeting between Mr Zuckerberg and the European Parliament's political group leaders had originally been planned to be held in private.
But that sparked a wave of criticism resulting it being livestreamed via the web.
However, the format of the meeting is very different from that of Mr Zuckerberg's testimony to US lawmakers in April.
While the US politicians took turns to cross-examine the Facebook chief in a series of back-and-forth exchanges, the leaders of the European Parliament's various political groups are each asking several questions apiece, and the tech chief is waiting until they are all delivered before responding.
'Committed to Europe'
Ahead of the questions, Mr Zuckerberg made some opening remarks.
"It's become clear over the last couple of years that we haven't done enough to prevent the tools we've built from being used for harm as well," he said.
"Whether it's fake news, foreign interference in elections or developers misusing people's information, we didn't take a broad enough view of our responsibilities."
Mr Zuckerberg added that the company was "committed to Europe", with plans to employ 10,000 people across 12 European cities by the end of the year.
"I expect this will significantly impact our profitability," he said, adding that "keeping people safe will always be more important than maximising our profits".
UK MPs are keen to pose their own questions to Mr Zuckerberg about the Cambridge Analytica scandal - but the Facebook founder has so far declined to make a trip to the UK.
Facebook recently transferred 1.5 billion of its international users from the jurisdiction of its European headquarters, in Ireland, to that of its US headquarters, with some speculating this is to avoid costly legal action resulting from breaches of the EU's General Data Protection Regulation (GDPR).
The sweeping changes to data laws will give consumers much more control over how their personal details are used.
Several of the MEPs challenged Mr Zuckerberg over whether he was fully committed to obeying the new rules.
He responded that he expected Facebook would be fully compliant with the law by the time it comes into force on Friday.
He added that the app had already presented European members with the revised settings required and "a large percentage" of the users had already reviewed them.
More about: Zuckerberg