Several hundred people rallied outside the parliament building Monday calling for Shokin`s resignation.
"The Maidan (revolution) is not over," anti-corruption activist Vitaliy Shabunin told the crowd.
Blocking anti-corruption probes
Shokin, a loyalist of President Petro Poroshenko, has been widely accused of failing to investigate the alleged theft of state funds by the deposed Russian-backed leadership.
There are also allegations that he deliberately stalled probes into prosecutors who were dismissed after being discovered hoarding cash and diamonds in their homes.
The move adds to Prime Minister Arseniy Yatsenyuk`s coalition weakness after it lost its majority in February. Poroshenko`s allies reportedly want to oust Yatsenyuk, but have struggled to find support in parliament, where Yatsenyuk`s party is the second largest.
Yatsenyuk`s party has reportedly agreed with Poroshenko`s faction and the Fatherland party - all former coalition partners - to form a new coalition, but this may now be on less firm ground.
Ukrainian leaders have been in talks on reshuffling the government for several weeks after the pro-Western coalition splintered.
Shokin fired anti-corruption campaigner
Shokin tendered his resignation a month ago but was advised by Poroshenko to go on vacation.
Several senior officials have resigned but stayed in their posts in Ukraine in recent months. Five ministers have tendered their resignations from Prime Minister Arseny Yatseniuk`s government, but four have returned.
An hour before his resignation was accepted by parliament, prosecutor and anti-corruption campaigner Davit Sakvarelidze was fired by Shokin in a move that failed to firewall the senior figure.
Sakvarelidze was fired for "grave violation of prosecutorial ethics," according to a statement posted on the website of the Prosecutor General`s Office.
The anti-corruption campaigner is one of several foreign-born experts brought in to assist Ukraine with its Western-backed reform drive and had been an outspoken critic of the prosecutor`s office.
The US has been calling for fundamental reform of the general prosecutor`s office. Anti-graft campaigners say it plays a key role in protecting vested interests and allowing corrupt practices.
US Assistant Secretary of State Victoria Nuland said last week that Ukraine must "appoint and confirm a new, clean Prosecutor General" to tackle graft, including in the prosecutor`s office itself.
Ukraine`s efforts to tackle endemic graft helped derail disbursement of $1.7 billion (1.58 billion euros) in new aid from the International Monetary Fund. IMF chief Christine Lagarde warned recently that the Fund may stop aid altogether if matters do not improve.