"The detainees were able to force open the doors" in order to leave the Ain Zara prison, the local police said.
They added that guards, fearing for their lives, were unable to prevent the breakout following riots at the jail.
Clashes between militias in the city have led Libya's UN-backed government to declare a state of emergency.
The incident on Sunday took place during armed violence between rival factions around the facility, which houses only male inmates.
Many of the prisoners held at the Ain Zara prison in south-east Tripoli were reportedly supporters of the late Libyan leader Muammar Gaddafi, and had been found guilty of killings during the uprising against his government in 2011.
Separately on Sunday, two people were killed and several others were injured when rockets hit a camp in Libya's capital housing hundreds of displaced people, according to emergency services and witnesses.
Libya's Ministry of Health reports that some 47 people, including civilians, have died in rival militia clashes in Tripoli over the past week, with dozens more injured.
A UN-backed government is nominally in power in the capital, but militias occupy much of the rest of the country.
Why is there violence?
The violence broke out last week when militias from a city to the south of Tripoli attacked southern areas, leading to fighting with local militias that support the internationally recognised government, the Government of National Accord (GNA).
The GNA has described the clashes as "an attempt to derail peaceful political transition" in the country, adding that it "could not remain silent over the attacks on Tripoli and its suburbs, which is a violation of security in the capital and of citizens' safety".
Human Rights Watch has also condemned the violence, adding that at least 18 of the dead were reportedly civilians, among them four children.
Hundreds of migrants trapped by the fighting have been moved to other detention centres, while the city's airport was closed for two days on Friday.
Libya has faced continuing chaos since Nato-backed militia forces, some of them rivals, overthrew long-serving ruler Colonel Gaddafi in October 2011.
What has the international community said?
UN Secretary General Antonio Guterres has said the "indiscriminate use of force is a violation of international humanitarian and human rights law" and has urged "all parties to grant humanitarian relief for those in need".
On Saturday, the US, UK, France and Italy called for an immediate end to deadly violence in Libya's capital.
A joint statement said attempts "to weaken the legitimate Libyan authorities and hinder the ongoing political process are not acceptable".
"We are calling on the armed groups to immediately stop all military action and warn those who seek to undermine stability, in Tripoli or elsewhere in Libya, that they will be made accountable for it," the joint statement read.
However successive attempts at a truce have so far failed to stop the fierce fighting between several militias.
BBC
More about: Libya
