Gharyan was the main supply base for an offensive on the capital, Tripoli, by fighters led by Gen Khalifa Haftar.
His Libyan National Army (LNA) has not yet confirmed the loss, and details are still emerging.
Libya has been torn by violence and division since long-time ruler Muammar Gaddafi was deposed and killed in 2011.
Dozens have died since the LNA began its campaign earlier this year against the UN-backed Government of National Accord (GNA), which is led by Prime Minister Fayez al-Serra.
The LNA has held Gharyan, south of Tripoli, since 2 April when it launched a rapid advance against the capital from the south and east.
But progress stalled along the front line and an LNA spokesman had earlier accused "sleeper cells" of allowing GNA forces to enter Gharyan.
Witnesses told Reuters news agency that forces loyal to the GNA launched a surprise attack on the town on Wednesday, backed by air strikes.
Images have also circulated on social media of GNA forces in Gharyan, and of prisoners alleged to be pro-Haftar fighters.
A spokesperson for the forces told AFP news agency that its capture was a "significant victory" that should lead to the "collapse" of Gen Haftar's campaign.
Gen Haftar has his powerbase in the east of the country where he is allied to a rival government.
He and his supporters say they are trying to restore security and fight "terrorism".
But critics have accused him of deepening unrest by trying to seize power through force.