At least two buses have arrived in rebel-held territory to the north, carrying a number of armed fighters.
It marks the end of any rebel presence in the heart of the major city once dubbed the "capital of the revolution" against President Bashar al-Assad.
Earlier this year, about 1,400 people were evacuated from the Old City under an operation overseen by the UN and Red Crescent.
However, a group of fighters and civilians, including many injured, stayed behind despite the increasingly tough conditions.
The BBC`s Paul Wood in Beirut says the rebel fighters and their families are sad and bitter as they say goodbye to a place they swore they would never leave.
They buckled finally, our correspondent adds, after two years of siege - the government`s forces following a tactic of what some Syrian army officers called "surrender or starve".
The siege of the Old City was tightened in recent months with intense shelling and air strikes.
"The rest of the world failed us," one activist told the BBC by Skype as he prepared for the evacuation.
A video posted online by activists showed a group of fighters, some with their faces covered, walking in a line towards green buses and a white UN 4x4 vehicle, watched by government police officers.
Later, rebel negotiator Abdul Hareth al-Khalidi told the AFP news agency that three buses had left, "carrying 120 people in total, a mixture of wounded and non-wounded civilians and fighters".
More about:
