Pro-Russia separatists have captured official buildings in dozens of towns in eastern Ukraine in recent weeks.
The Kiev government has sent troops to restore its authority.
Ukrainian Interior Minister Arsen Avakov said government troops had taken over Mariupol city hall following a joint operation by ministry troops and the army early on Wednesday.
Mr Avakov told the BBC that several people had been detained, including the defence minister of the separatist "Donetsk People`s Republic", Igor Kakidzyanov.
The BBC`s Richard Galpin in eastern Ukraine says this is potentially a significant step as Mariupol is an important city in an industrial heartland.
However Ukraine`s Channel 5 TV later reported that some rebels had regrouped and marched on the building.
A local website said Ukrainian troops eventually left and Russian and rebels hoisted Donetsk People`s Republic flags outside the building
More about: