They said Australian staff were seen running for a boat moored nearby.
Iranian refugee Amir Taghinia was on a bus being driven into the centre when he heard shots.
"It's crazy, many people are in shock," he said.
"The Navy is shooting people.
"The staff were running out of the compound like crazy, I think they were getting on a ship."
Iranian asylum seeker Benham Satah said Australian security staff were sheltering inside the complex with the detainees.
"Security staff were inside, they were hiding with us," he said.
Mr Satah said the attackers were using high-powered weapons and their fire left bullet holes in the accommodation buildings.
"They were repeatedly shooting and a couple of rooms have been shot," he said.
Asylum seekers inside the centre said they did not know if anyone had been shot.
More about: #ManusIsland
