Scientists say the discovery could push back the date for the oldest fungi by one to two billion years.
The find suggests that fungi arose not on land but in the deep sea. If not a fungus, the organism could be from an extinct branch of life that has not been described before.
Prof Stefan Bengtson of the Swedish Museum of Natural History led the research team.
He said, in the past, scientists may have been looking in the wrong place for the oldest fossil fungi - on land or in shallow seas rather than in the deep sea.
"The deep biosphere (where the fossils were found) represents a significant portion of the Earth, but we know very little about its biology and even less about its evolutionary history," Prof Bengtson told BBC News.
The fossils are almost indistinguishable from those found in similar environments on land, although they are much older.
They are made up of jumbles of tangled threads some hundredths of a millimetre thick.
Scientists have already discovered such structures - known as mycelia - in similar rocks of a much younger age and identified them as fungi.
There is a "clear possibility" that they are the world's oldest fossil fungi - twice as old as generally accepted in the fossil record, Prof Bengtson said.
If they are not fungi, they are probably an extinct branch of eukaryotes "or even giant prokaryotes", he added.
Eukaryotes are any cell or organism with a clearly defined nucleus, while prokaryotes have no nucleus.
The fossils were collected by drilling into ancient rock formed from bubbly lava that was once beneath the sea bed in what is now the Northern Cape Province of South Africa.
Known as the Ongeluk Formation, the location was underwater at the time the organisms would have been alive.
The fungi may have colonised cavities of rock deep beneath the sea floor.
Co-researcher, Magnus Ivarsson, said it would have been an extreme environment far away from sunlight.
"Fungi in this environment most probably lived in symbiosis with microbes utilising chemically stored energy for their metabolism," he said. "They may not even have needed free oxygen."
The research is published in Nature Ecology & Evolution.
Writing in the journal, Nicola McLoughlin of Rhodes University in Grahamstown, South Africa, who is not connected with the study, said the fossil challenges current thinking on when and where eukaryotes evolved.
It "raises the question of whether we have been looking in the wrong place for the earliest eukaryotes and fossil fungi in particular", she said.
More about: #science
