It was found by workers building a car park at the site where a four-decade-long decommissioning process is under way.
Tens of thousands of residents had to evacuate the area after a reactor meltdown in 2011 following an earthquake and tsunami.
The incident at the Tokyo Electric Power Co (Tepco) site was the world's most serious nuclear accident since Chernobyl in 1986.
No-one died directly in the meltdown but three former Tepco executives are facing trial on charges of negligence because of deaths related to the area's evacuation.
Tepco said construction work was immediately suspended after the object was found and a temporary exclusion zone put in place while bomb disposal experts were deployed.
It is not uncommon for unexploded WW2 devices to be found in Japan over 70 years on from the end of the war.
The Fukushima area was previously home to a Japanese military base.
More about: #Fukushima #nuclear #WW2