All barracks will be searched for material linked to the Wehrmacht, the army which served Adolf Hitler.
The move follows a growing scandal over far-right extremism within the army, with an officer accused of plotting an attack disguised as a Syrian refugee.
The army lieutenant, who had expressed far-right views, was arrested in late April.
Prosecutors in Frankfurt said the 28-year-old suspect had a "xenophobic background".
Germany's Defence Minister Ursula von der Leyen cancelled a planned trip to the US and travelled to his garrison in Illkirch, northeastern France.
Officials found Wehrmacht memorabilia openly displayed there in the common room, despite a ban on Nazi symbols.
A second barracks, Fürstenberg in Donaueschingen, southwest Germany, was then sucked into the scandal on Saturday after a display cabinet was found containing Nazi-era helmets.
Spiegel online reported that pictures of Wehrmacht soldiers were found on a wall, along with Nazi pistols, more helmets and military decorations.
A Defence Ministry spokesman told Reuters the items found in Donaueschingen did not include Nazi objects punishable under German law, such as swastikas.
However, Ms Von der Leyen said last Wednesday that she would not tolerate the veneration of the Wehrmacht in today's army.
She said the latest scandals were no longer isolated cases, adding that "a misunderstood esprit de corps" had led senior officers to "look the other way".
The minister later apologised for the tone of her criticism, after political opponents accused her of smearing the whole army.
More about: #Nazi
