Mr Carter is to give the armed services 30 days to submit plans to make the change. The US military has been easing restrictions over the past three years.
A ban on women serving in combat roles was lifted in 2013 but the military was given until 2016 to make the case for specific posts they thought should remain closed.
`No exceptions`
The chairman of the Joint Chiefs of Staff, Gen Joseph Dunford, had argued the Marine Corps should be allowed to keep women out of certain roles, citing studies showing that mixed-gender units are not as effective as all-male ones.
But Mr Carter rejected the argument. "There will be no exceptions," he said.
"As long as they qualify and meet the standards, women will now be able to contribute to our mission in ways they could not before.
"They`ll be allowed to drive tanks, fire mortars, lead infantry soldiers into combat."
Nancy Duff Campbell of the National Women`s Law Center said it was a "thrilling day for women serving in the military, and for women across the country".
More about: