From next April foreigners will be allowed to take up jobs in sectors such construction, farming and nursing.
Japan has traditionally been wary of immigration but the government says more foreigners are needed because of Japan's ageing population.
Opposition parties say the law risks exposing newcomers to exploitation.
Under the new system, more than 300,000 foreigners could be allowed to work in sectors facing a labour crunch.
The law creates two new visa categories. Workers in the first category will be allowed in for five years if they have a certain level of skill and some proficiency in Japanese.
Workers with a higher level of skills would qualify for the second visa category and would eventually be allowed to apply for residency.
The opposition has argued that an influx of foreign workers would depress wages and lead to exploitation of migrant labour.
The BBC's Rupert Wingfield-Hayes in Tokyo says the current programme of "technical training" for low-skilled foreign workers is open to abuse by unscrupulous employers.
Businesses in Japan have long argued for changes to immigration rules to recruit workers from other countries.
But Prime Minister Shinzo Abe has stressed the proposed law is not an overhaul of immigration policy and Japan will only accept foreigners "who have specific skills and can work immediately to address serious labour shortages, only in sectors that genuinely need them".
Japan's fertility rate dropped below 2.1 childbirths per woman - replacement level - in the 1970s and now stands at around 1.4. The country also has one of the world's longest life expectancies (85.5).