Taro Aso, 78, denied that the elderly were the cause of the country's declining population and increasing social security costs during a speech in Fukuoka prefecture in southern Japan on Sunday.
"There are lots of strange people who say the elderly are to blame, but that is wrong," said Aso, who also holds the role of finance minister. "The problem is with those who didn't give birth."
After being grilled by an opposition lawmaker during a budget committee session, Aso said Monday that he would retract the remark "if it caused misunderstanding."
"It gave a false impression without conveying the meaning of my original remark," Aso explained, without clarifying what he had actually wanted to say.
Japan is a "super-aged" nation, meaning more than 20% of its population is older than 65. It has been on a steady demographic decline since the 1970s.
In 2017, fewer than 950,000 babies were born while the number of deaths rose to a postwar high of 1.3 million, according to the Ministry of Health and Labor.
Tokyo, a city of more than 9 million people, has the lowest birth rate of all the 47 prefectures in Japan, at 1.17.
However, mothers who do have children are struggling to find adequate enough care for their children to continue with their careers.
Tokyo has the largest number of children on waiting lists for day care facilities. More than 5,400 children are seeking an elusive spot at day care centers in the city, nearly 30% of the national total.
Unconscious gender bias
Since the 1990s, Japan has introduced policies to boost its birth rate, such as enhancing child care services and improving housing and public facilities for families with children.
However, structural issues still prevent both working men and women from balancing their careers with family life.
"The government should be doing more to help households balance work and family care responsibilities rather than condemning women," Jeff Kingston, a Japanese studies professor at Temple University.
"With dinosaurs like (Aso) in power, enacting effective policies to mitigate Japan's demographic challenges is more difficult."
In just three decades, the percentage of Japanese women age 30 to 34 who return to the labor force after becoming mothers has risen from 50% to 75%, according to recent reports.
But many of those who go back to work have to accept lower wages or get stuck on the career ladder, according to a report published in 2017 by the Organization for Economic Co-operation and Development.
Japan is ranked at 110 out of 149 countries in the World Economic Forum's (WEF) latest global gender gap index measuring gender equality.
Women only make up 16% of ministerial positions in the government, according to a 2017 report from Inter-Parliamentary Union, a global organization of national parliaments.
More about: Japan