Guinness World Records gave the certificate to Chitetsu Watanabe on February 12.
The organisation and the funeral home handling his services confirmed on Tuesday he had died on Sunday. No cause was given.
He had not been able to eat recently and developed a fever and difficulty breathing a couple of days before his death, Japan's nationally circulated newspaper Mainichi reported, citing family sources.
Mr Watanabe is survived by his five children, 12 grandchildren, 16 great-grandchildren and one great-great-grandchild, Mainichi said. His family did not immediately answer calls.
Mr Watanabe was born in 1907 and worked in Taiwan for 18 years.
After returning to Niigata, northern Japan, he worked for the prefectural government until his retirement.
He grew fruit and vegetables on the family farm and loved cream puffs and bonsai, the Japanese traditional art of raising small sculpted trees.
He used to say the secret to longevity was to keep smiling. The oldest living person is also Japanese, Kane Tanaka, a 117-year-old woman.
More about: Japan