The Japanese government, Tokyo metropolitan government and Tokyo 2020 organising committee are leaning towards holding the massive event in front of a domestic audience only, the Yomiuri Shimbun and other outlets said.
Officials fear that an influx of visitors from abroad will endanger the Japanese public.
"As the spread of coronavirus continues in every country, they have decided that allowing visitors from overseas on a large scale will only make people anxious," the Yomiuri report said.
The three parties met with International Olympic and Paralympic Committee heads on Wednesday and agreed to take a decision on spectators by the end of the month, ideally before the nationwide torch relay begins on March 25.
But comments made by organisers before and after the talks hinted that overseas fans will be locked out, with games president Seiko Hashimoto saying public safety is the "top priority".
IOC chief Thomas Bach said organisers will "focus on the essentials of the Games", while Japan's Olympic Minister Tamayo Marukawa said "a cautious decision is necessary".
Reports said Japanese organisers will soon make their position official and convey it to IOC and IPC chiefs.
Bach said last week that a decision on overseas fans would be taken in late April or early May, but Hashimoto said Wednesday that fans, hotels and travel operators needed to know sooner.
Around 900,000 tickets have reportedly already been sold outside Japan.
Organisers also said Wednesday that they will take a decision on the number of spectators allowed in venues in April.
Hashimoto said "scientific insight" would be used to determine venue capacities, and that the ruling would be in line with government policies on attendance.
Tokyo is currently under a Covid-19 state of emergency that limits capacity at sporting events to 5,000.
The state of emergency is due to expire on Sunday, but Prime Minister Yoshihide Suga has said an extension of around two weeks is necessary amid concern over the slowing decline of infections in the capital.
More about: Tokyo-Olympics