Hakubun Shimomura, the minister in charge of education, sports and science, told Parliament Wednesday the stadium designed by award-winning British-Iraqi architect Zaha Hadid would cost 300 billion yen ($3 billion), and that was "too massive a budget."
The 80,000-seat, futuristic-looking stadium has been billed as costing 130 billion yen ($1.3 billion). The minister's updated estimate includes surrounding construction and infrastructure costs.
"We need to rethink this to scale it down," he said in response to a question from a ruling party lawmaker. "Urban planning must meet people's needs."
The plans for the stadium were approved earlier this year by the city and central governments. Shimomura's remarks signal a policy change.
He did not give specifics on how construction will be trimmed, but he stressed that the design concept will be kept.
He also said the new stadium will still have all the basic features needed to host the Olympics. It is replacing the smaller 54,000-seat main stadium that was used for the opening and closing ceremonies of the 1964 Tokyo Olympics.
Japanese architect Fumihiko Maki, a recipient of the prestigious Pritzker Architecture Prize, recently criticized the new stadium's size and urged that it be reworked to "a more sustainable stadium.
About 100 experts, including other architects, support his view and question whether the new stadium is environmentally responsible and practical.
The site sits in the middle of a downtown Tokyo park within walking distance of shopping malls, high-rise buildings, a Shinto shrine and a famous venue designed by Kenzo Tange for the 1964 Olympics.
Zaha Hadid Architects office has said the venue is flexible and can be used for events beyond the Olympics, such as concerts. But it has expressed willingness to talk about design changes.
Construction is scheduled to begin next year.
Follow Yuri Kageyama on Twitter at twitter.com/yurikageyama