Sir Michael Lyons, chair of the Lyons Housing Review Commission, was speaking at the Town and Country Planning Association Annual Conference in London this morning.
His report, published last month, called for development corporations and larger-than-local planning structures to build 200,000 homes a year.
Speaking this morning, Lyons referred to a report published this week by the Campaign to Protect Rural England which, using official figures, found that 1 million homes could be built on previously-used brownfield land.
Lyons praised the report and said there should be more focus on developing brownfield land, but added that 1 million homes was "not enough".
If current building trends continue, there will be a 2 million home deficit by 2020, he said.
Lyons also criticised the coalition government for not doing enough to build new garden cities and only selecting Ebbslfeet in Kent as a location.
"It's simply not enough," he said. "We need to go back to the ambition of the 1940s when new towns were providing 20,000 homes a year."
Elsewhere, Lyons said the key to getting more housing built is persuading communities of its benefits.
He said this was the reason why local politicians were often opposed to development and cited neighbourhood planning as an example of how communities can be persuaded.
Lyons said it was important that a balance was struck between the "national imperative" of building more homes and local decision-making on where they should be built.
He said national governments should take a stronger lead on housebuilding, but said: "There simply isn't a way that central government can determine where development can take place. All that would do is strengthen local resistance."
Speaking earlier, TCPA chief executive Kate Henderson said planning was "in a tough place" with no national or regional planning framework and legal challenges against local plans at their highest level.
Henderson singled out the new permitted development rights allowing commercial buildings, including offices, to be converted into homes without needing planning permission.
She said such developments make no contribution to local education or transport needs, do not comply with housing or space standards, and provide any affordable housing.
"This has a profound impact on how the public view planning," said Henderson.
"We have to hold the government to account on this because that is not planning."