Forget stocks. Don’t bet on gold. After four years of plunging home prices, the most attractive asset class in America is housing.

A home under construction in Austin. The number of new homes in the pipeline nationwide is quite low. 

From his wide-rimmed cowboy hat to his roper boots, Mike Castleman fits moviedom’s image of the lanky Texas rancher. On a recent March evening, Castleman is feeding cattle biscuits to his two pet longhorn steers, Big Buddy and Little Buddy, on his 460-acre Bar Ten Creek Ranch in Dripping Springs, a hamlet outside Austin in the Texas Hill Country. The spread is a medley of meandering streams, craggy cliffs, and centuries-old oaks. But even in this pastoral setting, his mind keeps returning to a subject he knows as well as any expert around: the housing market. “I’m a dirt-road economist who sees what’s happening on the ground, and in 35 years I’ve never seen a shortage of new construction like the one I’m seeing today,” declares Castleman, 70, now offering a biscuit to his miniature donkey Thumper. “The talking heads who are down on real estate will hate to hear this, but America needs to build a lot more houses. And in most markets the price of new homes is fixin’ to rise, not fall.”

Castleman is in a unique position to know. As the founder and CEO of a company called Metrostudy, he’s spent more than three decades tracking real-time data on the country’s inventory of new homes. Each quarter he dispatches 500 inspectors to literally drive through 45,000 subdivisions from Baltimore to Sacramento. The inspectors examine 5 million finished lots, one at a time, and record whether they contain a house that’s under construction, one that’s finished and for sale, or a home that’s sold. Metrostudy covers 19 states, or around 65% of the U.S. housing market, including all the ones hardest hit by the crash: Florida, California, Arizona, and Nevada. The company’s client list includes virtually every major homebuilder and bank — from Pulte (PHM) and KB Home (KBH) to Bank of America (BAC) and Wells Fargo (WFC).

The key figures that Metrostudy collects, and that those clients prize, are the number of homes that are vacant and for sale in each city, and the number of months it takes to sell all of them. Together those figures measure inventory — the key metric in determining whether a market has a surplus or a shortage of new housing.

Today Castleman is witnessing an extraordinary reversal of the new-home glut that helped sink prices just a few years ago. In the 41 cities Metrostudy covers, a total of 78,000 houses are now either vacant and for sale, or under construction. That’s less than one-fourth of the 343,000 units in those two categories at the peak of the frenzy in mid-2006, and well below the level of a decade ago. “If we had anything like normal levels of buying, those houses would sell in 2½ months,” says Castleman. “We’d see an incredible shortage. And that’s where we’re heading.”

If all the noise you’re hearing about housing has you totally confused, join the crowd. One day you’ll read that owning a home has never been more affordable. The next day you’ll see news that housing starts have plunged to nearly their lowest level in half a century, as headlines announced in March. After four years of falling prices and surging foreclosures, it’s hard to know what to think. Even Robert Shiller and Karl Case can’t agree. The two economists, who together created the widely followed S&P/Case-Shiller Home Price indices, are right now offering sharply contrasting views of housing’s future. Shiller recently warned that the chances were high for a further double-digit drop in U.S. home prices. But in an interview with Fortune, Case took a far brighter view: “The lack of new home building is a huge help that a lot of people are ignoring,” says Case. “People think I’m crazy to be optimistic, but housing is looking like the little engine that could.”

To see where real estate is truly headed, it’s critical to keep your eye firmly on the fundamentals that, over time, always determine the course of prices and construction. During the last decade’s historic run-up in prices, Fortune repeatedly warned that things were moving too fast. In a cover story titled “Is the Housing Boom Over?,” this writer’s analysis found that the basic forces that govern the market — the cost of owning vs. renting and the level of new construction — were in bubble territory. Eventually reality set in, and prices plummeted. Our current view focuses on those same fundamentals — only now they’re pointing in the opposite direction.

So let’s state it simply and forcibly: Housing is back.

Two basic factors are laying the foundation for dramatic recovery in residential real estate. The first is the historic drop in new construction that so amazes Castleman. The second is a steep decline in prices, on the order of 30% nationwide since 2006, and as much as 55% in the hardest-hit markets. The story of this downturn has been an astonishing flight from the traditional American approach of buying new houses to an embrace of renting. But the new affordability will gradually lure Americans back to buying homes. And the return of the homeowner will start raising prices in many markets this year.

Drumming up sales 

Of course, home prices are low and home construction is weak for a reason: incredibly low demand. For our scenario to play out, America will need a decent economy, with job creation and consumer confidence continuing to claw their way back to normal.

One big fear is that today’s tight credit standards will chill the market. But we’re really returning to the standards that prevailed before the craze, and those requirements didn’t stop prices and homebuilding from rising in a good economy. “The credit standards are now at about historical levels, excluding the bubble period,” says Mark Zandi, chief economist for Moody’s Analytics. “We saw prices rising with fundamentals in those periods, and it will happen again.”

To see why, let’s examine the remarkable shift in home affordability. A new study by Deutsche Bank measures affordability in two ways: first, the share of income Americans are paying to own a home. And second, the cost of owning vs. renting. On the first metric, the analysis finds that homeowners now pay just 9.8% of their income in after-tax mortgage, tax, and insurance payments. That’s down from 17.2% at the bubble’s peak in 2007, and by far the lowest number in the Deutsche Bank database, going back to 1999. The second measure, the cost of owning compared with renting, should also inspire potential buyers. In 28 out of 54 major markets, it’s now cheaper to pay a mortgage and other major costs than to rent the same house. What’s most compelling is that in all of the distressed markets, owning now wins by a wide margin — a stunning reversal from four years ago. It now costs 34% less than renting in Atlanta. In Miami the average rent is now $1,031 a month, vs. the $856 it costs to carry a ranch house or stucco cottage as an owner. (For more, see The top 10 cities for home buyers)

Not all markets will bounce back equally, of course. Housing resembles the weather: The exact conditions are different in every city. But in general the big U.S. markets fall into two different climate zones right now. We’ll call them the “nondistressed markets” and the “foreclosure markets.” A more detailed look shows why the forecast for both is favorable.

Nondistressed markets: Ready for launch

No cities went untouched by the collapse in prices over the past few years. But markets such as Northern Virginia, Indianapolis, Minneapolis, San Diego, the San Francisco suburbs, and virtually all of Texas held up reasonably well. In those areas prices spiked far less than in bubble cities — the foreclosure markets we’ll get to shortly — chiefly because they didn’t get nearly as many speculators who thought they could flip the homes or rent them to snowbirds.

The nondistressed markets will be able to get prices rising and construction growing far faster than the harder-hit areas for a simple reason: Although some of these markets are still suffering from foreclosures, they don’t need to work through the big overhang haunting a Las Vegas or a Phoenix. The number of new homes for sale or in the pipeline is extraordinarily low in nondistressed markets. San Diego is typical. It has just 921 freestanding homes for sale or under construction, compared with 4,425 in late 2005. The challenge for these cities is to generate enough demand to reduce inventories of existing, or resale, homes. In the entire country the resale supply stands at 3.5 million houses and condos. That’s a fairly high number, since it would take more than eight months to sell those properties; seven months or below is the threshold for a strong market.

But in the nondistressed cities, the existing home inventory is lower, closer to seven months on average. So a modest increase in demand will translate into strong gains in both prices and new construction. That should happen quickly, because most of those markets — including Silicon Valley, Northern Virginia, and Texas — are now showing good job growth.

Zandi of Moody’s Analytics expects that prices will rise three to four points faster than inflation for the next few years in virtually all of the nondistressed markets. His view is that prices will increase in line with rents, which are now growing briskly because apartments are in short supply. Those higher rents will encourage buyers to cross the street from an apartment to a home of their own.

In Northern Virginia, Chris Bratz, an engineer, and his wife, Amy DiElsi, a publicist, are planning to leave their rental apartment and become homeowners for the first time. The main reason? Buying has simply become a far better deal than renting. “The market got completely inflated, then it crashed, so prices are coming back to where they should be,” says Chris. As the couple have watched prices fall, they have also watched the rent on their apartment spiral upward, reaching $2,700 a month. They calculate that they should be able to purchase a townhouse for between $400,000 and $500,000 and pay less per month for a mortgage.

The nondistressed markets will also lead the way in construction. Zandi predicts that for the nation as a whole, single-family housing “starts” — measured when a builder pours a foundation for a new home — will rise from 470,000 in 2010 to as much as 700,000 this year. A large portion of that activity will happen in nondistressed markets where a tightening supply of resale houses will start making new homes look like a good deal. “Our main competition is from resales,” says Jeff Mezger, CEO of KB Home. “The prices of those homes have stayed so low, because of low demand, that it’s hampered the ability of builders to sell new houses.”

But many would-be buyers simply prefer a brand-new house. Eventually they’ll move from renters to buyers, and the trend will accelerate now that prices are no longer dropping. In Minneapolis, Yuan Qu and her husband, Xiang Chen, a researcher at the University of Minnesota, just moved from a two-bedroom rental to a new light-blue four-bedroom ranch with a chocolate-colored roof on a spacious corner lot. They paid $400,000, a bargain price compared with a few years ago. The couple, both in their early thirties, moved to Minnesota from China six years ago. “We wanted to buy a house, and we’ve been waiting and waiting and waiting,” says Qu. “The prices went down for so long, we finally thought they couldn’t keep falling.” For Qu the only choice was new construction. “We’re not very handy people,” she admits.

Foreclosure markets: The outlook is brightening

A home off the market in Mesa, Ariz. 

The true disaster areas for housing since the bubble burst have been Sunbelt cities such as Las Vegas, Phoenix, and Miami — places that boasted great job and population growth in the mid-2000s, only to suffer a housing crash that swamped them with empty homes and condos and crushed their economies. But people always want to live in those sunny locales, and their job markets are starting to recover, albeit slowly. In foreclosure markets the inventory problem is far greater because it includes not just traditional resale homes but millions of distressed properties. Fortunately those houses are now such a screaming deal that investors, including lots of mom-and-pop buyers, are purchasing them at a rapid pace. To be sure, some foreclosure markets won’t rebound for years because they’re both vastly overbuilt and far from big job centers; a prime example is California’s Inland Empire, a real estate disaster zone 80 miles east of Los Angeles.

But the outlook is brightening for Phoenix, Las Vegas, Miami, and parts of Northern California. A big positive is the tiny supply of new homes entering the market. Phoenix, for example, has a total of just 8,100 new homes that are either for sale or under construction, down from 53,000 in mid-2006. The big test in these cities is absorbing the steady stream of distressed properties. The foreclosures put downward pressure on the market far out of proportion to their numbers because of markdown pricing. “We had levels of inventory even higher than this in 1990 and 1991,” says MIT economist William Wheaton. “But they were traditional listings, not foreclosures, so they didn’t create the big discounts you get with foreclosures.”

Wheaton reckons that we’ll see a flow of around 1 million foreclosures a year, at a fairly even pace, from now through 2013. That figure is frequently cited as evidence that the market is doomed for years in most foreclosure markets. Not so. The reason is that the vast bulk of those units, probably over 600,000, according to Gleb Nechayev, an economist with real estate firm CB Richard Ellis (CBG), are being converted to rentals either by investors or their current owners. Those properties are finding plenty of renters, since the rental market is still extremely strong across the country. Remember, the millions who lost their homes to foreclosure still need somewhere to live.

A typical investor is Alex Barbalat, a Russian immigrant who’s purchased seven homes east of San Francisco in the towns of Bay Point, Antioch, and Pittsburg. His average purchase price is around $100,000 for homes that once sold for between $300,000 and $500,000. But he has no trouble finding renters, since his tenants can commute to jobs in San Francisco on the BART transit system. Barbalat is pocketing rental yields on the prices he paid of around 12%, and he’s in no hurry to sell. “I’m holding them until prices drastically rise,” he says.

Investment funds are also entering the game. Dotan Y. Melech looks for bargains in Las Vegas for UnitedAMS, a firm he co-founded that manages apartments and other real estate investments. The firm has raised more than $20 million from outside investors to purchase distressed properties. So far, Melech has bought around 300 houses and plans to purchase another 200 this year. He has no trouble renting the houses he buys, since, he estimates, occupancy rates in Las Vegas are touching 95%. The “cap rate,” or return on investment after all expenses, is between 8% and 10% — twice the rate on 10-year Treasuries. Melech rents to people who lost their homes but are reliable renters. “A lot of people can’t be buyers because their credit got hurt,” he says.

Even with investors jumping in, buying activity in foreclosure markets hasn’t yet increased enough to bring inventories down. It will soon. Zandi thinks prices will fall a couple of percentage points lower in the distressed markets in the short run. “But that will be overshooting,” he says. “It’s like an elastic band. If prices do drop this year, they will need to bounce back because they’ll be far too low compared with rents and replacement cost.” Renters will come off the sidelines to purchase homes in the years ahead, precisely the opposite trend of the past few years.

Consider the example of Michael Dynda, a retired Air Force avionics technician who now works for a government contractor in Las Vegas. Dynda, 49, is a first-time buyer who put off purchasing for years, in part because prices were falling so rapidly in Las Vegas, with no bottom in sight. But last year the combination of bargain prices and low mortgage rates became too good to resist. He ended up purchasing a 2,300-square-foot stucco home for $240,000, or about half what it would have fetched in 2007. Dynda got a 4.38% home loan, and pays the same amount on his mortgage as on the rent on the house he left to become a homeowner. “The timing was about as good as it could get,” says Dynda.

Mike Castleman’s company tracks the inventory of new homes in 19 states across the country. He sees supply getting tight. “Home prices are fixin’ to rise,” he says. 

Back on the ranch, Mike Castleman is lounging in his creek-front mansion, built from “a hundred tons of fine central Texas limestone.” As he shows off his collection of custom-made guitars, including one crafted to resemble the skin of a rattlesnake, the homespun housing guru once again returns to his favorite topic.

Castleman claims that this recovery will look like all the others: It will bring a severe shortage of housing. He invokes the livestock business to explain. “It takes three years between the time a bull mates with a cow and when you get a calf ready for market,” he says. “That’s how it is in housing too. We’ll get a big surge in demand and the drywall companies will take a long time to ramp up, and it will take years to get new lots approved. Buyers will show up looking for a house in a subdivision, and all the houses will be sold. The builders will tell them it will take six months to deliver a house.” But those folks, says Castleman, will be set on buying a place. “And they’ll want it so bad they’ll bid the prices up!” In other words: Beat the crowd.

It’s a Great Time to Buy a House
Mike Castleman, the Texan with the best realtime view of housing in the U.S., tells editor-atlarge Shawn Tully that the naysayers are about to get a big surprise: rising prices for new homes.

–Reporter associates: Anne VanderMey and Christopher Tkaczyk

More from Fortune:

Foreclosure vote could rock the banks

Homeownership should not be part of the American Dream

Obama has a mortgage plan (or three) worth reading