U.S. mortgage bankers attending an industry conference in Chicago this week received something they did not originally bargain for -- a heavy dose of the consumer anger against the financial system that has boiled into protest rallies across the country.

The Mortgage Bankers Association's annual conference in Chicago this week coincided with a protest march against joblessness and income inequality that drew about 3,000 demonstrators to downtown Chicago on Monday evening.

And while many of the attendees of the MBA event, which ends on Wednesday, say they sympathize with the protesters, others think their industry is being used as a scapegoat for deeper economic woes.

To some mortgage executives, the message is clear: the industry is under siege. I think anyone who thinks we aren't under siege is kidding themselves, said one bank executive as he watched anti-banker protests outside the conference on Monday along with other participants who snapped pictures.

Four years after the U.S. housing bubble burst, mortgage banking executives say their business is under attack from angry homeowners and lawmakers who view the industry as the culprit behind the 2007-2009 financial crisis and subsequent recession.

Some mortgage bankers said the public criticism has begun to intrude on their personal lives.

One former senior industry executive, who declined to be named, said he was confronted at a recent charity event.

A woman asked me how I could sleep at night, and (said) she was glad that Lehman Brothers and Bear Stearns failed, the executive said. When asked how often he is confronted, he said it happens all the time.

Others believe their industry has become a scapegoat, and that they are being punished for loose lending practices that that have long since been rectified.

We're easy to blame, said Hank Cunningham, president of Greensboro, North Carolina-based Cunningham & Co, a mortgage banking company.

TRUST DEFICIT

Throughout the first two days of the conference, attendees said they sympathized with the millions of U.S. borrowers who face foreclosure and, in light of the protests, the MBA issued a statement that amounted to a mea culpa.

We all recognize that our industry faces a trust deficit with policymakers and the public and that people in our industry contributed to the events that led to the financial crisis, it said.

Easy lending terms helped many Americans stretch to buy homes and a plunge in home prices has left more than a quarter of borrowers in homes worth less then their mortgages.

With the nation's unemployment rate stuck above 9 percent, banks are coping with millions of delinquent home loans, and the total number of foreclosed homes is also in the millions.

In August alone, there were 228,098 default notices, scheduled auctions, or bank repossessions on U.S. properties, according to RealtyTrac, a real estate research firm.

The sour housing market was the backdrop to the protest on the conference's doorstep on Monday which drew a few hundred participants pushing for foreclosure relief and calling for the ouster Bank of America Corp CEO Brian Moynihan and JPMorgan Chase & Co CEO Jamie Dimon, among others.

During an afternoon question-and-answer session, two protesters who had managed to get into the conference asked Wells Fargo & Co home loans president Michael Heid how he could sleep at night. Heid said the adversarial questioning reminded him of testifying before Congress.

MBA Chief Executive David Stevens said he understood the protesters' frustrations, and he recalled that after graduating from college he had participated in a protest against a nuclear weapons production plant in Colorado and was arrested.

But he said the industry and its critics must work together to fix the housing market. You're not getting anywhere to rebuild this economy simply by yelling outside, that's the unfortunate reality, he said.

Indeed, in his opening remarks at the conference, Stevens struck a tough tone, saying the trade group was on the war front fighting against new industry rules, including those included in the Dodd-Frank financial reform law put in place last year in an effort to try to prevent future crises.

(Additional reporting by Margaret Chadbourn, editing by Matthew Lewis)

Mortgage