Here is a very telling paragraph from the HUD's new proposed fair housing rule
Despite the existing obligation to AFFH, in too many communities, the Fair Housing Act has not had the impact it intended — housing choices continue to be constrained through housing discrimination, the operation of housing markets, investment choices by holders of capital, the history and geography of regions, and patterns of development and the built environment.
So, they list "discrimination" as a problem, but then look at the other four items they list as problems. These can all be summarized as "the normal operation of free markets, property rights, and individual choice."
Oddly missing from this list of causes is what many historians consider to be the #1 cause of lack of neighborhood diversity and ghetto-ization: The Federal Government and the New Deal. New Deal rules essentially forced the concentration of blacks into just a few neighborhoods. The biggest unmixing of races in New York can be seen between 1930 and 1950. Blacks in Brooklyn went from fairly evenly mixed to concentrated in Bed-Stuy, all directly attributable to New Deal rules. Basically, ever since then, we have just been living with the consequences. Via NPR in an interview with Richard Rothstein
On how the New Deal's Public Works Administration led to the creation of segregated ghettos
Its policy was that public housing could be used only to house people of the same race as the neighborhood in which it was located, but, in fact, most of the public housing that was built in the early years was built in integrated neighborhoods, which they razed and then built segregated public housing in those neighborhoods. So public housing created racial segregation where none existed before. That was one of the chief policies.
On the Federal Housing Administration's overtly racist policies in the 1930s, '40s and '50s
The second policy, which was probably even more effective in segregating metropolitan areas, was the Federal Housing Administration, which financed mass production builders of subdivisions starting in the '30s and then going on to the '40s and '50s in which those mass production builders, places like Levittown [New York] for example, and Nassau County in New York and in every metropolitan area in the country, the Federal Housing Administration gave builders like Levitt concessionary loans through banks because they guaranteed loans at lower interest rates for banks that the developers could use to build these subdivisions on the condition that no homes in those subdivisions be sold to African-Americans.
Postscript: Here is how the Ken Burns New York documentary series explained it, though the source page is no longer available:
Government policies began in the 1930s with the New Deal's Federal Mortgage and Loans Program. The government, along with banks and insurance programs, undertook a policy to lower the value of urban housing in order to create a market for the single-family residences they built outside the city.
The Home Owners' Loan Corporation, a federal government initiative established during the early years of the New Deal went into Brooklyn and mapped the population of all 66 neighborhoods in the Borough, block by block, noting on their maps the location of the residence of every black, Latino, Jewish, Italian, Irish, and Polish family they could find. Then they assigned ratings to each neighborhood based on its ethnic makeup. They distributed the demographic maps to banks and held the banks to a certain standard when loaning money for homes and rental. If the ratings went down, the value of housing property went down.
From the perspective of a white city dweller, nothing that you had done personally had altered the value of your home, and your neighborhood had not changed either. The decline in your property's value came simply because, unless the people who wanted to move to your neighborhood were black, the banks would no longer lend people the money needed to move there. And, because of this government initiative, the more black people moved into your neighborhood, the more the value of your property fell.
The Home Owners' Loan Corporation finished their work in the 1940s. In the 1930s when it started, black Brooklynites were the least physically segregated group in the borough. By 1950 they were the most segregated group; all were concentrated in the Bedford-Stuyvesant neighborhood, which became the largest black ghetto in the United States. After the Home Owners Loan Corp began working with local banks in Brooklyn, it worked with them in Manhattan, the Bronx, and Queens.
The state also got involved in redlining. (Initially, redlining literally meant the physical process of drawing on maps red lines through neighborhoods that were to be refused loans and insurance policies based on income or race. Redlining has come to mean, more generally, refusing to serve a particular neighborhood because of income or race.) State officials created their own map of Brooklyn. They too mapped out the city block by block. But this time they looked for only black and Latino individuals.
This site has some redlining maps, including one of Brooklyn, prepared by the Feds. Remember, this is not some evil Conservative business CABAL, these are Roosevelt Democrats making these maps. This site adds:
While the HOLC was a fairly short-lived New Deal agency, the influence of its security maps lived on in the Federal Housing Authority (FHA) and the GI Bill dispensing Veteran’s Administration (VA). Both of these government organizations, which set the standard that private lenders followed, refused to back bank mortgages that did not adhere to HOLC’s security maps. On the one hand FHA and VA backed loans were an enormous boon to those who qualified for them. Millions of Americans received mortgages that they otherwise would not have qualified for. But FHA-backed mortgages were not available to all. Racial minorities could not get loans for property improvements in their own neighborhoods—seen as credit risks—and were denied mortgages to purchase property in other areas for fear that their presence would extend the red line into a new community. Levittown, the poster-child of the new suburban America, only allowed whites to purchase homes. Thus HOLC policies and private developers increased home ownership and stability for white Americans while simultaneously creating and enforcing racial segregation.
The exclusionary structures of the postwar economy pushed African Americans and other minorities to protest. Over time the federal government attempted to rectify the racial segregation created, or at least facilitated, in part by its own policies. In 1948, the U.S. Supreme Court case Shelley v. Kraemer struck down explicitly racial neighborhood housing covenants, making it illegal to explicitly consider race when selling a house. It would be years, however, until housing acts passed in the 1960s could provide some federal muscle to complement grassroots attempts to ensure equal access.