London is entering a confident new era of public housing, says Paul Karakusevic, thanks to housing projects with pioneering new financial models.

The great cause of the early 21st century global city is public housing. While new museums, galleries and beautiful one-off houses grab attention the world over, it is affordable housing that is the fundamental enabler to the future health and vitality of the city.

After nearly four decades of neoliberal housing policy, London is waking up to the fact that its future prosperity depends on tackling a severe housing crisis and worsening inequality. To do that, it is reinventing public housing and embracing new architectural approaches, and its local democracy is again playing a central role in delivery.

London gave the world public housing in the form of the council estate

At the end of the 19th century, London gave birth to big-city municipalism. It gave the world public housing in the form of the council estate, and with it some of the finest, most innovative dwellings the world had seen.

But in the 1970s, as the British economy faltered, the great urban project of housing the people started to crumble, inflation soared and funding became unworkable. Then, in the 1980s, a UK led by Margaret Thatcher's Conservative government followed the example of the USA and shifted housing supply firmly to the right of politics. In doing so, the UK walked away from public housing, indeed the whole idea of the public city, and it turned almost exclusively to the forces of the market to determine its urban future.

The result for London at the start of the 21st century, like global urban contemporaries such as New York or San Francisco, is a mixed and polarised urban landscape.

The city's economy has been completely transformed. It is gifted with one of the most diverse and resilient creative economies in the world, at the hub of global capital flows, which is manifested in an ever-shifting skyline of commercial towers and one of the fastest growing populations of any capital in Europe.

However the trade-off for Londoners, along with anyone within a 45-minute train rise of the city, is the most expensive housing in Europe. For many, around half of all monthly earnings goes on rent or mortgage repayments.

With a chronic shortage of truly affordable housing, high prices and rising living costs are exacerbating housing waiting lists, pushing people into temporary accommodation and increasing homelessness. Local authorities in the city are now burdened with growing social-care bills and many people are being housed in dreadful and inadequate temporary housing in the private rental sector, which is one of the most unregulated in Europe.

London's boroughs delivered hundreds of thousands of homes in the postwar years

These dire consequences are now affecting the city's ability to function. Key workers on low incomes are forced to travel for hours a day for work, while low to middle income families are leaving the city as decent accommodation becomes harder to find. The situation and people's frustration is shifting British urban politics.

Responding to widespread market failure to provide adequate housing at required numbers, London's local municipal boroughs are now demonstrating that, even with the constraints and heated conditions of the UK capital, another housing world is possible. It provides valuable lessons for cities across the world.

Set up in 1965, London's boroughs delivered hundreds of thousands of homes in the postwar years, via direct subsidies and managing rental income streams. They strived to create mixed communities linked to local economies and other public services, by providing a variety of housing for all kinds of needs and as well as low-middle incomes. From the late 1970s, in line with tough conditions of an immense International Monetary Fund (IMF) loan that bailed out the UK economy, subsides were rolled back and after 1980 they more or less disappeared altogether.

In a downward pattern witnessed in many North American and European cities as well, public housing programmes collapsed, housing security for many people was removed and decades of accumulated public design capacity and expertise was lost. For most of the rest of the decades that followed, the role of London's boroughs was simply to facilitate and enable the activities of the private market, for example selling vast amounts of land at discount and transferring publicly built housing stock to not-for-profit housing associations,as local-led management and maintenance regimes failed.

It did not have to be this way.

Indeed, European cities such as Vienna pursued completely different courses and the situation in those cities is widely different. In the case of Vienna, there has been steady support for public-led housing programmes and urban projects of scale for over 50 years. Today around 60 per cent of the inhabitants live in public housing, and it is available for middle and low-income earners.

The Viennese have some of the best quality housing and highest living standards anywhere in the world; while there are pressures, there is no housing crisis. Today, the city oversees projects that also facilitate co-operative and community-led housing options as part of overall urban plans.

Boroughs such as Hackney are now responsible for sophisticated and highly nuanced housing programmes

The origins of housing change in London lie in 2007, when the UK's then Labour prime minister Gordon Brown started a process that meant that London's boroughs could borrow money against the value of their land and rent incomes. In doing so, he enabled them to access the means to tackle new housing programmes for themselves and thus start building.

Since this time, boroughs such as Hackney in the east of the city and Brent, Camden and Enfield in the north have taken the initiative, and are now responsible for sophisticated and highly nuanced housing programmes that are delivering some of the best new homes anywhere in the UK.

Over the past 15 years, my practice Karakusevic Carson Architects has been working with Hackney and 12 other boroughs on what is now emerging as a confident new era of public housing.

Embracing a range of strategies including stock refurbishment, strategic infill and large housing estate redevelopment, they are using their own land and acquiring new sites to reverse years of underinvestment. In addition, with the help of the Greater London Authority – London's strategic upper tier of government – and mayor Sadiq Khan, boroughs are developing in-house design and development expertise. It is an ambition for quality public housing design that the UK has not seen since the 1970s.

A key difference to this new generation of projects has been a move back to public responsibility and oversight that is maintained through to construction stages, which has kept standards high. This simple shift is essential in the long-term success and stewardship of housing stock. By retaining architects and clerk of works, boroughs have ensured that value for money, quality and design intent is upheld and delivered.

To achieve housing numbers and ensure large sites are financially viable within constrained borough finances, cross subsidisation has been necessary. Where this occurs, boroughs sometimes work with private-sector partners to develop for-sale properties that help fund future stages of development, introduce a mix of home types, and support a range of local community facilities and infrastructure such as new parks, leisure amenities or schools.

This was the case with the Kings Crescent and Colville estates in Hackney, two neighbouring projects we have been working on over the past eight years. Despite very uncertain times and an era of council austerity, Hackney wanted to fulfil its promises to provide residents with new homes. Through cross-subsidy, the council was able to bring forward multiple phases of mixed-tenure homes, which tenants and leaseholders are now starting to move in to. By the council staying in control and managing the project, we have been able to design the new rented properties to a very generous standard, and they are built using the highest quality of materials by extremely proficient builders on tightly controlled contracts.

There are also projects now exploring new types of mix that, like in Vienna, support community and self-build options alongside market and direct public delivery across all types of tenure, ensuring all value is retained by the city.

If this new era of public housing is to succeed, in London or elsewhere, it will need more than finance

This new generation of public projects and their model of development is not without its critics. Debate in London has and continues to be coloured by a number of schemes where local residents subject to estate transformation have been treated poorly, creating understandable suspicion and raising local tensions. For processes to improve, these cases need to be studied and understood. But many more boroughs are working successfully with the support of the vast majority of their communities and delivering on promises made to them for improved or new homes.

In October last year, the UK government reversed years of embedded ideological resistance to the public sector becoming a major part of housing delivery and lifted the caps on borrowing for boroughs. This is extremely welcome and an acknowledgment of the scale of the challenge both in London, but also in the UK's next largest cities such as Manchester, Bristol and Birmingham.

It means that in future cities, in the absence of meaningful subsides, may be able to act independently and with a stronger degree of civic autonomy to access greater finance and tackle a greater number of projects for themselves. However, if this new era of public housing is to succeed, in London or elsewhere, it will need more than finance.

The future health and vitality of London and cities like it depends on a fundamental long-term shift in attitudes about the role of public housing, but also the capabilities of the public sector to deliver good urbanism. Resilient cities are places that look after their citizens and incubate opportunity. London's new generation of public projects are promoting such values and demonstrating that change is possible even amidst political and economic adversity. They represent an ambition not just for affordable homes, but fundamental principles of city making and public action that is born out of thinking long term, embracing design quality and working with and for communities and citizens.

The case for new public housing is being made across the world and the clamour for it is getting louder. If the 21st century city is to evolve and realise its potential, architects and designers of all backgrounds need to align themselves with one of the great urban struggles of our time, and challenge the economic and political assumptions of the past four decades that says it cannot be done. It can, it is and we must demand more of it.

Mike Althorpe, research and development manager at Karakusevic Carson Architects, provided research for this piece. Photograph of Colville Estate is by Pete Landers.