Sean Griffiths pays tribute to postmodernism pioneer Robert Venturi, who died earlier this week, describing the huge influence he had on the way we view architecture and sharing personal stories about their meetings.

"Call that a f**king compliment?" replied a revered architectural theorist when the dean of a prominent American school of architecture, rather over-generously, introduced me as one of the few people for whom Robert Venturi would fly in to see talk.

That's what we loved about Bob. He'd pissed them off so much that they were still seething more than 40 years after he'd first ruffled their feathers by adding a few wooden mouldings to a house he'd completed for his mother in a suburb of Philadelphia in 1962. In doing so he had inadvertently invented postmodernist architecture and until quite recently, they'd never really forgiven him for it.

Of course, for me, and my colleagues Sam Jacob and Charles Holland it was about as big a compliment as anyone could give. Because Bob, together with his wife and collaborator, Denise Scott Brown, were our architectural heroes. They were the ultimate outsiders – producing buildings, exhibitions and publications that everyone knew were brilliant, but because they contained truths that disabused the architectural profession of the lies it told itself, it couldn't be admitted.

The message of those exceptional books, Complexity and Contradiction in Architecture (1966) and Learning from Las Vegas (1972) was that architecture was not a pure art form made manifest in pristine, ideal, uncontaminated forms. It was messy, compromised and inarticulate.

Bob and Denise were the ultimate outsiders

Nor was "good" architectural taste the preserve of privileged white males. Architecture should be inclusive of popular taste, "superficial" decoration and historical allusion. It should embrace "the ugly and the ordinary" as well as the heroic and original.

These are mainstream ideas today, but they were provocations in the 1960s and 1970s, and even by the mid 1990s, to be openly interested in the work of Venturi and Scott Brown remained a provocative stance. Which is, of course, precisely what our practice, FAT found so attractive about it.

We had rediscovered VSBA via a series of essays written by the artist Dan Graham, which placed their work in the tradition of American pop and conceptual art. These astute critiques played up the profoundly political nature of VSBA's recognition of a wide array of taste cultures, providing a refreshing contrast to those of the in-house critics of the perennially blinkered UK architectural scene who dismissed VSBA's, now Grade 1-listed, extension to the National Gallery in London as "picturesque, mediocre slime".

In their pathetic obsession with VSBA's use of classical figuration – a highly original use, one might add – they had missed a number of that building's genuinely radical moves. These included the violent, Gordon Matta Clark – like cuts through the rippling classical facade and the treatment of the building as, in a kind of self-parody of Venturi's concept of "the difficult whole", as a deconstructed fragment.

This is not to mention the morphing of recognisable classical spaces as if they had been recast in moulds made by Alvar Aalto, the violent dislocations of scale and the sophisticated plays of symmetry against asymmetry. Here was a masterful, if discordant symphony, that embraced grandeur alongside the ugly and ordinary, dismissed as superficial pastiche by superficial critics.

To be openly interested in the work of Venturi and Scott Brown remained a provocative stance

The same critical ineptitude was no doubt behind the Royal Institute of British Architects' shameful failure – or was it a refusal – to award Venturi and Scott Brown its prestigious Gold Medal for Architecture, as if designing a host of internationally significant buildings, writing two of the most influential books on architecture ever written, and pioneering an entire architectural movement were not enough qualification.

I wrote words to this effect in a review of a monograph on their work for the RIBA's in-house journal in 2002 and, to my delight and surprise, received in return a letter from Mr Venturi himself, thanking me for the review, and offering an invitation to come and visit him and Denise if we were ever to find ourselves in Philadelphia.

As it happens, in a delicious instance of serendipity, FAT had been invited to run a workshop and give a lecture at the University of Pennsylvania, and so came about one of the most memorable days that Sam, Charles or I will ever enjoy, as we were given a personal tour of Philadelphia by Bob and Denise.

The excursion took in the Venturi Scott Brown Home, Guild House, the Vanna Venturi House and across the street, Louis Kahn's Esherick House. Bob had worked on Esherick when he'd spent time in Kahn's office early on in his career and he delighted in pointing out which of its ideas had been his (most of them as it turned out, according to him at least).

As they drove us round, they playfully bickered over where to post a letter that ended up never getting posted, and the surreal qualities of the day extended into the evening where we agreed on the brilliance of the title sequences of the TV show The Sopranos, and the great architects revealed an unexpected passion for English sit-coms. "What's the one with the woman priest?" asked Bob as it dawned on us that our heroes were huge fans of, none other than, The Vicar of Dibley.

The next day Bob and Denise, along with their son, Jimmy, attended our lecture at U Penn after which Bob issued the immortal words, "Terrific! Keep up the bad work!". This was the first of many meetings between FAT and VSBA, during which they never failed to offer generous encouragement and support.

They never failed to offer generous encouragement and support

One of the last of these was at Yale School of Architecture, where in another of the more privileged moments in our careers, we had gotten to follow in the footsteps of Bob and Denise, and teach a studio in the graduate school.

The occasion was a symposium to celebrate the impact of Learning from Las Vegas whose featured participants, including architects Raphael Moneo and Peter Eisenman, and artists Peter Fischli and David Weiss, stood as testament to the profundity and breadth of influence that that marvellous book has had.

I've no idea whether FAT's championing of Venturi and Scott Brown has contributed to the renewed interest in, and respect for, their work that exists amongst today's generation of architects. I'd love to think that it did.

I've also no idea whether Bob, in his final years, during which he suffered from Alzheimer's disease, was aware that of the new esteem in which his and Denise's work was held. I'd love to think that he was. After an amazing life of producing seminal buildings, books and necessary controversies, as part of one of the great architectural partnerships of our time, it would be the least that he deserved.

RIP Bob Venturi, so many of our lives would not have been the same without you.