Poor Stanley Johnson. It must be galling to have carved out a five-decade career as an award-winning conservationist and travel writer yet find yourself best known to the wider world as “Boris’s dad”.

The 72-year-old has been gracious in the past, saying, “I have no difficulty having children who are more famous than I am. In fact, I regard it as a bit of an incentive.”

But the endless questions about his progeny are clearly starting to wear thin. “Hey hey hey,” he protests affably when I ask how he rates the London mayor’s chances of making Prime Minister, “let’s move on.”

Up until now, he’s cleverly managed to evade this line of conversation (and indeed others he doesn’t care for) by talking almost without pause; the similarities between Johnson Snr and Johnson Jnr run deeper than the matching blond mops.

So we don’t quite reach any conclusions on how the committed environmentalist handles Tory climate change deniers at dinner parties but he assures me he “absolutely does” believe in climate change (in fact, he was at the first intergovernmental meeting on the topic back in 1988) and thinks the people who don’t are fewer than we are led to believe. Also, he adds, he spends a lot of time on his bicycle – “if you’re wondering how I stack up as an environmentalist”.

Neither do we get very far on the irony of how Johnson, who devoted much of his early career to issues of population control, ended up with six children (four with his first wife, the painter Charlotte Fawcett, and two with his current wife Jennifer Kidd). Did he not feel any compulsion to do his bit? He humphs and hedges and agrees it’s a tricky one. “But all I can say is at the time I started thinking about this issue, the world population explosion was taking place in Asia, Africa and Latin America…”

And while he recently wrote about this divorce from his first wife, I’m much mistaken in thinking he might elaborate on it today.

“I think I’ve said everything I want to say on previous occasions but I’ve stayed very, very good friends with my first wife. Two nights ago we were all having a party in the garden to celebrate Boris’s 40th birthday so there have never been any problems there.”

He’d prefer, he says politely, understandably, to stick to talking about Where The Wild Things Were, his recently published collection of essays on his travels across the globe.

From horse-fighting in the Philippines to tracking pandas in rural China, via a rather less exotic trip to Glastonbury festival, the book is the latest in an impressive body of work (nine novels, various academic texts and lots of journalism).

Johnson developed his taste for adventure on leaving school in 1959 when he managed to travel across Europe, Asia and South America all on £40 put in his Post Office account by his grandfather.

Days after arriving at Cardiff on a cargo ship he joined in the port of Vitoria, Brazil, he started at Oxford, where he studied classics and spent his summers employed as a travel courier and, on one occasion, crossing China on a motorbike.

There followed weighty roles at the World Bank (he brought a little levity by writing a spoof paper outlining proposals to build a new pyramid in Egypt), a stint as an MEP, a period in Rome working for the food and agriculture arm of the United Nations and, towards the end of 2003, an unsuccessful attempt to revisit his earlier political career by running for election as the Conservative candidate for Teignbridge in Devon.

Although there is a tendency to see the Johnson clan as jokers, one would be foolish to underestimate them. Stanley Johnson, especially, has done a lot of serious stuff “and it’s out there and anyone who knows that knows that I’m very serious”.

Does he worry that Boris’s humour might be taken as flippancy, that he may be taken less seriously by the voting public than he deserves to be? “Boris? No! He’s written very serious stuff. Look at his vision for London – 100 pages of well-argued prose that he’s clearly written himself. But having a sense of humour is important – it means you have a chance of retaining your audience’s attention.”

In the recent TV documentary on the Johnson family, Stanley Johnson’s daughter Julia described the family as tremendously competitive and their father as a driving force behind this. If one of them came second in an exam, she said, their father would want to know who came first.

But Johnson says now that his children’s considerable achievements – Rachel is editor of The Lady magazine, Jo is Conservative MP for Orpington, Leo is a film-maker – are “nothing to do with me. Although of course I’m delighted for them and follow their careers with great interest. Hey, well, listen, let’s talk about something else…”

Even in his eighth decade, Johnson has a packed summer of trips ahead. He’s recently returned from a trip to Madagascar and he’s off to Papua New Guinea as soon as we’ve finished our conversation (if he manages to get hold of his missing visa).

Later he will be studying polar bears in the Arctic; he has been heavily involved in the campaign to introduce a ban on the international trade of their pelts, although in his telling of it this amounts mainly to unveiling a statue of a polar bear in Sloane Square.

Yes, he concludes, before I get a chance to get another word in, it’s a good life. “I do keep going,” he says of his career. “I can’t complain, I don’t complain. Hey ho, lovely to talk, so nice, take care.”