The first mistake … detail from The Expulsion of Adam and Eve from Paradise, by John Martin. Photograph: Laing Art Gallery, Newcastle

One day in 1972, Village Voice journalist Ross Gelbspan attended a press conference. It was being held to promote a book called The Limits to Growth, which postulated that, because of increasing population and pollution and diminishing resources, our future world would be a place where no one would want to live. During the conference, Gelbspan was struck by the happy sight of one of the book’s co-authors, Donella Meadows. How heartening, he thought, that despite her book’s grim prognostications, she was pregnant. He went back to the office and typed up a story about how there was some hope amid the gloom, symbolised by Meadows’s swollen belly. His editors liked the story so much they put it on the front page.

There was only one problem: Meadows wasn’t pregnant. As I write this, I can feel blood rushing to my face in empathetic embarrassment. Even today, nearly 40 years after the error and almost a decade after Meadows’s death, Gelbspan is still mortified. At the time, he wanted to die. However, let’s snatch optimism from Gelbspan’s understandable anguish. As Aristotle wrote in the Ethics, it is not good to feel shame – since it is bad to have done something one should feel ashamed of – but to do something wrong and not feel shame is a sign of wickedness. In an increasingly shameless world, Gelbspan’s authentic distress is a sweet sign that not everything about us is going wrong.

In this lovely book about human mistakes the sickeningly young, forbiddingly clever and vexingly wise American journalist Kathryn Schulz doesn’t cite Aristotle, but he is a kindred spirit. Where Aristotle saw the value in a painful, ostensibly demeaning emotion, Schulz argues passionately for the value of error. The experience of being wrong, she argues, helps to make us better people, with richer lives.

We have all sensed the shame of being publicly wrong. I will always drag around, like Jacob Marley’s chains, the clutch of errors I committed nearly a quarter of a century ago in an article about junior doctors’ working hours. But journalists’ mistakes are the least significant in the harrowing catalogue of human wrongology: think of the surgeon who cut off the wrong leg, the pilot who pushed the wrong button, the judge who hanged an innocent man; of waking up next to Mr Wrong, or being Boris Johnson’s barber.

Or consider William Miller, who predicted that Armageddon would take place on the morning of 22 October 1844. Hundreds of thousands of devotees around the world were more than a little chagrined when the second coming of Christ foretold by Miller did not take place – not least because many of them had not bothered to harvest crops or even have breakfast, such was their conviction that the Rapture would occur before lunch. They experienced what came to be known as the Great Disappointment, while Miller, to his credit, produced a marvellous mea culpa in his tract “Apology and Defense”. “We expected the personal coming of Christ at that time,” he wrote. “And now to contend that we were not mistaken, is dishonest. We should never be ashamed to frankly confess all our errors.”

Few of us, when we err, feel the urge to confess thus. We prefer to be in denial than to confront the unacceptable truth that we were ever wrong; to be in the right is our natural home. This is why Keats’s negative capability – “when man is capable of being in uncertainties, Mysteries, doubts without any irritable reaching after fact & reason” – is so challenging to rational animals like us. To be in uncertainty is to be conceptually roofless. And to be in the wrong is even worse – not just a challenge, but a shattering experience of self-alienation. It is for that reason, no doubt, that according to Elisabeth Kübler-Ross’s five stages of grief, denial is a proper reaction to trauma.

Of course, one can go too far with this eulogy to obduracy. The refusal of South Africa’s former president Thabo Mbeki to admit that Aids is caused by HIV, and can therefore be controlled using antiretrovirals, contributed to the deaths of around 320,000 and made orphans of many others. And while Schulz is very good at exploring the manifold human mechanisms of denial, even she can’t explain why Hans Blix never managed to convince either Bush or Blair that their conviction about the existence of those weapons of mass destruction was wrong.

Denial is hard work. Sartre wrote that, in order to be self-deceiving, “I have to know this truth very precisely in order to hide it from myself the more carefully”. Tough gig: to be in denial we need to be both the trickster and the tricked.

What is most cherishable about this bumper book of other people’s booboos is its insistence that to experience error is, at its best, to find adventure – and even contentment. Schulz takes as her model Don Quixote, the knight-errant who was wrong about almost everything. “Countless studies have shown that people who suffer from depression have more accurate world views than non-depressed people,” she points out.

But Schulz is hardly counselling that we choose to be wrong. That would be a contradiction: error is like happiness, perhaps, in that we can only stumble across it rather than seek it out. Instead she proposes that once we find ourselves in the wrong, we should be optimistic. To be wrong, after all, is to depart from the facts into creativity, to become artists in our own lives. Error may feel like despair, but it is more akin to hope: “We get things wrong because we have an enduring confidence in our own minds; and we face up to that wrongness in the faith that, having learned something, we will get it right next time.” That may be no consolation to the patient who loses a healthy limb or the pilot whose mistake chucks a 747 into the Atlantic, but for those of us whose errors are less disastrous it’s a cheering perspective.

There is, if you apply Schulz’s sunny philosophy of wrongology, a parallel between the Guardian’s Corrections and Clarifications column and evolutionary theory: both are premised on the usefulness of error. When the Guardian launched its forum for nitpickers, journalists feared that public shaming over errors would be a poor exchange for the furtive pleasure of denial, and the delusion that one is incapable of being wrong. It proved otherwise: openness about committing mistakes, even if they are ones that make you want to lie face down in a shallow grave and cry for all eternity (I speak as a regular Corrs and Clars contributor), is essential for public trust and – that demanding monster – personal growth. Similarly, Schulz argues, “because of errors in the replication of genetic sequences, we wind up with variation among individual members of a species; and because of that variation, the species as a whole can adapt and survive. Such errors literally keep their hosts alive.”

She takes a similar upbeat moral from Thomas Kuhn, the great historian of science who noted that the periods between the breakdown of one system or theory and the entrenchment of another are always marked by an explosion of competing hypotheses. Each one is a response to the inadequacies of the previous orthodoxy. “By determining where we went astray in the past, in other words, we also determine where we will wind up in the future.” Being wrong, therefore, is important for scientific progress.

Schulz’s self-styled discipline of wrongology contains a great deal of lightly worn learning and careful reflection on psychoanalysis, philosophy, science and art. The book, if too long, is for the most part delightfully written. It reminds me of what we want clever Americans to be: positive, without a raincloud for a heart, and yet with the wit to make that optimism compelling.

There is a philosophy called “pessimistic meta-induction from the history of science”, which proposes that, since the most watertight of old theories have been disproved, we must assume today’s theories will be disproved, too. This holds for economics, medicine, education – any discipline you’d care to mention. It’s a good theory but not quite as appealing as Schulz’s rival account, which she calls “optimistic meta-induction from the history of everything”. This states that our capacity to err is inseparable from our imagination. “In the optimistic model of wrongness, error is not a sign that our past selves were failures and falsehoods,” Schulz writes. “Instead, it is one of those forces, like sap and sunlight, that imperceptibly helps another organic entity – us human beings – to grow up.” Schulz doesn’t relate whether she converted her colleague Ross Gelbspan to her optimistic meta-induction theory, but I hope she did. He’s surely suffered enough.