Things We Cannot Get Wrong
If you prefer to listen rather than read, this blog is available as a podcast here. Or if you want to listen to just this post:
Michael Crichton, the author of Jurassic Park, and the Andromeda Strain, along with a bunch of other great books, and who, for my money, died too soon at the age of 66 from lymphoma, said many very astute things (and probably some dumb ones as well) and I’d like begin this post by relating something he said about the limits of expertise, what he labeled Gell-Mann Amnesia:
Media carries with it a credibility that is totally undeserved. You have all experienced this, in what I call the Murray Gell-Mann Amnesia effect. (I call it by this name because I once discussed it with Murray Gell-Mann, and by dropping a famous name I imply greater importance to myself, and to the effect, than it would otherwise have.)
Briefly stated, the Gell-Mann Amnesia effect works as follows. You open the newspaper to an article on some subject you know well. In Murray’s case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward-reversing cause and effect. I call these the “wet streets cause rain” stories. Paper’s full of them.
In any case, you read with exasperation or amusement the multiple errors in a story-and then turn the page to national or international affairs, and read with renewed interest as if the rest of the newspaper was somehow more accurate about far-off Palestine than it was about the story you just read. You turn the page, and forget what you know.
That is the Gell-Mann Amnesia effect…
I have certainly experienced this effect and I imagine the rest of my readers have as well to one extent or another. Crichton’s larger point is about the danger of speculation, but for our purposes the key takeaway is that no matter how authoritative something sounds, there’s a better chance than you think of it being mostly wrong and it may, in fact, advance a point which is the exact opposite of the truth.
I bring this up because we appear to have an example of this happening, and in an area I’m very much interested in. Just recently a Detailed Critique of the “Existential Threats” chapter of Pinker’s Enlightenment Now was posted online. It was written by Phil Torres, a noted scholar of existential risk, and having spent several of the last few posts discussing the book, and in particular Pinker’s dismissal of existential threats (combined with my larger interest in that topic). This seemed right up my alley. The critique is quite long, so I’m mostly going to focus on areas where I have something to add, or where I disagree. In particular, though this may not be an addition or a disagreement, I want to emphasis, that of all the subjects Pinker discusses, the one he (and really all of humanity) can least afford to be wrong about is the subject of existential risk. We can survive if the murder rate is much higher than he thinks, or if we got it wrong on same sex marriage, we can even survive most of the predicted outcomes of climate change, but if we get existential risk wrong, nothing else we got right is ever going to matter.
The key findings of this critique are:
Two quotes being used by Pinker in the chapter are in the “wet streets cause rain” category (my words) in that, their original meaning is not what Pinker claims and may in fact be “outright contradictory”.
The chapter spends most of its time attacking straw men.
Pinker’s citations are poorly vetted, and largely non-scholarly, but presented as scholarly.
And from those sources, Pinker ignores content which undercuts his arguments. Meaning the sources themselves are far more equivocal than Pinker represents.
Finally:
Overall, the assessment presented below leads me to conclude that it would be unfortunate if this chapter were to significantly shape the public and academic discussions surrounding “existential risks.” In the harshest terms, the chapter is guilty of misrepresenting ideas, cherry-picking data, misquoting sources, and ignoring contradictory evidence… Because, so far as I can tell, almost every paragraph of the chapter contains at least one misleading claim, problematic quote, false assertion, or selective presentation of the evidence.
Torres then goes on for an additional 20,000 words, and yet, in the end, Pinker’s errors are so dense, at least in this chapter, that he only manages to cover the first third of it. Obviously I have even less space available, so to start with I’d like to focus on the role of pessimism and religion. And to do that I need to identify some of the different groups in this debate.
Pinker opens the chapter by framing things as a battle between those who are entirely optimistic (like himself) and those who are entirely pessimistic. These are the first two categories. What’s interesting is that for all Pinker disparages religion, believers technically don’t fall into the category of those who are entirely pessimistic. To find people who are entirely pessimistic you have to look at people like the antinatalists, who I recently discussed or really hard-core environmentalists. Believers are optimistic about the future, particularly over a very long time horizon, they may just be pessimistic in the short term.
Torres takes immediate issue with framing the issue of existential risks in this fashion, and he points out that many of the people Pinker talks about are very optimistic about the future:
Pinker’s reference to “pessimists” is quite misleading. Many of the scholars who are the most concerned about existential risks are also pro-technology “transhumanists” and “techno-progressives”—in some cases, even Kurzweilian “singularitarians”
In other words these people all firmly believe that a technological utopia is not only possible but likely. They just believe it’s even more likely if we can eliminate potential existential threats.
The fact that Pinker simplifies things in this fashion is emblematic of his entire approach to this subject. And frankly represents some pretty appalling shoddiness on his part, but that’s not the point I want to get at.
As I said I want to identify the various groups, so let’s get back to that.
As I pointed out, the first group is composed of the entirely or mostly optimistic, and I think it’s fair to put Pinker in this category. The group of people who believe that things have never been better and in all probability these improvements are going to continue. To put it in terms of my overarching theme, these are people who believe that technology and progress have definitely saved us.
Our second group is the entirely or mostly pessimistic. I already mentioned the antinatalists, but I would also include people who are convinced that the earth will be made uninhabitable by climate change, or that the holocene extinction could lead to some sort of global tipping point. Pinker puts people concerned with AI Risk and those concerned with the possibility that genetic engineering could make it easy to create superbugs in this category, though Torres argues he probably shouldn’t. That said, there are certainly people who believe that technology has definitely doomed us (and that there is no religious salvation around to mitigate this doom) it’s just not clear how large this group is.
As Torres points out in the quote above, there is a third group which contains those who experience a mix of optimism and pessimism. Those who feel that the future is incredibly promising, but caution must be exercised. They believe that technology can save us and hopefully will, but that if we’re not careful it could also doom us. Given that the majority of these people don’t expect any religious salvation you could see why they’re particularly worried about getting it right. For myself, despite not falling into this group it seems clearly superior to group one.
The fourth and final group also contains those who see reasons for both optimism and pessimism, but in this category the optimism is primarily based on faith in a divine being rather than being based on faith in technology and progress. Here, I imagine that Torres and Pinker might set aside their disagreements to declare this group the worst of all. Obviously I disagree with this. And before the end of this post I’ll get around to explaining why.
I suppose it’s not entirely accurate to label that last category as the final group, since there is, of course, the largest group of all: people who never really give much thought as to whether they should be optimistic or pessimistic about the future. Or rather, they may give some degree of thought to their own future, but very little to humanity's future. The big question is how much influence do they wield? Part of the reason why Pinker writes books and why Torres writes rebuttals, is that it’s hoped that the future will not be determined by this group, that it will be determined by people who have taken the time to read books like Enlightenment Now (and even better people who might have read rebuttals like Torres’) though that’s by no means certain. Still, I guess I too will make the same assumption as Pinker and Torres and say no more about this group.
Pinker wants to frame things as a battle between groups one and two, and while I agree that there is a group of hardcore technology pessimists, I don’t think they’re that large. Also, all (or at least the majority) of the people Pinker call out more accurately belong in group three. Which is another way of saying that this is a good example of the strawmanning Torres is complaining about.
If we set group two aside, both because it’s too small, and also because of its relative lack of influence then we end up with most of the attention being focused on the contest between groups one and three. As I said I think group three is clearly superior to group one, but it’s useful to spend a moment examining why this is.
The first big question is what are the risks? I mentioned that this is one thing I want to focus on, in fact it’s the point of the title. The risk of being wrong about existential hazards, are, from a certain perspective, infinite. If we make a mistake and overlook some risk which wipes out humanity, that’s basically an infinite risk, at least from the standpoint of humans. If you’re not comfortable with calling it an infinite risk, then it’s still an enormous risk as Torres points out:
...This is not an either/or situation—and this is why Pinker framing the issue as an intellectual battle between optimists and pessimists distorts the “debate” from the start.
(i) given the astronomical potential value of the future (literally trillions and trillions and trillions of humans living worthwhile lives throughout the universe), and (ii) the growing ability for humanity to destroy itself through error, terror, global coordination failures, and so on, (iii) it would be extremely imprudent not to have an ongoing public and academic discussion about the number and nature of existential hazards and the various mechanisms by which we could prevent such risks from occurring. That’s not pessimism! It’s realism combined with the virtues of wisdom and deep-future foresight.
This is the position of group 3, and as I said I think it’s pretty solid. The risks of not paying attention to existential hazards is enormous. On the other side what is the argument for group one? What are the risks of paying too much attention to existential hazards? Here’s Pinker explaining those risks:
But apocalyptic thinking has serious downsides. One is that false alarms to catastrophic risks can themselves be catastrophic. The nuclear arms race of the 1960s, for example, was set off by fears of a mythical “missile gap” with the Soviet Union. The 2003 invasion of Iraq was justified by the uncertain but catastrophic possibility that Saddam Hussein was developing nuclear weapons and planning to use them against the United States. (As George W. Bush put it, “We cannot wait for the final proof—the smoking gun-that could come in the form of a mushroom cloud.”) And as we shall see, one of the reasons the great powers refuse to take the common-sense pledge that they won’t be the first to use nuclear weapons is that they want to reserve the right to use them against other supposed existential threats such as bioterror and cyberattacks. Sowing fear about hypothetical disasters, far from safeguarding the future of humanity, can endanger it.
Honestly this seems like a pretty weak argument. And Torres points out several problems with it, which I’ll briefly recap here:
There’s a big difference between, for example, the warnings about AI provided by Elon Musk and Stephen Hawking and whatever it was Bush was doing.
This overlooks all the times when people warned of catastrophe, and it turns out we should have listened. Exhibit 1 for this is always Hitler, but I’m sure I could come up with half a dozen others.
This also overlooks times when warnings were acted upon, and the problem was fixed, sometimes so well that people now dismiss the idea that there was ever a problem in the first place. Pinker offers the Y2K bug as an example of techno-panic, and Torres goes to show it really wasn’t, I don’t have to time to get into that, but Pinker seems to assume that warnings of catastrophe are never appropriate and always bad, which is almost certainly not the case.
To this list I’ll point out that neither of his examples are particularly good.
The Iraq War was bad, and in hindsight, almost certainly a mistake. But it wasn’t a catastrophe, certainly not compared with other potential catastrophes throughout history. Perhaps when considered only from the perspective of the Iraqis themselves, it might be, but I’m not sure even then.
If the nuclear arms race had lead to World War III, then Pinker would certainly have a point, but it didn’t. However mistaken you think the arms race was, we avoided actual war. This despite the fact that many people were pushing for it. (I think the number of people who thought it might be okay went down as the numbers of nukes went up.) How sure are we really, that a world where the arms race never happened would be better than the world we currently have?
Pinker brings up other risks, which Torres covers as well, but none of them, when set in the balance, outweigh the colossal risks of potential existential hazards.
Before we move on there’s another argument Pinker makes that deserves to be mentioned. One of the key points which determines how risky technology has made things, is the ease with which an individual or a small group can use it to cause massive harm. Pinker claims that technology’s interconnectedness has made it harder. He makes this claim primarily based on the increase in the number of intersecting technologies, all of which would require separate areas of expertise in order for an individual to cause any harm. He concludes that this makes harm less likely than in the past, and that we have been mislead in this respect by the hollywood image of a loan genius. That, rather, it would take a whole “team of scientists” and that maybe they wouldn’t be able to do it either.
This certainly doesn’t match my experience of things, and Torres take serious issue with it as well and goes on to provide nine counter-examples of small groups either causing massive harm or having done all of the work necessary to cause massive harm but stopping in advance of any actual harm. If you read nothing else from the original paper, I would at least review these nine examples. (They start on page 31 and go through page 33.) They’re quite chilling.
The overall feeling I came away with after reading Pinker’s chapter on existential risks, a feeling Torres appears to share, is that Pinker thinks that people who are pessimistic about technology aren’t acting in good faith to prevent some disaster, but rather they’re doing it as part of some strange intellectual exercise, a weird game perhaps. Here’s an example of Pinker expressing this sentiment:
The sentinels for the [old] horsemen [famine, war, etc.] tended to be romantics and Luddites. But those who warn of the [new] higher-tech dangers are often scientists and technologists who have deployed their ingenuity to identify ever more ways in which the world will soon end.
Torres joins me in thinking that this makes it sound like people who are concerned with existential risk, are: “devising new doomsday scenarios [as] a hobby: something done for the fun of it, for its own sake.” and goes on to state, “That’s not the case.” So far Torres and I are in agreement, but I would venture to say that Torres makes a similar claim about people who are worried about some sort of apocalypse for religious reasons, and in a couple of places in his paper he goes out of his way to put as much distance as possible between his worries about existential risk and the apocalyptic worries of traditional religions. And here’s where we finally turn to group four: the religious pessimists (who nevertheless hope for divine salvation).
I’m not an expert on the status of apocalyptic beliefs among all the world’s religions, but I get the impression that it’s pretty widespread. Certainly it’s a major element Christianity, the religion I am the most familiar with, but regardless of how widespread it is, Torres wants to make sure that his worries about existential risk are not lumped in with the apocalyptic concerns of the religious.
The question is why? Why are the religious fears of an apocalypse different than the fears Torres is defending? Torres takes objection to the idea that researchers in existential risk, are “devising new doomsday scenarios [as] a hobby” but where does he suppose that religious apocalyptic fears come from? Is he copying Pinker now, and assuming that this was the hobby of early Christians? Something to spend their free-time on while undergoing persecution and attempting to spread the gospel?
I suppose that’s possible. That if it was, not exactly a hobby, that it at least, served no useful purpose, that it was just pointless baggage which for some reason accumulated within Christianity and did nothing for either the religion in which these ideas accumulated or for the followers of that religion. As I said, this is possible, but it seems unlikely.
Another possibility is the Talebian possibility, that there was something in these beliefs which made those who held them less fragile. It’s not hard to imagine how this could be the case, if apocalyptic beliefs led those who held them to be more prepared for eventual (and historically inevitable) disaster, then it’s easy to see where those beliefs came from and how they might have persisted. This possibility would appear to make a lot more sense than the first possibility.
Of course there’s one final possibility, that there is in fact a religious apocalypse on it’s way, and John of Patmos was actually warning us of something real. But if you reject this possibility, as I assume Torres does, then the second possibility still makes a lot more sense than the first. Which is to say, that it could be argued that it’s a huge support for Torres’ point.
The central argument between Torres and Pinker boils down to a question of whether it’s a net negative to worry about future apocalypses (Pinker’s view) or whether it’s a net positive (Torres’ view). To which I would argue that historical evidence suggests that it’s definitely a net positive, because that’s basically what people have been doing for thousands of years, and it’s safest to assume that they had a reason for doing it. Particularly given the fact that, as I’ve been saying, this is one of those things we cannot get wrong.
In closing I have two final thoughts. First, I think one of the benefits of bringing religion into the discussion, is that it allows us to tap into thousands of years worth of experience. Contrast this with Pinker (and to a lesser extent Torres), who are arguing about a world that has only existed for the last few decades. It’s really difficult to know if we’ve recently reached some new plateau where existential risk is so low that worrying about it causes more harm than good. But if you bring in religion and tradition more broadly the answer to the question is, “probably not”. We probably haven’t reached some new plateau. There probably is reason for concern. The last 70 or so years are probably an anomaly.
Second, I would be remiss if I didn’t mention that Torres ends his paper by referencing The Great Silence, which is another name for Fermi’s Paradox, and points out, as I have on many occasions, that if we don’t have to worry about existential risk, then where is everyone else? Sure, we all agree that there are lots of potential explanations for the silence, but one of them, to which we have no counterfactual, is that Pinker is horribly, fantastically wrong, and that technology introduces a level of fragility which will ultimately and inevitably lead to our extinction, or in any case will be inadequate to save us.
If you think there’s some point to religion, consider donating. If you think there’s some point to being worried about existential risk, consider donating. If you think Pinker could use more humility consider donating. And finally if you think it’s a tragedy that the Netherlands is not in the World Cup, consider donating. Because it is...