Turn Toward Politics

What are the most pressing problems in the world? Where should those looking to do the most good go? One obvious problem is X-risk. The most urgent X-risk, I suppose, is insufficiently-aligned ASI.

In the world where ASI kills us all, that's that. In the world where it doesn't, though, what then? Does it not kill us because it obeys some human or group of humans? Or is it because it values our well-being, having been programmed to do so? (Maybe then it's a "benevolent dictator"?) Or could it have been programmed with a respect for us, maybe such that it is a minimalist world government protecting our agency. Maybe it wants us to figure out the Long Reflection, and, since morality for some reason has something to do with human instincts, defers to us to define what is of value.

If ASI has enough "respect" for us and our decision-making abilities, or is programmed explicitly to obey certain persons or groups, then humans may or will in some sense still be masters of the ASI, no matter how much smarter it is than us.

So what might happen in the future, is that the bottlenecks to altruism on a high level will no longer be in the economic-technological world, but instead will be in coming up with political will to unify people to make important decisions (or to not be in a state of conflict with each other -- "cold" conflict if outright war is suppressed by the ASI), and also to manage the danger of bad unities (totalitarianism, for instance). ASI can provide arbitrary amounts of economic and technological development (perhaps), but can't do anything about the human political order, by its own (self-)limitation.

So those who want to do good (whether secular or religious), who have a personal fit for the political world (and adjacent areas like religion, art, and whatever else goes into "culture"), or who simply can't help in much of a direct way with whatever other things might seem more urgent as of 2022 (AI alignment, other X-risk aversion, etc.) -- they could turn toward politics (and areas adjacent to it).