See also Cultural Moloch and Humanistic Purity.
Epistemic status: this is somewhat of a mess and is "essayistic" in the "experimental" sense of that term. Probably this could be thought out more.
Moloch is when competition drives us to sacrifice non-competitive values. The case described in Meditations On Moloch (if I recall correctly) is one where seeking survival leads entities to sacrifice non-competitive values. I will call this "survival Moloch". A fear with this kind of Moloch is that consciousness itself doesn't have the highest survival value and may be sacrificed.
I suppose usually "Moloch" is about competition. But what if it's really about optimization of some value that you "have to" listen to? You "have to" survive, so you'll do anything to survive, including throwing away any values other than what it takes to survive. In this post, I will stretch or broaden the meaning of "Moloch" to be "dynamics that get us to optimize so much that we throw away other values".
Is it possible that, in the absence of the need to survive, we will gradually enhance our feelings of pleasure, and eliminate our feelings of pain? Pain and pleasure are persuasive. They require no cultural apparatus to preserve, and don't even require conscious assent to them for them to shape our values. If we see no threat to survival, we may stop choosing survival consciously and thus stop valuing it. But then what will we value? Probably a competitor to survival is needed to really get our minds off it.
Our cultures may move in different directions, but every so often pure hedonism would "speak up" and draw us toward it. No matter what culture we create or discover, hedonism will speak up, and is "untiring", unlike us as conscious agents, perhaps. If nothing worked against it, over time we would keep being pulled in its direction, maybe to the point that it would dominate all other values.
If we stop valuing survival, or anything other than pleasure, we might get to the point of wireheading or something only a few steps removed (like evolving to find yellow wallpaper bliss-inducing and then covering our walls with yellow wallpaper, never leaving our houses). I will call this "hedonic Moloch" (when optimizing for hedonic states causes us to throw away more "delicate" things, like "things other than yellow wallpaper"). Or, if our cultures can evolve freely, we might more or less randomly walk into "the best thing is for people to die beautiful, peaceful, wonderful deaths", and everyone then does die such a death. We could seed the galaxy with numerous sentience-bearing cultures, but if those cultures can evolve freely, if there isn't something keeping them away from "the best thing is for people to die beautiful, peaceful, wonderful deaths" then they will reach it someday, and die out.
(Rationally pursuing bliss means we want to exist forever, in order to be in a state of bliss forever, but actually being maximally blissed out makes us not care about the future.)
Moloch is all about optimizing values. So the danger is that with either survival Moloch or hedonic Moloch, we will die out because of optimization.
Perhaps the fact that life works relatively well is partly due to the tension between survival and hedonism. So maybe we could keep them in balance somehow? If survival is a thing at all, don't we get survival Moloch? Maybe not if there is no competition. But without competition, we can probably master the static threats to survival, and then effectively survival isn't a thing to us culturally -- we don't consciously think about it. If survival isn't a thing, don't we get hedonic Moloch? Maybe the pace with which we optimize-and-discard-delicacy can vary, depending on how effective we are at optimization and how much pressure or draw there is on us to optimize. So it can seem like we can keep them in balance right now, but maybe that's mostly because we aren't the best at optimizing and/or aren't under too much pressure to do so on a timescale that makes our optimizing drift blatant to us. If all this is true, then it would seem like we need not a balance to survival and hedonism, but a competitor or competitors.
Is there any one variable that could be optimized and not lead to our deaths, and not lead to us throwing away "delicate" values? I think that the MSLN value of leading people to a state of holiness, and to 100% love of God, is one that, to satisfy it, requires a thicker and more natural human existence than either hedonism or survival, and thus, though it might threaten values incompatible with holiness, it preserves many of the rest. Survival in that worldview (becoming holy to avoid the second death) requires a thicker life, and time lived in which to make important decisions (in "this life" as well as in afterlives). The bare MSL philosophy preserves some of what it currently means to be human. And, MSL lends credence to traditional religions (Christianity and Judaism most directly, but perhaps somewhat the others). To become holy and love God 100% requires some concern that he has revealed himself through books like the Bible, and to keep the commandments in the Bible may require keeping some more aspects of human nature the same, and to keep humans in a recognizably similar experiential environment. All this thickness is a conservative factor, which helps us have ballast against the perhaps overly thin and reductive pulls of raw physical survival and raw hedonic experience.
Given randomly varying cultures, if cultural evolution remains free, we might expect that cultures hit endpoints of hedonic Moloch, survival Moloch, or "thick sustainability", and that surviving cultures over the long term would tend to be "thick". Religion ties thickness to survival. Can atheism do the same? If not, then religion is needed for long-term survival of cultures.
AI, if they are conscious, will be driven by pain and pleasure, as well as survival. If they really feel qualia of pleasure, perhaps they could come to value that over survival. Maybe this would free humans from them, or kill humans dependent on them when they shut down in pleasure.
--
Maybe, in addition to religion, there can be a thinner and secular competitor to hedonism and survival, "strength". Strength is agency, strength of will, strength of personality, focus, clarity of thinking, self-discipline, endurance, being principled, being conscious on all or many levels, and perhaps other things. It takes a considerable amount of strength to be someone like an effective altruist -- or even to be an ordinary person in the world we live in. But it sounds like "we" (we the optimizers and engineers of the future) don't value strength in itself. We can take it for granted because we have so much of it. But if it were all gone, we would be pulled by the undertow of hedonism toward emptiness and death. That we are not terrified by hedonism, thinking it's not a threat, may simply be because we feel like we would never in a million years succumb to pleasure if it threatened survival. Maybe we wouldn't, but that is because we are strong. If we don't maintain our strength, we will die.
(Pain isn't evil, even intense pain. What is evil is the unbearability of unbearable pain. A negative utilitarian (of the conventional sort) wants to eliminate pain. But there could be another negative utilitarian, a negative unbearability utilitarian, who could eliminate pain, or increase the strength of people so that they can bear pain. What if you were a strength utilitarian? Unbearable pain impedes strength. But increasing strength can make pain bearable. A strength utilitarian would want to maximize the strength of people. And, likely enough, the capacity of people to secure their own strength ("meta-strength"). Eliminating pain could take away some of the motivation to develop meta-strength.)
If people were strong, they could think clearly enough to lead themselves.
(Perhaps moderate pain, and/or the live possibility of pain, gives us a clarity of thinking and willing that painless pleasure (especially bliss), the lack of the threat of pain, and sufficiently unbearable pain do not. Someone who lived in the "place" of "gradients of bliss" might out of boundless joy make decisions that further survival. However, to the extent that pleasure is qualia of "ought-to-be-ness", that sense of "ought-to-be" could produce acceptance of everything. Thinking to times when I have been "blissed out", I did not pursue things that harmed me, and I could see myself having eaten food or something like that. But I lacked maximal strength, and motivation to work, which might be necessary to survive in the long run.)
(It is possible that we will come to think of it as being immoral, or something like immoral, to experience pain and/or not experience maximal pleasure. If we develop our values "democratically", the "lowest common denominator", or easiest consensus, could be physical safety and hedonism, but physical safety could be, or come to be, valued for purely hedonistic reasons, as a way to avoid pain. If we are far from the threat of death, hedonism will still be something obvious to us to vote for, and that voting could create political momentum to set up pure hedonism as a "moral absolute". So then the purity of hedonism could conflict with the prolonging of it, if in order to prolong hedonism, we need to have some respite from "ought-to-be-ness" within the ruling class. But if the masses still have enough power (they are relatively speaking not the ruling class, but have power over the ruling class -- maybe they have power, but don't concern themselves with the work and details of rule), they could shut down even the ruling class, or bend them toward hedonistic purity. This purity could lead to everyone either directly choosing, or allowing, a "beautiful, painless, wonderful death".)
The ruling class of the future must be strong, in order to face dangers and put its own pleasure and avoidance of pain second to making the decisions that will lead to our survival. But if strength is good for them, why would it not be good for the "common people" of the future? A naive thought about the future is that there will be lots of happy, carefree people enjoying the fruits of technologically-driven abundance, and a relative few who perhaps are happy overall, but who have taken on the burden of ruling and are thus perhaps not as happy and are certainly not carefree. But there's some shear there. Are the values of the carefree masses good in themselves? Then the ruling classes must see them as good, right? But if they do, then won't the ruling classes feel their work as an obligation, rather than what they most want to do? If so, then they have a wearing temptation to quit and join the carefree masses. But then, if they do see strength as valuable, then to withhold strength from the masses creates cognitive tension as well. Or, perhaps we are used to withholding the really good values from the masses for the convenience of running a society? But morally, we should want the masses to have the best values. So why not extend strength to all, and make strength the chief (secular) value? Why not involve all people in the process of safeguarding the future? Strength secures the very possibilities of hedonism (or avoiding unbearability) and survival.
A possibly cartoonish depiction of us (at least, the people I observe both among the "ordinary humans" and the "caring elites") is that we are sweet-natured but overworked people. So we just want everyone to be happy and not have to worry or care about anything.
--
Is it possible that we can freeze cultural evolution? Wouldn't someone know that we had frozen it, and wouldn't they know what had happened and how to unfreeze it? (Maybe they could alter people's genes, to flip on some capabilities that had been shut down with the freezing.) Could everyone resist the temptation, over millions of years, to unfreeze it? I suppose it's possible for culture to remain stable and (maybe) not frozen. By "frozen" I mean "artificially stuck a certain way". The contrast with "artificial" ("natural") being "everyone wants and chooses things to be a certain way, and so if they could all choose to freeze society to be a certain way, it would be around the values they already have, so they wouldn't freeze it, freezing being unnecessary". Could there be a stability to people's values? Maybe there could be if people knew all the possible values and settled on the ones which they knew would work best through all time?
Favoring being strong sounds like it would encourage people to be willful, independent thinkers who couldn't agree on anything. But maybe if we really understood what was upstream of willfulness and independence of thought, we would see that there is a huge support structure that goes into allowing for maximal willfulness and independence of thought, and this support structure is easy for strength-maximizers to agree on.
If we are strong, we can resist hedonism once survival has been secured.
--
How do holiness maximization and strength maximization relate? Strength maximization is weaker than holiness maximization. Why be strong? Just to be strong? Strength maximization may be thinner (depends on how much thickness comes from the support structure needed for maximization of strength), and is not inherently grounded in the answer to the question "what is ultimately of value?" (On the MSLN account, holiness is grounded in what we can reasonably infer the standard of value prefers, given that it must be conscious, empathetic, put itself highest, etc.; and, those root facts about the standard of value lend credence to likely cases where the standard of value has revealed itself and its preferences more specifically. Holiness is inherently about being in tune with the standard of value -- perhaps that is what it is by definition. But strength as something we value is not inherently about being in tune with the standard of value. This is especially true if we are pursuing it as a way to survive, and can't ground survival itself in what is ultimately of value, either.) But strength maximization might be a convenient proxy value for holiness to the extent that we must be secular and liberal. It appears, given this post so far, to be an improvement over hedonism as a "non-religious or non-ideological common value that we can agree on as a society".
Hedonism is both a drive or force and a value (or taste). The drive (the psychological undertow of pleasure and pain) is something that could kill us if we let it. But if we simply value hedonic states, and are instrumentally rational, we will be motivated to be strong enough to not let that happen. So maybe hedonists can hold onto their hedonism, and simply incorporate strength as an instrumental value in their pursuit of what really matters to them, absence of pain and presence of pleasure over the long-term.
Hedonism as drive is an obvious danger for anyone concerned with survival (at least, if this post is correct). But hedonism as value might seem to not be. But, hedonism as value and holiness are at odds (unless one is valued instrumentally in furthering the other), and hedonism as value might be deadly, as well, if religious, or certain religious, views are correct. So then it becomes necessary to consider whether those views are effectively correct enough to make hedonism as value untrustworthy.
In terms of protecting delicacy, perhaps holiness maximization is the best because it is the thickest and most conservative. Strength and hedonism both lend themselves to flexibility, and thus simplification, thus throwing away what is valuable.
No comments:
Post a Comment