I may rewrite this, maybe try to make something for the EA Forum. Also, for a mildly "continental philosophy"-leaning treatment of fiducialism, see this booklet on the subject.
15 Sep 2020: Receptivity to Reality is an important addition to this.
11 Sep 2020: edited for clarity.
What is fiducialism?
Here is a replacement for a common idea that affects how people approach caring. The idea is that by changing the idea, there might be some change in how people actually approach reality. I think culture is embodied in people rather than in words. But the idea matters somewhat, even as only a signal or as an image to appreciate like you might appreciate a beautiful landscape. Also, with effort, maybe this idea can be put into practice or embedded in technological design or policy.
Two popular versions of utilitarianism are: the one based in maximizing pleasure and/or minimizing pain (hedonic), and the one based in people getting what they really want (preference). Opinions may differ on which one is best. If I had to choose between the two for myself, I think I might choose preference over hedonism. But I can see the value in pursuing both. Some sort of balance between the two seems to be preferable to only having one or the other.
What I would suggest is to consider replacing hedonism with something else -- maximizing trust / minimizing betrayal. This could be called "fiducial utilitarianism". Or, in contrast to "hedonism", "fiducialism". My definition of "trust" is "receptivity to enhancement", which is taken from Joseph Godfrey's Trust of People, Words, and God. ("Fiducial" is his word for "pertaining to trust".) My definition of "betrayal" is "that which is an insult to the 'organ of trust'", "insult" in the medical sense. You can imagine that many things that cause pain are a form of betrayal. If you get sick enough, you don't feel like socializing. When you're depressed, the world seems small and lacking depth. After a break-up, it can take a while to get to where you want to date new people. Some bad experiences can (seemingly) permanently keep you from seeking out new experiences in a certain vein.
When in a bad situation, do you trust it, or not? You can try to trust it as much as it ought to be trusted. So you might resist the damage that the situation "wants" to do to you. But when you think about it late at night, rather than seeing it as unfair chance, or some kind of cosmic punishment, you can see it as a discipline, or you can practice acceptance. The discipline view is receptive to enhancement from the bad situation. In a sense it calls it "good". Similarly with the acceptance view. If you understand the situation, you know that it is also bad and unacceptable. But you can be disposed to find exactly whatever good there is in it.
When you are in a good situation, do you appreciate it? A fiducialist ought to not fall prey to the hedonic treadmill. A fiducialist seeks to trust a good situation even more than what seems called for by how it immediately presents. Maslow's Hierarchy seems like something that inevitably leads to the pinnacle ("self-actualization", or whatever might be higher), but it doesn't have to. People can settle. But a fiducialist doesn't settle. Their receptivity tends to cause them to find reality insufficient -- not necessarily bad, often good, but they are receptive to something more.
This disposition leads a fiducialist to reach out beyond the present moment, to imagine a better world or a better reality. A fiducialist doesn't just see the present. They want to connect to the deepest, truest reality. A fiducialist is disposed to work, explore, observe, and endure.
Fiducialists ought to be people who care. People who care are the very people who go into pursuits like altruism. It is also possible that a fiducialistic orientation can be protective against value drift, in the cases that receptivity to certain values fade over time, causing drift.
Comparison with hedonism
To a large extent, hedonic utilitarianism is satisfied by fiducial utilitarianism, although the reverse is less true. The disjunction could be, on the one hand, hedonism's concern for small pains on the one hand, and fiducialism's concern for settling at a comfortable local maximum of suffering, and fiducialism's desire to trust even suffering. I think any example of suffering that a hedonist could give, saying "how could you not get rid of this suffering if you had the choice?" would tend to be something that a fiducialist would object to as well. Suffering does tend to deepen people -- some people. But it embitters others. So a good goal for a fiducialist is to maximize trust without there being suffering. The deepest trust tends to be developed in times of breakdown, and suffering. But that is only a tendency, and in principle, we could learn to trust just as deeply without any of that. It could be argued that some people choose to suffer so that they can be broken down, and then trust (maybe EAs don't think this way...).
Hedonism has some edge cases that fiducialism might disregard. For instance, in thought experiments involving billions of people getting paper cuts, a fiducialist would say "let those people learn to deal with paper cuts -- they can live with that -- life is rich enough for there to be paper cuts". But the things that are harder for people to live with would be relevant material for fiducialism to target for changing.
I think a lot of people lean fiducialist already, and would be more motivated to deal with the problems that hedonism targets through the means of an intensified, self-conscious fiducialism than they would be by the language of hedonic utilitarianism.
Comparison with preference and hedonistic utilitarianism
Fiducialism would seem to produce people who had high life satisfaction, and one could "integrate" one's life over many moments of satisfaction to produce an overall satisfaction with life. In other words, those prone to trusting the moments of their life end up trusting their lives as a whole. Aspects of fiducialism resemble preference utilitarianism. You could look at it like, you adjust your preferences to fit your situation, and that is trust.
Preference-orientation says "I get what I want". A preference- orientation that says "what I want is to accept the moment" is not the same as trusting the moment out of trust-orientation. Trust-orientation says "I am receptive, open to, connecting with reality outside myself except insofar as it limits my ability to connect with reality outside myself." So the question, for a hedonist, is "how do we reduce suffering / increase pleasure?", for a preference utilitarian, "how do we give people what they want?", and for a fiducialist, "how do we facilitate, encourage, teach, lead, become trustworthy toward people such that, they connect with reality?" Out of this connection with reality comes caring.
(Incidentally, this emphasis on connecting with reality could unify the opposition between instrumental and epistemic rationality. Instrumental rationality in fiducialism is defined as "connecting with, or helping someone connect with, reality" and epistemic rationality is a subset of connecting with reality.)
Psychological and political considerations
One reason why people are more trust-oriented than otherwise is that they have burned out on other forms of processing reality. From my own experience, trust is easier on the brain and body than morality. If you care out of trust of the good of good work rather than out of a horror of suffering (or hatred of those, yourself and/or others, who do less good or more bad than they should) then you can stay in the game longer (or, I can). It's an open question who works harder or can put in more work in in the long run. A fiducialist has to remember some of the lessons of the horror-oriented altruist -- trust horror-orientation for what it's worth.
Relevant to polarization, political opponents tend to be hatred- or horror-oriented, rather than trust-oriented. Ostensibly, everyone is on the same basic side: "Doing what's right for America" -- a kind of altruism. Hatred and horror see things in a bad light -- so some altruists hide from politics. But if politics were trust-based, it would not be much like that.
I don't know the exact connection between hatred, horror, and hedonism. Probably there is some non-fiducialist approach to things that is neither hatred- nor horror-based but which is still intellectually hedonist. I think there is a kind of correlation between hatred, horror, and hedonism, at least in ways that correlate with moving away from trust, which fiducialism avoids.
The word "love" is attractive at large, and could be seen as a substitute ideal to trust, but I think "trust" is better. Trust precedes love. Also, trust is more basic and simple than love. From Godfrey's basic definition, I think the simplest conscious beings trust, when they may not love. We always trust -- even our doubts and suspicions.
Since fiducialism can adopt the goals of hedonism to such a great extent, since it may protect against burnout and value drift, since it is consonant with cultural changes that favor altruism, and since it ought to promote the supply of people who might become altruists or altruism-adjacent people in the future, I think it is a good replacement for hedonism in moral calculations and policies.