When we think about the future of what we call artificial intelligence (AI), we wonder if it will be possible to give a “robot” embedded with AI feelings and consciousness. In the first place, is it possible to make a machine learn such things? What is needed for that? These are just a few of the questions that come to mind. At times like this, reading futuristic novels can stimulate readers and make them feel as if they have caught a glimpse of the future.

With this expectation in mind, I read Klara and the Sun by Kazuo Ishiguro. Here I would like to talk about the issues of feelings and consciousness that this book stimulated in me. When I study geopolitics, I find it useful to think about the near future by looking into the distant future and how to project those predictions into the present. From this perspective, this book is interesting.

 

Clara, the “Artificial Friend”

The main character is Klara, a robot called “Artificial Friend” (AF). Klara is an AF in the service of humans, and she acts for her master, a young girl named Josie, by learning about her emotions or feelings. In a sense, one of the important questions raised is whether AI can understand human feelings, even empathize with human hearts, and act for that human being.

I would like to introduce a conversation that Josie’s mother had with Klara.

 

”  When her mother opened her mouth again, I knew immediately that this time she was talking to me.

　”I think it’s great to be emotionless sometimes. I envy you”.

　I thought about it for a while and said, “I think I have feelings too”.

　”The more I observe, the more I feel.”

Her mother suddenly burst out laughing, startling me, “Then maybe you shouldn’t try so hard to observe”. Then she said, “I’m sorry, that was rude. Yes, you have many emotions too,” she added. “

 

Consciousness: The Problem of Intelligence and Feelings

AI stands for “artificial intelligence,” but even if AI has intelligence, does it have feelings? In the first place, when intelligence is working, do we have consciousness? What happens to consciousness when feelings are at work? Does the fact that AF Klara has emotions mean that she is conscious and acts independently based on her own consciousness?

In order to solve this difficult problem, let us first consider the issue of consciousness. This is because there were Karl Marx and Friedrich Engels who found consciousness between human beings and nature, and believed that there is a law of production activity that includes not only material production but also spiritual production, which is production related to that consciousness. Behind the advocacy of Marx and others as “scientific” is the materialist view of history (Historischer Materialismus), which holds that human society, like nature, has objective laws, and therefore regards history as a developmental unfolding of the various forces of production. This is why the materialist view of history regards history as the developmental development of the forces of production. The social nature of the actual activities of human beings is regarded as the development of the forces of production and changes in the relations of production, and historical laws can be found there.

On this point, a Japanese philosopher Wataru Hiromatsu explains in “Marxist Philosophy: Its Perspectives and Horizons” (in “What Was Marx’s Fundamental Idea?”) that consciousness is not materialistically viewed by reducing it to the physiological and physical functions of the brain. He explains that consciousness is what appears in reality through language in the form of ecological relations in the form of “who is conscious of whom,” and that this is the very existence of human beings, and that he tried to find a law in this.

 

From “passion” to “profit”

This view, however, did not come about so easily. In the first place, we must realize that before modernization, there was a shift in values from “passion” to “interest,” which was related to the shift from “virtue” to “manners” and from “feelings” to “consciousness. “

　According to Albert Hirschman’s “The passions and the interests,” the dominant view in medieval Europe was that of St. Augustine, which condemned the lust for power, sex, and money, and regarded such feelings as pride, jealousy, greed, ambition, lust for power, and greed as “evil” and to be suppressed. In broad terms, this means that feelings should be suppressed. Roughly speaking, in a society where passions are regarded as evil, the desire for money, which forms one of the passions, is despised, and therefore, money-making is regarded as evil, and those who are dedicated to making money are looked at with contempt. This leads to a disregard for commerce and a disdain for merchants. The religious issues surrounding the collection of interest (levied interest) also led to more spiteful treatment of moneychangers. This is similar to the way in which commerce was despised in Confucianism. However, the creation of the concept of profit, while pitting sentiment against profit, at least creates a more tolerant view of seeking monetary gain.

In late 16th century Western Europe, “interest” meant interest, desire, and convenience, which included not only the material well-being of the individual but also an element of calculation about how to pursue one’s desires. As this concept came to include the broader notion of “profit” and was applied to nations, monarchs, lords, and individuals, the perception of a world guided by profit rather than sentiment spread. The concept of profit, which encompassed the previous feelings of greed, covetousness, and lust for money, was seen as an effective way to counteract and control feelings such as ambition, power, and sexual desire. Proverbs such as “profit never lies” and “profit rules the world” become household words.

Let us recall that in the first part of the second chapter of the first volume of Adam Smith’s Wealth of Nations, published in 1776, he points out that the division of labor is a consequence of “a certain disposition in human nature”, that is, “the disposition to trade, barter, or exchange one thing for another”. Let’s remember that this is a consequence of “the propensity to trade, barter, and exchange one thing for another. Let us remember, then, that each of us needs the help of his fellows at almost all times, but that it is useless to expect help only from their benevolence; rather, if we can stimulate their self-love to our advantage, and if we do for them what they ask of us, it is for their own good. If you can show your peers that it is in their own interest to do for you what you want them to do for you, Smith saw this as leading to a division of labor based on the “interests” of each individual. The emphasis here is on “sympathy,” which is assumed to be compatible with self-interest, as opposed to what Smith’s mentor Francis Hutchison called “moral sentiment” (the opposite of self-interest) (Kojin Karatani, The Structure of World History , 2014). This is the first time that I have seen such a thing. We are in a position to recognize the self-interest of others. This is where the importance of imagination increases, but what is important is that the Western tradition that morality must follow a transcendent cosmic order is now broken. Instead of judging right and wrong from a transcendental perspective such as divine providence, morality is created by human experience based on sympathy.

As we enter the age of “profit,” we can see the human race changing from an emotional state of “sentiment” to an age of “consciousness.

 

Neuroscience Today

However, it is now known that it is not consciousness that defines social phenomena, but the “facticity of action”. In this context, “action-factuality” refers to what we do without being aware of it. Rather than focusing on consciousness, it is this act-factual event that has created the history of human beings and nature, and it is impossible to find “science” in it. If the conscious “I” represents only a part of the “I” consisting of the unconscious, the body, etc., then it can be argued that even if the “I” makes a “social contract” in the form of an election, it does not reflect the entire “I” in the first place. It can be said that democracy is based on the delusion that the conscious “I” is absolute, and that “I” can make choices according to my free will.

According to Mark Solms’ The Hidden Spring, just published in February 2021, neuroscientists are unanimous in their view of consciousness: (1) consciousness is generated in the upper brainstem, (2) it is fundamentally affective, and (3) it is an extended form of homeostasis. In addition, Solms argues that consciousness is based on the the Free Energy Principle. This principle, proposed by Carl Friston, is a normative principle that states that “any self-organized system must minimize its (informational) free energy in order to remain in equilibrium within its environment.” This is the normative principle that “the conditions necessary for an adaptive system to persist against the natural tendency toward disorder.”

I’m going to go into a little more detail here, referring to “A Sexy New Theory of Consciousness Gets All Up in Your Feelings” [https://www.wired.com/story/sexy-new-theory-consciousness-will-give-you-feels/]. Let me explain it in more detail.

It is important to note that what (1) is saying is a wake-up call for what many people misunderstand. People may feel that consciousness is there to help them overcome their feelings. This is true, but it is a higher form of thought, and it is the cerebral cortex, the outermost part of the brain, that does this. Therefore, before and after Sigmund Freud, brain researchers have regarded the cerebral cortex as the seat of consciousness. However, both Solms and credible modern neuroscientists have rejected this as the “cortical fallacy.” For example, hydranencephalic kids, born without a cortex, laugh, cry, and move about the world in ways that can only be described as intentional. On the other hand, if the nucleus of the brain stem is destroyed, consciousness disappears and the child automatically goes into a coma. The brainstem, called the reticular activating system (RAS), which is the “hidden spring” of the book’s title, is the “spring.” Solms writes, “we are guided by a constant stream of feelings, flowing from a wellspring of intuition, arising from we know not where.”

 

Feeling is an inheritance

This assertion is consistent with the reality that humans move their values from “passions” to “interests” and from “feelings” to “consciousness,” yet suppress their feelings in higher consciousness based on their feelings as a nested structure.

On the other hand, applying this argument to “animals,” Solms writes, “presents us with an opportunity to think a little more deeply about animal suffering. For example, “sheep and cows and pigs (which feature so prominently on Western menus) are fellow mammals,” and like humans, they have the same basic emotions that we are, such as FEAR, PANIC/GRIEF and CARE. “Mammals possess a cortex, too, which means they are capable – all of them, to some degree – of consciously ‘remembering the future’ and feeling their way through its probabilities and likelihoods”.

Furthermore, Solms points out that feeling is a precious inheritance. It carries within it the wisdom of the ages: an inheritance that extends backwards over aeons to the beginning of life itself. He describes as follows:

“When homeostasis eventually gave rise to feelings, the crux of this new capacity was that it enabled us to know how we are doing within a biological scale of values. Feelings entrain predictions that are grounded in the accumulated experiences in situations of biological significance of literally all our ancestors. Feelings enable us to do what is best for us, even we do not know why we do so. I have asked you before to imagine what would happen if each of us had to learn afresh which foodstuffs contain high energy supplies and if we had to discover for ourselves what happens when we jump off cliffs.”

 

The Potential of AI

When you think about it this way, it is natural that Klara has feelings. If she didn’t have feelings, she wouldn’t have the desire to care for and be close to her human friends. But is it possible for AI to have feelings?

Solms makes a disturbing point. “Unless it is possible to design a computer that has feelings […] it will probably never be possible to design a computer with a mind […] The problem of the mind is therefore probably not a problem of intelligence.”

It is important to note that Hormes is not concerned with intelligence, but with consciousness. Note that it is consciousness, not intelligence, that Hormes is trying to artificially create. He says, ” it seems like a good strategy to try to engineer consciousness in its most elementary form.” His position is to engineer a self-evidencing system with no objective end in view other than remaining the means to that end. In other words, he says, “we will be trying to make a being that has no aim and purpose other than to carry on being.”

It is a life-like system that is ” an unconscious self-organizing system equipped with a Markov blanket (and therefore with sensory, active and internal states) which automatically models the world on the basis of sensory samples, ninimizing the effects of entropy upon its functional integrity by improving its generative model. However, this will be a lifelike system, but it will not be alive. Consciousness has evolved in living organisms, and the goal of this project is to show that it is possible to artificially create consciousness by reverse-engineering a functional organization (disassembling an existing product and analyzing its mechanisms, components, and technological elements).

What is interesting here is that “many current AI project and applications are commercially motivated” and clearly states that “to the extent that artificial consciousness might be used for financial gain, to that extent we are at risk of facilitating a new form of slavery.” This is the point. It is also important to note that “as soon as machines acquire consciousness (even the most rudimentary forms of raw feeling) then more general questions concerning their potential for suffering necessarily arise.” The pain and suffering of the machine become deeply implicated in the Western tradition of ethical theory of “consequentialism” (i.e. the notion that the consequences of one’s conduct are the ultimate basis for judgements about its rightness or wrongness), the idea that the consequences of one’s actions are the ultimate criterion for judging whether they are good or bad. It is a weighty statement that “by creating artificial feeling beings, we therefore enter into the jurisdiction of that of ethical calculus.”

But for now, humans will still have a long way to go to develop an artificial consciousness.

 

AI and Feelings

What is attracting attention today is feeling analysis, not the creation of feelings. For example, the Nihon Keizai Shimbun reported in an article titled “NEC targets emotional data economic zone beyond face recognition” (https://www.nikkei.com/article/DGXZQOFK282MG0Y1A120C2000000) that NEC will enter the unexplored field of emotional analysis services in 2021. NEC is aiming for a world in which it can use its emotion estimation technology to understand users’ feelings and provide optimal services in various situations throughout society.

In an article titled “How Pixar Uses Hyper-Colors to Hack Your Brain,” which appeared in Wired’s 2021 No. 5, it states the following.　

“The point is, ‘color’ means a lot of different things, depending on how you’re using it. And using it has been a defining trait of humanity since we all first started thinking. We see colors in the world, in nature, and we use what we see and learn to make newly colored things. It’s a hallmark of human activity, practice, and culture. We started by collecting objects with colors, turned to grinding rocks into powders and pastes and smearing them onto cave walls and on our bodies—and have arguably reached an evolutionary acme with the ability to control and create light with the precision and fidelity of a Pixar.”

It may be a blessing in disguise that the development of AI has not actually progressed much, but there is still a concern that if we do not pay attention to the issues discussed here, we may soon see the development of artificial consciousness that will promote slavery. However, if we do not pay attention to the issues discussed in this paper, there is a lingering concern that we may soon see the development of an artificial consciousness that promotes slavery.

This point is very much related to Klara’s end at the epilogue of “Klara and the Sun.” I would like to recommend this book as a summer vacation read with children.