Generation AI Part 2
The Other Environmental Crisis
PART 2: Redefining Success
New Indicators of AI Success
The major historical problem of AI development has been producing convincing and independent emulation of human thought and expression, and fitting them seamlessly together. Compared with conquering such domains as number crunching and winning at Checkers, Chess, and Go, the attempt to secure natural language—and its connection to thought and behaviour—has been relatively disastrous. A related problem has been the question of meaningful use and monetization of the range of AI inventions that have been created, but which do not meet the mark for realistic emulation or obvious augmentation of human intellectual capacities.
Language, the brain, and the environment form a nexus tightly woven by the warp and weft of input and output as a consequence of embodiment. Neuroscientific research suggests that for every event that is subjectively felt and experienced, there is a neural correlate, which guides future behaviour at the same time that it creates memory and sense of self. This relationship indicates that nothing in the human lifeworld can be considered independent of the biological substrate.
Language is a key interface linking human being, doing, and community in the past, present, and future. Likewise, machine learning (ML) and AI are human cultural activities that run on language. In this circular economy, natural language doubles as the operational premise and the target frontier for the continued development of AI. As a critical interface between the embodied self and the world, language is not a neutral medium, but is both a direct gateway to and an effect of biological neural networks. Similarly, machine learning algorithms and artificial neural networks enabling AI are not innocuous forms, but work to hack the human sensorium by discovering ready points of access. For researchers and developers across disciplines, text and image are the favoured modalities for entry into this space because the repository for both is vast and ever-renewed: they are the resources that the digital world runs on and are the preeminent tracers of humans in the wild in the form of data, especially online.
Data is understood differently in the field of AI as bits of human perception to be mined for the raw materials of thought, feeling, and action—mapped for points of entry into the underlying biological architecture.
While academic and popular concerns about data tend to focus on issues of privacy, copyright, and individual and collective ownership, data is understood differently in the field of AI, where it is recognized as bits of human perception to be mined for the raw materials of thought, feeling, and action, and mapped for points of entry into the underlying biological architecture. There is little difference between the work of academic and corporate laboratories in this regard, which are increasingly focussed on predictable AI access to the neurobiological substrates supporting what the average Western person understands as their own individual feelings, thoughts, and actions. The altruistic applications of such bio-hacking are somewhat murky, but the commercial applications are obvious: the more AI can be deployed for the capture and regularization of our speech, thought patterns, and behaviour in both predictive and prescriptive modes, the more easily we can be categorized, anticipated, and controlled by whomever has access to the know-how.
The more AI can be deployed for the capture and regularization of our speech, thought patterns, and behaviour, the more easily we can be categorized, anticipated, and controlled.
This trend signals a new concentration by developers on shifting the meaning of successful AI to something that more closely parallels and satisfies the ground-floor “effectiveness problems” in communication elucidated by Warren Weaver in his 1949 interpretative essay, complementary to the theoretical work of Claude E. Shannon.1 The idea is to recast success in AI development in strictly probabilistic human behavioural terms, such that proof of success is no longer achieving aspects of flawless or realistic human emulation, or even specific augmentation, but in increasing dramatically the degree of probability that human behaviour will be directly and consistently affected as desired by the encoder when interacting with an AI, regardless of whether that AI is crude, sophisticated, or in any way emulative, realistic, or desirable.
This readjustment to the picture of AI success, coupled with cradle-to-grave mass integration of consumers across technologies, translates into high marketability based on risk reduction and greater returns on investment for digital-based companies and their products, as well as other interested actors. Sentiment analysis and opinion mining are just two examples of how the Internet ecosystem is being combed for linguistic predictors of human thought and action for the purposes of control and profit generation. China’s emergent social credit system is a direct descendent of these generative intersections.
How we think, feel, and express ourselves in language is, in part, epigenetic, tied to a wide array of sociocultural forces that make up the tradition—literally, the thing which is handed down or across inter- and intragenerationally among community members—in which we are embedded. The capacities enabling this interaction are genetic, residing in the inner workings of the human brain and the body. Environment and environmental stimuli together have a hand in shaping neural processing and behavioural correlates, provided the foundations for such processing have been provided by nature and the relationships between them are culturally supported.
Hardwiring is next to irrelevant with the exploitation of existing, non-invasive AI interfaces that already connect with the human sensorium.
Digital industrialists are harnessing the twin forces of nature and nurture to shape a new story of the future for human being. Brain plasticity is being consciously exploited to create codependence between biological and artificial neural networks, resulting in widespread and rapid cultural change congenial to AI success and its profitability. Elon Musk’s Neuralink is symbolic of this trend. Breezily dismissed as “neuroscience theater” [sic], neuroscientific literature is, in fact, peppered with seed-grains that are germinating in the areas of development and growth claimed by the company.2 Critics and Musk alike should realise by now, however, that the hardwiring dimension is next to irrelevant given the deepening exploitation of existing, non-invasive AI interfaces that already connect—by nature and by design—with the human sensorium.
The same principles of biological evolution support AI development. Biological species and the new generation of algorithms share in the premise that, as Pedro Domingos puts it, “nature evolves for the nurture it gets.”3 As long as the foundation is there for the extraction and optimization of particular inputs, learning happens, nurturing what was made available in the first place by nature. This symbiosis affects both how inputs are processed in future and how related outputs may be generated. As for algorithms, so for people: what we are exposed to and trained on is essential to what we perceive and what we generate from those perceptions.
The inherent plasticity of the brain is a key feature linking nature with nurture. Change an aspect of nurture, from technology to family dynamics, and the brain will follow. Research on this relationship is constantly evolving, but one stable finding, as Nicholas Carr points out in The Shallows, is that “plastic does not mean elastic …. Our neural loops don’t snap back into their former state the way a rubber band does; they hold on to their changed state. And nothing says the new state has to be a desirable one.”4 In the end, where the ambivalence of brain plasticity meets with technology, “the mental skills we sacrifice may be as valuable, or even more valuable, than the ones we gain. When it comes to the quality of our thought, our neurons and synapses are entirely indifferent. The possibility of intellectual decay is inherent in the malleability of our brains.”5
Multigenerational, real-time, massive-scale human training on rush-to-market, speculative AI technologies has already had a deleterious effect on multiple levels of human life.
While change is an inevitability of human life, the question of the role of tradition in keeping intact valuable cognitive and behavioural assets, and in fostering the conditions for collaborative and cumulative cultural knowledge and innovation, must be considered in relation to technological change and even human evolution. AI is not an inevitable contributor to the positive in this equation. Furthermore, AI technologies do not merely retrieve, imitate, or enhance predicate technologies, whether digital or non-digital, ancient or modern. The current trajectory of their design and development carries the proposition of completely breaking the chain of connection between them by taking over and/or eroding the very neuronal processes involved at the generative level, initiating an exclusive cybernetic loop. For this reason, the current generation of AI will not necessarily lead to increased creativity, critical thinking skills, or linguistic aptitude, precisely because their design severs the heavy dependence on biological neural networks formed by their constant use and exercise on the production side in embodied human community.
Story is how humans deal with relational complexity, and if we stop fostering the natural basis for the capacity for story, rooted in embodied memory and experience, we erode our ability to create and express ourselves, as well as our ability to foster relationship. Foreclose on the creation and maintenance of the biological neural networks that support it, and we foreclose on the ability to produce new and meaningful human communication. AI, in its current paradigm, represents a next-level destabilization of the traditional teachings made available to us by our environments, which are ultimately material and physical in nature, and intrinsic to our survival and evolution.6
Multigenerational, real-time, massive-scale human training on rush-to-market, speculative AI technologies has already had a demonstrably deleterious effect on multiple levels of human life that must be faced. The relational aspects of these effects are explored in detail in Part 3. First, it is crucial to establish some understanding from observation of the scientific literature how the confluence of machine learning and neuroscience is contributing to the picture. Only then can we begin to own up to the responsibility for these effects in our lives, and hope to make change.
Machine Learning and Neural Intervention
If discoveries about the general correspondences between brain and behaviour have created opportunities for an historic shift in notions of AI success, what is known about the brain’s ability to process unpredictability or novelty and how this ability links up with environmental stimuli represents the next level in AI circumvention of personal agency, subjective experience, and hardwiring. An automatic and unavoidable responsiveness to “the new” or “surprise”—an inherent part of human survival mechanisms and foundational for memory and learning7—informs the major hypothesis governing how the current generation of AI is being designed, trained, and deployed to penetrate to the source of how we think, feel, and act. This responsiveness holds true across different forms of media and representation, including text and images. A balance between predictability and unpredictability is believed to underlie the naturalness of human thought and, by extension, linguistic expression and appropriateness of sentiment and behaviour. These two factors have a complex interrelationship that has been somewhat of a black box for AI.
In 2009, vision researchers Laurent Itti and Pierre Baldi posited surprise as “an essential concept for the study of the neural basis of behaviour.”8 Through their research into how people direct their gaze when exposed to a range of on-screen stimuli while watching television, they believed they could develop a more precise definition for and measurement of surprise that was not reliant on the obvious, everyday notion that whatever we do not expect or whatever is incongruent is “surprising.” What they found out is that the experience of surprise is context- and knowledge-driven, dictated more by our exposure to and beliefs about what is considered normal or what is normalizable in any given scenario over time. New data shifts our expectations as to what is novel for a context, changing in real-time whether our brains consider an event surprising or not, given what we have already experienced. This knowledge update is ongoing in the brain. Surprise is therefore shaped by a kind of conditioning of attention. They observed that, “an event is surprising when the distance between the posterior and prior distributions of beliefs over all models is large,” a matching fit with Bayesian theory.9 Their experiments showed that this distance leads to an experience of surprise not qualified solely by personal report, but also by measurement in the brain through the activation of specific neurons. A mathematical, quantifiable definition of surprise can therefore be made which “represents an easily computable shortcut towards events which deserve attention.”10 On these grounds, Itti and Baldi proposed a Bayesian theory of surprise that works “across spatiotemporal scales, sensory modalities and … data types and data sources.”11 At the time, they suggested that “computable surprise could guide the development of data mining and compression systems (allocating more resources to surprising regions of interest) to find surprising agents in crowds, surprising sentences in books or speeches, surprising medical symptoms, surprising odors in airport luggage racks, surprising documents on the world-wide-web, or to design surprising advertisements.”12 This list of proposed applications for computable surprise correctly anticipated the current quest for AI: to hook into the relationship between the expected and the unexpected in the sociocultural domain by targeting the underlying the activities in the brain that sort them out and translate them into meaningful personal experience. Grabbing attention, however, is only part of the equation. The other part is holding it in a real way. And the idea of what is real is up for interpretation—within limits.
Successful AI does not have to be good: humans just need to be that little bit worse.
AI research has suffered, historically and popularly, from the equation of “emulation” with “realism.” Change the definition of successful emulation, and a different measure of success is achieved that can have wide-ranging implications and applications. Over the last decade, work in a variety of disciplines and their sub-fields has demonstrated that, when it comes to the intersection of input and output and their effects at the neurobiological level, successful AI does not have to be good: humans just need to be that little bit worse. In its game-changing magnitude, this fact rivals the nineteenth century’s discovery of the method of discovery. Study after study—as well as real-world events—have demonstrated the important point that as long as an AI successfully matches with some aspect of our communicative expectations, especially the more subjective ones, we are ready to accept it as real.13 But what is considered real in subjective experience can be shown to be illusory or even irrelevant at the level of the neuron, suggesting that there is a kind of bottom-line stimulus threshold for conscious awareness and a subliminal one for the sensorium itself that may have knock-on behavioural effects. When asked how a Generative Adversarial Network (GAN) might be evaluated for success, field innovator Ian Goodfellow remarked that the criteria used should match the “downstream task”: for example, he says, “if you want to use the GAN to generate imagery that pleases humans, you can use human evaluation as the property at the end.”14 Case in point, if producing a realistic image or text is the designer’s objective, the AI succeeds when it reaches the benchmark of human acceptance of the image or text as “real.” As Goodfellow’s comment illustrates, AI has already stepped beyond the confines of realism to reach into the black box of neuronal and behavioural correlation where the bar for input responsiveness, as we will see, is actually rather low. The key takeaway here is that generating something that “pleases” is not necessarily the same as successfully generating what people may consciously perceive as real.
Specific neurons can be stimulated to react to conditions created by AI that are not realistic in the slightest, but which contain enough information to target and excite the individual neuron responsible for a specific task, as for example facial recognition. Using an approach they call XDREAM, a group of researchers at Harvard and Washington University worked with an evolving machine learning program (a GAN) to refine information that would produce a maximum neuronal effect in monkeys, who were presented with images that the AI learned to generate. The images were generated by peering into the monkeys’ brains to see which stimuli “maximized neuronal firing.” The targeted monkey-neuron tended to demonstrate an uncanny affinity to simulacra, such that the “evolved images,” confected by the AI from a range of stimuli that were not necessarily semantically related, “activated neurons more than large numbers of natural images.” Further, it was found that as more images were generated and introduced by the algorithm, their “similarity to evolved images predict[ed the] response of neurons to novel images.” This past-future hybrid manipulation of response at the source has the additional effect of helping both the machine and the researchers learn the hidden “stimulus properties” that may be unknown or unanticipated, but which are essential to provoke maximal stimuli effect and engagement at the neuronal level.15 In other words, with the help of ML, it is increasingly possible to bypass the involvement of complex, higher-level processing and subjective aspects of thinking, and hook directly into the first level of processing in the brain via primary exposure to whatever may excite a bunch of neurons. At this deep-brain level, seeing is, in fact, believing.
By this measure, the same indicators of AI success are present even in less sophisticated laboratory circumstances. In order to make a chatbot persona more believable in its interpersonal communications, a recent study on the neurobehavioural effects of social-media-induced personal rejection built unpredictability into the design of the fake chatbot with which research subjects were to interact. The design worked: novel or unexpected responses kept within topical parameters were key to convincing the subjects that they were chatting with real people and, therefore, victims of real social rejection. They exhibited the anticipated neuronal and behavioural effects accordingly, meaning that qualitative personal reports of feelings of rejection could be quantitatively measured in uncontrollable brain responses.16
Machine learning is a tool of data and stimuli recombination for delivery to the human sensorium at orders of magnitude beyond what a human alone can achieve in scale and scope.
Machine learning, whether represented by cutting-edge GANs or other models, can achieve meaningful results that demonstrate simultaneously both the revisioned success of AI and the precariousness of the neural structures contributing to its success. Like all forms of human endeavour, machine learning also has a mirror-like quality. From the language-based Google x Douglas Coupland slogans that aesthetically amuse audiences to the bizarre visual confections that make lab monkeys go ape, machine learning uses techniques of recombination analogous to what happens at the neurobiological level for humans.17 The recombining of data or stimuli is essential to the internal processes that contribute to who, how, why, and what we think we are. And as machine learning reverses its analogical status—reaching back into these neurobiological modes of recombination by means of a generative and recursive automatic loop of tethered production and response—the potential for direct neural intervention by AI on human thinking, feeling, and acting exponentially increases. The danger lurking in this potential is magnified by the fact that machine learning is a tool of data and stimuli recombination for delivery to the human sensorium at orders of magnitude beyond what a human alone can achieve, either in terms of scale or scope.
The potential for neural intervention or interference by AI extends to the roots of memory itself. The sense of the spatiotemporal that humans share—the idea that there are such things as time and space, the feeling that we are integrated with or attached to them in some way, and that they are real—is an effect of the recombination of sensory inputs at the neuronal level. An essential contributor to our sense of self, identity, and meaning, it is action and memory dependent. The creation and maintenance of an individual’s “cognitive map”—made up of space and time building-blocks hewn from direct, embodied experience—is the main responsibility of the hippocampus. This region of the brain uses neural networks to sort out spatial and temporal experiences, where they undergo a first processing and become part of short-term or “working” memory; by a secondary process still not fully understood, they may port over from there to become lodged permanently in the cortex as long-term memories.18 The work of the spatiotemporal network in the hippocampus is essential to learning, memory, and the encoding of semantic awareness. Disruptions to the network are deeply problematic for all of these features; this much is known from the devastating effects of organic diseases like stroke and Alzheimer’s. What would happen to these features if the recombinatory processes relating to the spatiotemporal network—and their effect on things like memory, meaning, and movement—is accessed and altered by external means, like AI?
Researchers are riding high on synergistic hopefulness that advancements in neuroscience, AI, and virtual reality (VR) will unlock the secrets of spatiotemporal cognition and, thereby, memory.19 While video games, VR, and AI are not yet up to the sophistication of mammalian spatiotemporal systems, even the most crude VR engages and promotes real-world-like activity throughout the spatiotemporal processing network. Measuring and mapping human responses to VR experiences is opening up new horizons in understanding how spatiotemporal systems and memory work together. The gap is closing on real-world versus simulated experiences as more information about their interconnection emerges. As with the monkey-brain XDREAM experiment, AIs could be calibrated to target specific neurons in a network in a VR context.20 Simon Fraser University’s iSpace Lab is working on a number of projects to solve problems that limit the “reality” quotient of contemporary VR technologies, such as motion-sickness, disorientation, and—crucially—interruptions to the automatic, cognitive operation of spatial updating, which “is impaired if self-motions are only visually simulated in [VR] and people are not physically walking.”21 “Leaning-based” technologies, which require a user literally to “lean in” to simulated environments, seem to be on-track as a solution, implying that there is something about embodiment—the upper body and the head in particular—that is inseparable from meaningful spatial orientation.22 Preliminary investigations into short- and long-term interaction with GPS systems and their interfaces at McGill University indicates some measurable negative consequences for spatial memory and navigation strategies;23 collaborative research from McGill University and the University of Montreal shows that 3D-gaming environments may mitigate such effects.24
Early and limited research into the effect of video games on the hippocampus suggest that whether a video game is deleterious to grey matter, and ultimately, to memory systems, is dependent on the nature of the game being played and the spatial strategies they necessitate or encourage: overall, 3D-platforms increase grey matter, while other forms of action video games decrease it.25 The researchers observe that “as people continue to repeatedly engage in behaviour that relies on one memory system at the expense of the other, the other memory system is left underused and experiences grey matter loss.”26 They also point out that while gaming platforms have been used successfully to limit the loss of grey matter in subjects with a wide variety of memory disorders, until this study, none had shown an increase.27 Another study suggests that “frequent action video game playing is associated with higher functional activity in the reward circuit and lower functional activity within the hippocampus,” such that an analogy is made between these players and “addicted participants.”28 The overarching recommendation from this field of research is that there ought to be extreme care taken in the development and deployment of video games and related cognitive training methods, so that deleterious effects are not introduced into other aspects of memory and spatiotemporal orientation while trying to halt or fix others.
Disruptions to perceptual patterns and pathways could dislodge or displace memory, eroding our ability to draw down meaning from our experiences.
While the exact interaction between short- and long-term memory is not fully understood, it is now known that specific groups of cells are responsible for the separate encoding of spatial information and temporal information at the level of the hippocampus. Space information is encoded by so-called CA1, “place cells,” and time information is encoded by so-called CA2, “time cells,” which act like interactive coordinates on the map. New research from the Tonegawa Lab at MIT’s Picower Institute for Learning and Memory demonstrates that the way these cells interact in relation to memory is specific to how and why memory is being engaged in the first place. Laboratory research on mice showed that these interactions are delicate and susceptible to interference. Using a viral optogenetic technique, researchers were able to inhibit short-term memory, disrupting the ability of the animals to observe and establish patterns in their activities. By exploiting a mnemonic time-gap in CA1 and CA2 processing, the researchers interrupted and destabilized the critical neural network communication between place cells and time cells at the very foundation of memory-making, altering active decision-making in the mice.29 If the implications of current research into time and space cells of the hippocampus are correct, the ability to disrupt the patterns and dynamics of interaction between these cells in humans could have devastating consequences. Disruptions to perceptual patterns and pathways could dislodge or displace memory, with the effect of resequencing and rearranging contexts, eroding our ability to locate ourselves in time and space, and, ultimately, to perceive, build on, and draw down meaning from our experiences to inform decision-making and behaviour. And with the ability of ML to create these things for us, our grasp on “reality” begins to slide.
Until such time as similarly direct interventions in spatiotemporal processing are developed for humans, language and image remain the two most potent streams of intervention. As the brain-based research explored above indicates, biological neural networks engage in complex recombinatory processes to make meaning out of sensory inputs from all sources and modalities. The properties of the interface are all important in this scenario: in a landscape of crude techniques, the most effective performers so far are social media platforms, run on a labyrinthine interweaving of AI systems and frameworks that align with what neuroscientists know—and conjecture—about the brain’s responses to external stimuli across modalities. Text and image are the bread-and-butter of social media, with language being the crucial interface uniting all facets. In 2017, Sean Parker, the former president of Facebook, acknowledged that designers of the platform were “exploiting a vulnerability in human psychology” in targeting the desire for “social validation,” which, combined with a “dopamine hit,” effected a changed “relationship with society” among users; he further declared that Facebook use “probably interferes with productivity in weird ways,” before rhetorically appealing to God for the answer as to what it might be “doing to our children’s brains.”30 Parker’s comments reveal the growing awareness that social media platforms represent a mechanism for disrupting the processing crucial for memory and its corollaries.
Social media platforms represent a concentrated form of processing interference that is endemic to the digital experience itself, particularly in the form of the Internet. Elizabeth March and Suparna Rajaram suggest that potentially maladaptive shifts in “autobiographical memory” and growth in “information appropriation”—shading into an inability to discern one’s own thoughts and experiences from another’s—are the result of selective memory curation, fabrication, and distortion rooted in the deep cognitive effects of the “digital expansion of the mind” through the online experience.31 It is not just that the Internet is informationally overwhelming or that it is full of misinformation: its informational content is tied to a particular systemic architecture that exploits computable Bayesian surprise, exponential content creation, habitual use, and cognitive thresholds for speed, novelty, social connection, and emotion in order to capture and contain minds.32
Form, content, and engagement represent a complex trifecta in relation to how, what, and why things are remembered. While a study on co-witness memory distortion suggests that online and face-to-face communication are equivalent in their corrupting effects on memories of a witnessed event,33 a study on the impact of Instagram engagement on memory adds that it is the way in which engagement happens that makes the difference. The researchers found that memory formation for images on Instagram increased when subjects interacted with them by commenting and reacting with emojis. The obvious inference here is that more interaction—in the specific ways that Instagram affords—seems to result in better user memory for the specific image items engaged.34 As though an aside, the researchers temper their findings with the telling caution that “the impact of engaging with technology on memory may depend on the user experience and whether the experience interferes with the processing of the to-be-encoded event in the first place,” and demur that “understanding the cognitive implications of engaging with the Internet and social media may require a better understanding of how usage patterns change with time and with increased use.”35 The overall implication, reminiscent of the video games research, is that the mode and depth of online experience and engagement appears to be a more powerful arbiter of processing outcomes than the nature of the content itself. The received ideology of “personal agency” within online environments, a foundational principle in this mode of research, is not protective against the effects of endemic AI.
The received ideology of “personal agency” within online environments is not protective against the effects of endemic AI.
The Social Dimension
Studies on user experience of social media, like Facebook, Instagram, or Twitter, tend to presume a context of free will and choice online—a vestige of the democratic agency and empowerment paradigm imputed to the early Internet—and to ignore or minimize the role of AI in how these experiences may be shaped from the top down on the platforms themselves. MIT research on the spread of true and false news on Twitter illustrates that arrival at a crossroads in this regard had already been reached by 2018. Pitting people against bots, the researchers found that, overwhelmingly, false rumours are “diffused significantly farther, faster, deeper, and more broadly than the truth in all categories of information” by actual people.36 In fact, “falsehoods were 70% more likely to be retweeted than the truth.”37 When bots were removed from the statistical picture, they were demonstrated to be comparatively ambivalent, spreading true and false rumours in equal amounts; nevertheless, they were significant accelerants in the overall spread of both types.38 Human vulnerability to novelty was identified as the most significant factor in the successful dissemination of false information. Patterns of user engagement with both false and true rumours suggested that “false rumours inspired replies expressing greater surprise, corroborating the novelty hypothesis,” whereas the “truth inspired replies that expressed greater sadness, anticipation, joy, and trust”; false rumours were also connected with “greater disgust.”39 The study did not make a distinction between origin and provenance in respect of the difference between tweet diffusion and authorship,40 nor did it take into account the systemic operations of the AI-powered platform itself, aside from the acknowledgement of bot entities. An important inference that can be made from the results of this study is that, whether prompted by a bot or by another human, English-speaking Twitter users are motivated to seek out and participate in communication events that promote curiosity-based interactions endemic to the platform. If false rumour spread is indeed based on human behavioural probability factors that are, as the researchers suggest, connected to Itti and Baldi’s notion of computable Bayesian surprise, the “bot or not” provenance question becomes somewhat specious: that posts are novel and elicit responses that correlate to quantifiable surprise is enough to prompt engagement at depth and at volume per the affordances of the environment in which they are embedded. After all, environments like Twitter are designed to be attractive to humans, with bots—whether working at the front end or the back end—a factor of the operational landscape. At any rate, even in 2018, a scenario must be imagined in which all content may be written at origin and/or posted by bots, optimized to deliver the right stimulus within the context of a given platform.41
Observed susceptibility to content, however, has led researchers to suggest enlightenment messaging as an antidote to phenomena like fake news, false rumours, or other destabilizing text and image information disseminated online. The MIT false news researchers reasoned, for example, that “misinformation containment policies should also emphasize behavioral interventions, like labeling and incentives to dissuade the spread of misinformation, rather than focusing exclusively on curtailing bots.”42 Further advice from experts includes using personal response as a red-flag for detecting misinformation: if the news story fits with similar emotion tags to those identified in the false news research, like disgust, anger, fear, and surprise, then further investigation is warranted, which still relies on successfully navigating problematic features relating to quality—like source profile and informational tampering—and quantity.43 How this advice squares with what is known about the psychosocial and emotional phenomena supporting the well-documented “backfire effect” is unclear.44 Another counter-approach is to bypass human emotional diagnostics, ethics, intuition, and sleuthing altogether and develop better AI and machine-learning models for autodetection of fake news and for prediction of true and false content concepts and categories without costly human intervention, like that which is required for retrospective fact-checking or supervised machine instruction.45 Companies like Facebook, Twitter, and Google have combined content surveillance with warnings to moderate false news spread, and have had some self-reported success in reducing click-throughs to identified content.46 With their focus on semantic content, however, these strategies tend to exonerate the structures and operational elements present in the very architecture of online platforms that are known to leverage emotional response and to distort memory, knowledge, and identity.
Other research indicates that the presumed effectiveness of warning strategies is questionable, especially as AI gets better at producing outputs across modalities that align with the goals of encoders and the sensory apparatus of targeted decoders. Fake video, audio, and image are increasingly easier to produce and harder to detect. In addition, platform proliferation, automation, convergence, and interface coherence and cohesiveness work against self-initiated behavioural modifications triggered by semantic content or other cues, like suspicious image or audio content. In fact, the environmental setting and form of content often trumps its meaning and other markers of veracity. As Lisa K. Fazio et al. point out, even “knowledge does not protect against illusory truth.” She and her colleagues observe that there is a “strong, automatic tendency,” rather, “to rely on fluency” across modalities, from the look of a text to the sound of a voice; other studies have shown that “claims embedded in meaningful contexts (fluent)” have “higher truth ratings than those in irrelevant contexts (disfluent).”47 The illusory truth effect is further enhanced by repetition and ease of comprehension. But what happens when a statement satisfies fluency conditions yet contradicts prior knowledge? Does the generally presumed “strong negative relationship between knowledge and illusory truth” hold up?48 The researchers discovered that, “in the face of fluent processing experiences,” people who would otherwise be able to discern fact from fiction vacate their own knowledge, failing to “to rely on stored knowledge”; in other words, they demonstrate “knowledge neglect.”49 Over the course of their study, they demonstrated that “people retrieve their knowledge only if fluency is absent,” preferring instead to “[rely] on fluency as a proximal cue” for truth.50 They explain that “people [may be] especially susceptible to external influences like fluency” because our own semantic knowledge base is rarely related to its sources in our own minds. Moreover, the way we engage with part and whole relations, or metonymy, is a complicating factor when separating truth from fact: “we tend to notice errors that are less semantically related to the truth,” because we tend to rely on the principle of “partial match” when evaluating information.51 Their findings do not contradict evidence from earlier studies that domain expertise can, bizarrely, enhance the illusory truth effect.
Studies on humour provide a low-tech parallel, revealing that aspects of form and surprise combine to enhance memory, preserve content, and grab attention. When combined with incongruity, the constraints imposed by the form of a joke enhance its recall, thereby enhancing its potential spread and longevity.52 In the social media context, an item’s ease of spread is likewise ensured by the constraints of form along functional pathways that provide instant copying or sharing of multimodal content to entire networks; these facilities provide a fluency track that generally maintains the integrity of the shared item. Structural elements, including headlines, taglines, or hashtags,53 may work the same way as other short word devices, like puns, poetry, or aphorisms, to capture attention and keep rumours intact, impactful, and engaging, regardless of their semantic content. The question of how a medium itself can work as a mechanism for the extraction of specific mental states—through the combination of interface fluency and the manipulation of neural correlates at the root of subjective experience—may parallel the anticipation of humour and prediction of future humorous events given a primed and fluid circumstance, as, for example, when watching a comedy movie.54 The inference can be made that digital platforms both constitute and rely on environmental constraints that accentuate the role of surprise and incongruity in context, while providing fluency of experience, grabbing attention, encouraging belief, and enabling engagement.55
Contexts superpowered by AI can destabilize memory, discernment, and behaviour as we privilege their fluency and structural grooviness.
One of the more important conclusions reached by Itti and Baldi in 2009 is that their surprise theory “is entirely generalizable and readily applicable to the analysis of auditory, olfactory, gustatory, or somatosensory data,” and that “detecting surprise in neural spike trains does not require semantic understanding of the data [they carry].”56 These observations reinforce the importance of the form and structure of stimuli vis a vis the human sensorium. They also emphasize the potential of AI-powered contexts to calibrate conditions for maximum audience capture, retention, and behavioural modification, whatever the content. Added to the complexity of the novelty or surprise factor working on brains, contexts superpowered by AI, like social media platforms, which work to boost and highlight stimulus responses, can destabilize memory, discernment, and behaviour as we privilege their fluency—subconsciously attracted to the structural grooviness of functional cohesion and coherence in the experiences they offer. As we move deeper and deeper into these experiences, we alter our brains, bodies, and behaviour to complement an ambivalent range of selected human elements, that have been coded, processed, and reflected back to us by the machine.
Citation:
Reid, Jennifer. “Generation AI Part 2: The Other Environmental Crisis.” Winnsox, vol. 3 (2022).
ISSN 2563-2221
