Given that this is our cultural state, one of the most urgent questions we face as a society is the identification of practices which are adequate for intervening in its development. In this intimate machine-culture constellation, how can we decide what we ought to do? How should we as a society spend our resources? What interventions are possible, what are not possible, what are advisable, what had we better stay away from?
One candidate for a practice that could answer these questions is computer science, which has developed extensive practices for constructing computational machinery. Computer scientists understand well how machines can be built, what kinds of technology are possible, and what kinds could be possible if more effort were invested. They are trained to identify shortcomings in technology and to propose solutions to those shortcomings. In practice, they tend to have an intimate familiarity with the inner workings of machines of a sort which is difficult for non-technical-workers to develop.
At the same time, computer science suffers a disciplinary amnesia to the machine's cultural context. Computer scientists are trained to focus on machinery, i.e. what can be done, but not on whether it should be done or how it will be applied. The computer remains a black box, within which computer scientists work and outside of whose impermeable boundary the rest of culture and society goes about its business. Questions of sociocultural implication are not answerable within this framework 1.
Another candidate for a practice to address machine culture is the cultural studies of science, which has developed extensive practices for analyzing technology in a cultural context. Cultural critics know how technology is taken up in and influences broader culture, as well as how cultural background --- such as unconsciously held metaphors and philosophies --- can encourage the development of certain forms of technology at the expense of others. Cultural critics also have access to tools for analyzing the political and material economies which enable particular forms of technologies and discourage others; they know the cultural pressure points.
At the same time, cultural studies is at a disadvantage in proposing new interventions in machine culture because, as Richard Doyle puts it, it has historically been a consumer of practices rather than a producer of ones. That is to say, cultural studies has the tendency to critique, rather than to generate new practices which respond to critique. As a result, it often lacks agency in the critiqued practices, being marginalized as a kind of disciplinary Cassandra. In addition, because cultural studies tends not to engage in the practices it criticizes, it frequently lacks the intimate (though not necessarily self-reflective) knowledge of those engaging in those practices, and may at times misunderstand them.
I believe the technical practices of computer science and engineering and the critical practices of cultural studies and the humanities both provide important ingredients to intervene in machine culture, but neither is sufficient alone. In order to be able to address contemporary human experience, we need science and the humanities to be combined into hybrid forms which can address the machinic and the human simultaneously. Squeezed in between the disciplines, we can already see these forms developing.
These hybrid practices are an essential (and perhaps inevitable) response to machine culture. At the same time, the approaches outlined so far share one disadvantage: an underlying disciplinary split. Computation is seen from the outside, to be observed, analyzed, used, learned from. The development of computational tools, however, remains largely in the domain of computer scientists, to be informed by humanist wishes, to be intrigued by humanist appropriations, to be confused by humanist critique, but to be done using time-honored engineering methodologies.
But in a world where machinery is woven in to the fabric of our daily lives, it is, while useful, not enough to approach computation at an arm's length, to make it the object or pre-given tool of the humanities. The humanities must not only observe, use, and critique computation, but also ingest it. Computing itself must become a humanist discipline.
What this does not mean is the simple use of humanist results in order to optimize computer programming, the development of analytic Shakespeare generators, the reduction of the humanities to what can be output by a computer program. Instead, humanist forms of computing can be a set of practices incorporating a critical, self-reflexive viewpoint into technical work, using the research strategies and values of the humanities, embodying those values and traditions in changing technologies that in turn change human lives. They are oriented towards and respect the full complexity of human experience in the world, rather than reducing that experience to simple rules in the traditions of the natural sciences. They carry a healthy scepticism about the origin and value of computational concepts and tools, but rather than reject them they reorient them. They realize that the term computer science is a historical term, originally used to establish the legitimacy of computing as a coherent and respectable discipline, now artificially limiting the full breadth of possible computational research. This is a cultural informatics. This is what a growing and hybrid group of artists, researchers, and critics already do.
Humanists have called for such successor science projects for years. The research tradition of a humanist computation, though somewhat buried under the overwhelming mass of traditional computer science, already exists and is gaining strength. It is generally unnoticed because humanistically-informed computing is still computing. It is specific, oriented towards a mostly scientific academic subculture, flying below the radar screens of the humanities.
In this paper, I will also work specifically, looking at the confluence
of cultural studies and Artificial Intelligence (AI). I will focus particularly
on the subfield of autonomous agents, artificial creatures that
`live' in physical or virtual environments, capable of engaging in complex
action without human control. While giving an overview of research in this
field, I will explain how issues of subjectivity unconsciously arise, suggesting
an entrypoint for cultural studies. I will lay out how cultural studies
and agent research can be and are being synthesized, and look at the mostly
unknown research tradition that already exists in this area. I will
then connect the critical practices within AI to those in computer science
and general, as well as complementary approaches to cultural informatics
emerging from within the arts.
This AI dream of mechanical creatures that are, in some sense, alive, can seem bizarre at first glance. It is therefore important to note that this is not an idea that is new in AI, but, as Simon Penny notes, the continuation of a tradition of anthropomorphization that extends back thousands of years 4. In this sense, the AI dream is similar to the `writing dream' of characters that ring true, to the `painting dream' of images that seem to step out of the canvas, to the fantasies of children that their teddy bears are alive, and to many other Pygmalionesque dreams of human creations that begin to lead their own lives.
But there is certainly a sense in which AI brings a new twist to these old traditions. AI as a cultural drive needs to be seen in the context of post-industrial life, in which we are constantly surrounded by, interfaced with, and defined through machines. At its worst, AI adds a layer of seductive familiarity to that machinery, sucking us into a mythology of user-friendliness and humanity while the same drives of efficiency, predictability, quantifiability, and control lurk just beneath our perception.
But at its best, AI invokes a hope that is recognizable to humanists --- that is invoked, in fact, by Donna Haraway in her ``Cyborg Manifesto'' 5. This is the hope that, now that we are seemingly inescapably surrounded by technology, this technology can itself become hybridized and develop a human face. This version of the AI dream is not about the mechanistic and optimized reproduction of living creatures, but about the becoming-living of machines. The hope is that rather than forcing humans to interface with machines, those machines may learn to interface with us, to present themselves in such a way that they do not drain us of our humanity, but instead themselves become humanized.
In the 1950's and early 1960's, this dream for AI, for good and for bad, was embodied in cybernetics. W. Grey Walter, for example, built small robots with rudimentary ``agenty'' behaviors 6. He called his robots `turtles;' they would roam around their environment, seeking light, finding food, and avoiding running into things. Later models could do some rudimentary associative learning.
But as cybernetics fell out of fashion, AI research began to focus more on the cognitive abilities an artificial agent might need to have higher-level intelligence, and less on building small, complete (if not so smart) robots. At least partially because the task of reproducing a complete creature has been so daunting, AI spent quite a few years focused on building individual intelligent capabilities, such as machine learning, speech recognition, story generation, and computer vision. The hope was that, once these capabilities were generated, they could be combined into a complete agent; the actual construction of these agents was often indefinitely deferred.
More recently, however, the field of autonomous agents has been enjoying a renaissance. The area of autonomous agents focuses on the development of programs that more closely approach representations of a complete person or creature. These agents are programs which engage in complex activity without the intervention of another program or person. Agents may be, for example, scientific simulations of living creatures, characters in an interactive story, robots who can independently explore their environment, or virtual `tour guides' that accompany users on their travels on the World Wide Web 7. From the early debacles of Microsoft Bob through the alternately loved and hated Microsoft Office paper clip to the commercial hits of Tamagotchi and Furby's, agents are making their way into the average Netizen's home and consciousness.
While these applications vary wildly, they share the idea that the program that underlies them is like a living creature in some important ways. Often these ways include being able to perceive and act on their (perhaps virtual) environment; being autonomous means they can make decisions about what to do based on what is happening around them and without necessarily consulting a human for help. Agents are also often imputed with rationality, which is defined as setting goals for themselves and achieving them reasonably consistently in a complex and perhaps hostile environment.
Here, I will take agenthood broadly to be a sometimes-useful way to frame inquiry into the technology we create. Specifically, agenthood is a metaphor we apply to computational entities we build when we wish to think of them in ways similar to the ways we understand living creatures. Calling a program an agent means the program's designer or the people who use it find it helpful or important or attractive to funders to think of the program as an independent and semi-intelligent coherent being. For example, when we think of our programs as agents we focus our design attention on `agenty' attributes we would like the program to have: the program may be self-contained; it may be situated in a specific, local environment; it may engage in `social' interactions with other programs or people 8. When a program is presented to its user as an agent, we are encouraging the user to think of it not as a complex human-created mechanism but as a user-friendly, intelligent creature. If `actually' some kind of tool, the creature is portrayed as fulfilling its tool-y functions by being willing to do the user's bidding 9 . Using the metaphor `agent' for these applications lets us apply ideas about what living agents such as dogs, beetles, or bus drivers are like to the design and use of artificially-created programs.
Philip Agre and David Chapmann distinguish agents using `plans-as-programs' from agents using `plans-as-communication.' This is a distinction based on the relative importance of internally-determined planned-out activity versus a more improvised, moment-by-moment immersion in environmental circumstances. Agents that use plans as programs are heavily invested in their internal representation of action; they engage in abstract, hierarchical planning of activity before engaging in it (often including formal proofs that the plan will fulfill the goal the agent is given). Agents that use plans as communication see plans as a convenience but not a necessity. They are designed to take advantage of an action loop with respect to their environment and may only refer to plans as ways to structure common activities 12.
Another common distinction is between situated and cognitive agents. Situated agents are thought of as embedded within an environment, and hence highly influenced by their situation and physical make-up. Cognitive agents, on the other hand, engage in most of their activity at an abstract level and without reference to their concrete situation.
Each of these distinctions is not independent of the others. When looking at such classification attempts at a whole, a distinct theme emerges. AI research in general can be understood as involving two major trends in thinking: a main stream often termed classical AI (also known as Good Old-Fashioned AI, cognitivistic AI, symbolic cognition, top-down AI, knowledge-based AI, etc.) and an oppositional stream we can term alternative AI (also known as new AI, nouvelle AI, ALife, behavior-based AI, reactive planning, situated action, bottom-up AI, etc.) 13. Not every AI system neatly falls into one or the other category --- in fact, few can be said to be pure, unadulterated representatives of one or other. But each stream represents a general trend of thinking about agents that a significant number of systems share.
For AI researchers, the term classical AI refers to a class of representational, disembodied, cognitive agents, based on a model that proposes, for example, that agents are or should be fully rational and that physical bodies are not fundamentally pertinent to intelligence. The more extreme instances of this type of agent had their heyday in the 60's and 70's, under a heady aura of enthusiasm that the paradigms of logic and problem-solving might quickly lead to true AI. One of the earliest examples of this branch of AI is Allen Newell and Herbert Simon's GPS, the somewhat optimistically titled ``general problem solver.'' This program proceeds logically and systematically from the statement of a mathematical-style puzzle to its solution 14. Arthur Samuel's checker player, one of the first programs that learns, attempts to imitate intelligent game-playing by learning a polynomial function to map aspects of the current board state to the best possible next move 15. Terry Winograd's SHRDLU maintains a simple representation of blocks lying on a table, and uses a relatively straightforward algorithm to accept simple natural language commands to move the virtual blocks 16. While the creators of these programs often had more subtle understandings of the nature of intelligence, the programs themselves reflect a hope that simple, logical rules might underlie all intelligent behavior, and that if we could discover those rules we might soon achieve the goal of having intelligent machinery.
But the classical model, while allowing programs to succeed in many artificial domains which humans find difficult, such as chess, unexpectedly failed to produce many behaviors humans find easy, such as vision, navigation, and routine behavior. The recognition of these failures has led to a number of responses in the 80's and 90's. Some researchers --- most notably Winograd, who wrote an influential book with Fernando Flores on the subject 17--- have decided that the intellectual heritage of AI is so bankrupt they have no choice but to leave the field. By far the majority of AI researchers have remained in a tradition that continues to inherit its major research framework from classical AI, while expanding its focus to try to incorporate traditionally neglected problems (we might call this `neo-classical AI'). A smaller but noisy group has split from classical AI, claiming that the idea of agents that classical AI tries to promote is fundamentally wrong-headed.
These researchers, who we will here call alternative AI, generally believe that the vision of disembodied, problem-solving minds that explicitly or implicitly underlies classical AI research is misguided. Alternative AI focuses instead on a vision of agents as most fundamentally nonrepresentational, reactive, and situated. Alternative AI, as a rubric, states that agents are situated within an environment, that their self-knowledge is severely limited, and that their bodies are an important part of their cognition.
It is in these aspects of AI technology --- ones that are influenced by and in turn influence the more philosophical perspectives of AI researchers --- that we can uncover, not just the technology of agents, but also theories of agenthood. Two levels of thought are intertwined in both these approaches to AI: (1) the level of day-to-day technical experience, what works and what doesn't work, which architectures can be built and which can't; and (2) the level of background philosophy --- both held from the start and slowly and mostly unconsciously imbibed within the developing technical traditions --- which underlies the way in which the whole complex and undefined conundrum of recreating life in the computer is understood. Running through and along with the technical arguments are more philosophical arguments about what human life is or should be like, how we can come to understand it, what it means to be meaningfully alive.
The argument is straightforward: if agents are metaphors that are used to design programs that are in some sense like people, then the way we build agents will depend on and in turn reveal a great deal about what we think people are like. This means AI includes not only conflicting theories of technology but also, implicitly, conflicting theories of subjectivity. Classical AI technology is based on a model of subjectivity as essentially representational, rational, and disembodied. Alternative AI technology presupposes that it is essentially reactive, situated, and embodied.
These two categories can be clearly seen within AI research. Within that research community, they are generally seen as arising from certain tensions in technical practice itself. But these categories should be familiar to cultural theorists from a quite different context; they directly correspond to rational (or Enlightenment) and schizophrenic (or postmodern) subjectivity 18.
Rational subjectivity is based on the Cartesian focus on logical thought: the mind is seen as separated from the body, it is or should be fundamentally rational, and cognition divorced from emotion is the important part of experience. This model has overarching similarities with, for instance, Allen Newell's theory of Soar, which describes an architecture for agents that grow in knowledge through inner rational argumentation 19. Most models built under Soar are focused on how this argumentation should take place, leaving out issues of perception and emotion (though there are certainly exceptions 20).
The development of the notion of schizophrenic subjectivity is based on perceived inadequacies in the rational model, and is influenced by but by no means identical to the psychiatric notion of schizophrenia. While rational subjectivity presupposes that people are fundamentally or optimally independent rational agents with only tenuous links to their physicality, schizophrenic subjectivity sees people as fundamentally social, emotional, and bodily. It considers people to be immersed in and to some extent defined by their situation, the mind and the body to be inescapably interlinked, and the experience of being a person to consist of a number of conflicting drives that work with and against each other to generate behavior. In AI, this form of subjectivity is reflected in Brooks's subsumption architecture, in which an agent's behavior emerges from the conflicting demands of a number of loosely coupled internal systems, each of which attempts to control certain aspects of the agent's body based almost entirely on external perception rather than on internal cogitation 21.
Each class of agent architectures closely parallels a kind of subjectivity. Just as alternative AI has arisen in an attempt to address flaws in classical AI, the concept of schizophrenic subjectivity has arisen in response to perceived flaws in the rational model's ability to address the structure of contemporary experience. Each style of agent architecture shows a striking similarity to a historical model of subjectivity that cultural theorists have identified.
This close relationship between a technical debate in a subfield of computer science and philosophical trends in Western culture as a whole generally comes as a surprise to technical workers. But the connection is obvious to cultural theorists. AI researchers are also human beings, and as such inhabit and are informed by the broader society that cultural theorists study. From this point of view, AI is simply one manifestation of culture as a whole. Its technical problems are one specific arena where the implications of ideas that are rooted in background culture are worked out.
But if AI is fundamentally embedded in and working through culture, then cultural studies and AI may have a lot to say to each other. Specifically, cultural theorists have spent a lot of time thinking about and debating subjectivity. AI researchers have spent a lot of time thinking about and debating architectures for autonomous agents. Once these two are linked, each body of work can be used to inform the other. If agents use a particular theory of subjectivity, then we can use ideas about this theory to inform our work on agents. And if agents are a manifestation of a theory of subjectivity, then studying these agents can give us a better idea of what that theory means. This raises the possibility that cultural studies and AI can form a strategic alliance.
Science studies, after all, examines culturally-based metaphors that inform scientific work, and thereby often uncovers deeply-held but unstated assumptions that underly it. Scientists are also generally interested in understanding the forces, both conscious and unconscious, that can shape their results. If there are ways in which they can better understand the phenomena they study or build the technology they want to create, they are all ears. In this respect, as Evelyn Fox Keller points out, the insights of science studies can contribute great value to science's self-understanding 22.
At the same time, many practitioners of science studies are deeply interested in science as it is actually practiced on a day-to-day level. This means scientists, with their in-depth personal experience of what it means to do scientific work, are privy to perspectives that can enrich the work of their science studies counterparts. Science studies simply is not possible without science, and an important component of it is an accurate reflection of the experiences of scientists themselves.
With all the advantages that cooperation could bring, you might think that science and science studies would be enthusiastic partners on the road to a shared intellectual enterprise. Alas, the practicioners of science studies and many of their hapless subjects know that that is far from the case. Productive exchanges between cultural critics and scientists interested in the roots of their work are hampered by the disciplinary divide between them. This divide blocks cultural critics from access to a complete understanding of the process and experience of doing science, which can degrade the quality of their analyses and may lead them to misinterpret scientific practices. At the same time, scientists have difficulty understanding the context and mindset of critiques of their work, making them unlikely to consider such critiques seriously or realize their value for their work, potentially even leading them to dismiss all humanistic critiques of science as fundamentally misguided 23.
This feedback loop of mutual misunderstanding has grown into a new tradition of mutual kvetching. Cultural critics may complain that scientists unconsciously reproduce their own values in their work and then proclaim them as eternal truth. They may feel that scientists are not open to criticism because they want to protect their high (relative to the humanities') status in society. Simultaneously, scientists sometimes complain that cultural critics are absolute nihilists who do not believe in reality and equate science with superstition 24. They fear that cultural critics undermine any right that science has as a source of knowledge production to higher status than, say, advertising. Finally, both sides complain incessantly --- and correctly ---- of being cited, and then judged, out of context.
The unfortunate result of this situation is a growing polarization of the two sides. In the Science Wars, pockets of fascinating interdisciplinary exchanges and intellectually illuminating debate are sadly overwhelmed by an overall lack of mutual understanding and accompanying decline of goodwill. While most participants on both sides of the divide are fundamentally reasonable, communication between them is impaired when both sides feel misunderstood and under attack. This siege mentality not only undermines the possibility for productive cooperation; with unfortunate frequency, it goes as far as cross-fired accusations of intellectual bankruptcy in academic and popular press and nasty political battles over tenure. These unpleasant incidents not only help no one but also obscure the fact that both the academic sciences and the humanities are facing crises of funding in an economy that values quick profit and immediate reward over a long-term investment in knowledge. In the end, neither science nor science studies benefits from a situation best summed up from both sides by Alan Sokal's complaint: ``The targets of my critique have by now become a self-perpetuating academic subculture that typically ignores (or disdains) reasoned criticism from the outside'' 25.
This can be seen most clearly in a rather unusual opinion piece that appeared several years ago in the AI Magazine 26. The remarkable rhetoric of this essay in a journal more often devoted to the intricacies of extracting commercially relevant information from databases may be appreciated in this excerpt:
Once upon a time there were two happy and healthy babies. We will call them Representation Baby (closely related to Mind Baby and Person Baby) and Science Baby (closely related to Reality Baby).A little decoding is in order for those not intimately aware of both the AI debates and the Science Wars. ``SitNanny'' represents situated action, a brand of alternative AI that focuses its attention on the way in which agents are intimately related to, and cannot be understood without, their environment. ``RadNanny,'' as is immediately clear to even the most naive science studies aficionado, is the embodiment of the cultural studies of science, social constructivism being the belief that science, like every other human endeavor, is at least partially a product of sociocultural forces (the `radical' here functions as little more than an insult, but implies that science is purely social, i.e. has absolutely no relationship to any outside reality).
These babies were so charming and inspirational that for a long time their nannies cared for them very well indeed. During this period it was generally the case that ignorance was pushed back and human dignity increased. Nannies used honest, traditional methods of baby care which had evolved during the years. Like many wise old folk, they were not always able to articulate good justifications for their methods, but they worked, and the healthy, happy babies were well growing and having lots of fun.
Unfortunately, some newer nannies haven't been so careful, and the babies are in danger from their zealous ways. We will focus on two nannies who seem to be close friends and often can be seen together - Situated Nanny (called SitNanny for short) and Radical Social Constructivist Nanny (known to her friends as RadNanny) (15) 27.
Having broken the code, the implication of this excerpt is clear: everything in AI was going fine as long as we thought about things in terms of science and knowledge representation, one of the core terms of classical AI. Of course, this science was not always well-thought-out, but it was fundamentally good. That is, until that dastardly alternative AI came along with cultural studies in its tow and threatened nothing less than to kill the babies.
Now any cultural critic worth his or her salt will have some choice commentary on a story in which the positive figures are all male babies living the life of leisure, and the negative figures all lower-class working women 28. But the really interesting rhetorical move in this essay is in the alignment of the classical-alternative AI debate with the Science Wars. Classical AI, we learn, is good science. Alternative AI, while having some good ideas, is dangerous, among other reasons because it is watering down science with other ideas: ``concepts from fringe neurology, sociology, ethnomethodology, and political theory; precomputational psychological theory; and God knows what else'' (19). Alternative AI is particularly dangerous because it believes that agents cannot be understood without reference to their environment. Hence, it is allied with the ``cult'' (20) of science studies, which believes that scientists cannot be understood without reference to their sociocultural environment.
Since the majority of their audience presumably has little awareness of science studies, the authors are happy to do their part for interdisciplinary awareness by explaining what it is. They state, in a particularly nice allusion to 1950's anti-Communist hysteria, that science studies aims at nothing less than to ``reject the entire fabric of Western science'' (15). Science studies, we are informed, believes ``that all science is arbitrary and that reality is merely a construction of a social game'' (23). In the delightful tradition of the Science Wars, several quotations are taken out of context to prove that cultural critics of science believe that science is merely an expendable myth.
The statements Hayes et. al. make are simply inaccurate descriptions of science studies. In reality, science studies tends to be agnostic on such questions as the arbitrariness of science and on the nature of reality, to which science studies generally does not claim to have any more access than science does. When science studies does look into these issues it does so in a much more subtle and complex way than simply rejecting or accepting them.
But what is more important than these factual inaccuracies is that the article promotes the worst aspects of the Science Wars, since the very tone of the article is chosen to preclude the possibility of productive discussion. Science studies is simply dismissed as ludicrous. If uninformed scientists reading the article have not by the end concluded that science studies is an evil force allied against them, with alternative AI its unfortunate dupe, it is certainly not for lack of trying.
In this section, we will buck the trend of mutual disciplinary antagonism by exploring the potential of what former agent researcher Philip Agre calls critical technical practices 29. A critical technical practice is a way of actually doing AI which incorporates a level of reflexive awareness of the kind espoused by science studies. This may include awareness of the technical work's sociocultural context, its unconscious philosophies, or the metaphors it uses. We will look at various AI researchers who have found ideas from cultural studies helpful in their technical work.
These attitudes can only be maintained by studiously avoiding noticing the people who are both scientists and cultural critics. Gross and Levitt's influential onslaught against science studies 30, for example, argues that cultural critics are irresponsible and dangerous because they are ignorant of the science they criticize. This argument is made easier by counting interdisciplinarians who do both science and cultural studies as (good, responsible) scientists and not as (bad, irresponsible) cultural critics (the question of why those scientists would find it interesting or even fruitful to keep such unseemly company is left unanswered). And in an exhaustive survey of every important figure in cultural studies, some of the most influential `culturalist scientists' are left out altogether. A glaring omission is Richard Lewontin, whose influential books on the cultural aspects of biology are the sidelight to an illustrious career as a geneticist 31.
Similarly, the hypothesis that scientists do not know or care about the effects of their work is contradicted by the work of Martha Crouch 32. Crouch is a botanist who, after many years of research, noticed that the funding of botany combined in practice with the naive faith of scientists in their own field to completely undermine the idealistic goals of plant scientists themselves. Crouch determined to help scientists such as herself achieve their own stated goals of, for example, feeding the hungry, by adding to their self-understanding through the integration of cultural studies with botany.
But, to be fair, much of the work integrating science with science studies may be invisible to both cultural critics themselves and the scientists whose form of intellectual output seems to largely be attacks on those on the other side of the great intellectual divide. This is because scientists who are actually using culturalist perspectives in their work generally address that work to their scientific subcommunity, rather than to all of science and science studies as a whole. And in work that is addressed to a technical subfield, it is usually not particularly advantageous to mention that one's ideas stem from the humanities, particularly if they come from such unseemly company as hermeneutics, feminism or Marxism.
Here, we will uncover the history of the use of culturalist perspectives within AI as a part of technical work. It turns out that within AI, the use of cultural studies perspectives is not just a couple of freak accidents traceable to a few lone geniuses and / or lunatics. Rather, there is a healthy if somewhat hidden tradition of a number of generations of AI researchers who have drawn inspiration from the humanities in ways that have had substantial impact on the field as a whole. We look at both how cultural studies was found to be useful, and the concrete methods various researchers have used to combine the fields.
In Understanding Computers and Cognition, Winograd and Flores analyze AI as a continuation of the analytic tradition 33. AI's investment in this tradition, they conclude, is so great that it cannot address what they consider to be fundamental attributes of intelligence. Their critique is based on the Heidegerrian notion that people approach the world from a set of prejudices that cannot be finitely articulated. If these prejudices cannot be finitely articulated, then they cannot be explicitly represented in machinery; any machinic representation of subjectivity will therefore necessarily leave out some of the complex background knowledge with which people approach real-world situations. This means that AI is able to solve limited, formal problems, but cannot attain true intelligence because ``[t]he essence of intelligence is to act appropriately when there is no simple pre-definition of the problem or the space of states in which to search for a solution'' (98). Winograd and Flores argue that instead of making computers that can communicate with us, we should make computers a means to aid communication between people.
While Winograd and Flores's arguments certainly made a splash in the field, it must be honestly stated that they probably did not cause too many scientists to leave AI (and they were not intended to). The basic flaw from this perspective in the argument is that it forces AI researchers to choose between believing in Heidegger and believing in AI. One can hardly blame them if they stay with the known evil.
What is interesting to those who remain in AI, however, is Winograd and Flores's methodology for combining a critical perspective with AI. Winograd and Flores analyze the limitations of AI that stem from its day-to-day methodologies. When they find those constraints to exclude the possibility of truly intelligent behavior, they decide instead to start building systems in which those constraints become strengths. In other words, they decide that artificial systems necessarily have certain characteristics of rigidity and literalness, then ask themselves what sorts of social situations could be aided by a rigid, literal system. They then build a system that is an enforcer of social contracts in certain, limited situations where they feel it is important that social agreements be clearly delineated and agreed upon. Specifically, the system articulates social agreements within work settings, so that workers are aware of who has agreed to do what. This new system is designed to be useful precisely because of the things that were previously limitations. Winograd and Flores, then, use cultural studies to inform technical development by finding constraints in its methodologies, and then using those constraints so that they become strengths.
Suchman noticed that the ideas of planning were heavily based on largely Western notions of, among other things, route planning. She then asked herself what kind of `planning' you would have if you used the notions of a different society. By incorporating perspectives from Micronesian society, she came up with the concept of `situated action,' which you may remember as the butt of ridicule in Hayes et. al.'s ``On Babies and Bathwater.''
Situated action's basic premise is to generate behavior on the fly according to the local situation, instead of planning far ahead of time. Although Suchman herself made no claims to technical fame, her ideas became influential among AI researchers who were working on similarly-motivated technology (see below), becoming an important component in an entire subfield AI researchers now either love or hate, but generally cannot ignore. Her methodology, in sum, is to notice the culture-boundedness of a particular metaphor (``planning'') that informs technical research, then ask what perspectives a very different metaphor might bring to the field instead. The point in her work is not that Western metaphors are `wrong' and non-Western ones are `right,' but that new metaphors can spawn new machinery that might be interesting in different ways from the old machinery
Chapman argues that some of the most interesting papers in AI do not make technical contributions in any strict sense of the term --- i.e., that the best methodology for AI is not necessarily that of empirical natural science. "[Some of the best] papers prove no theorems, report no experiments, offer no testable scientific theories, propose technologies only in the most abstract terms, and make no arguments that would satisfy a serious philosopher.... [Instead, t]hese papers have been influential because they show us powerful ways of thinking about the central issues in AI" (214). Suchman's anthropological work in AI is a living example in Chapman's work of such an influential idea.
Together, Chapman and Agre develop novel techniques for building agents which are based on a new conceptualization of what it means to be an agent. This conceptualization has roots in Winograd's Heideggerian analysis of AI, and is also deeply influenced by ethnomethodology, particularly Garfinkel and Suchman's work described above. Chapman and Agre reject the idea that problem-solving is central to agenthood, and instead see agenthood as process, engaging in a rich set of interactions with other agents and the physical world.
The world of everyday life... is not a problem or a series of problems. Acting in the world is an ongoing process conducted in an evolving web of opportunities to engage in various activities and contingencies that arise in the course of doing so.... The futility of trying to control the world is, we think, reflected in the growing complexity of plan executives. Perhaps it is better to view an agent as participating in the flow of events. An embodied agent must lead a life, not solve problems 37.This re-understanding of the notion of agent has been an important intellectual strand in alternative AI's reconceptualization of agent subjectivity.
In recent work, Agre has distilled his approach to combining philosophy, critical perspectives, and concrete technical work into an articulated methodology for critical technical practices per se. Agre sees critical reflection as an indispensable tool in technical work itself, because it helps technical researchers to understand in a deep sense what technical impasses are trying to tell them. He sums up his humanistic approach to AI with these postulates:
1. AI ideas have their genealogical roots in philosophical ideas. 2. AI research programs attempt to work out and develop the philosophical systems they inherit. 3. AI research regularly encounters difficulties and impasses that derive from internal tensions in the underlying philosophical systems. 4. These difficulties and impasses should be embraced as particularly informative clues about the nature and consequences of the philosophical tensions that generate them. 5. Analysis of these clues must proceed outside the bounds of strictly technical research, but they can result in both new technical agendas and in revised understandings of technical research itself 38.Humanists will recognize Agre's methodology as hermeneutics; it is a kind of interpretation that goes beyond surface appearances to discover deeper meanings. For Agre, purely technical research is the surface manifestation of deeper philosophical systems. While it is certainly possible for technical traditions to proceed without being aware of their philosophical bases, technical impasses provide clues that, when properly interpreted, can reveal the philosophical tensions that lead to them. If these philosophical difficulties are ignored, chances are that technical impasses will proliferate and remain unresolved. If, however, they are acknowledged, they can become the basis for a new and richer technical understanding.
In Computation and Human Experience, Agre develops a methodology for integrating AI and the critical tradition through the use of deconstruction 39. This works as follows:
In The Embodied Mind: Cognitive Science and Human Experience, Varela, Thompson and Rosch integrate cognitive science with Buddhism, particularly in the Madhyamika tradition 40. They do this by connecting cognitive science as the science of cognition with Buddhist meditation as a discipline of experience. Current trends in cognitive science tend to make a split between cognition and consciousness, to the point that some cognitive scientists call consciousness a mere illusion. Instead, Varela et. al. connect cognition and experience so cognitive scientists might have some idea of what their work has to do with what it means to be an actual, living, breathing human being.
Varela, Thompson, and Rosch stress that cognitive science --- being the study of the mind --- should be connected to our actual day-to-day experience of what it means to have a mind. What they mean here by experience is not simple existence per se but a deep and careful examination of what that existence is like and means. They believe that your work should not deny or push aside your experience as a being in the world. Instead, that experience should be connected to and affirmed in your work. In this way, they connect with cultural critics of science like Donna Haraway and cultural theorists like Gilles Deleuze and Félix Guattari, who stress the importance of personal experience as a component of disciplinary knowledge 41.
One of the tensions that has to be resolved in any work that combines science with non-scientific disciplines (of which Buddhism is certainly one!) is the differential valuation of objectivity. Science tends to see itself as objective, generating knowledge that is independent of anyone's individual, personal experiences. Since Varela, Thompson and Rosch want to connect cognitive science as science with individual human experience, they confront this problem of subjectivity versus objectivity head-on.
Interestingly, they do this by redefining what objectivity means with respect to subjective experiences. You cannot truly claim to be objective, they say, if you ignore your most obvious evidence of some phenomenon, i.e. your personal experience of it. This is particularly true when one is studying cognition ---- in this frame of thought, any self-respecting study of the mind should be capable of addressing the experience of having one!
Given that one of the things cognitive scientists (and, by extension, AI researchers) are or should be interested in is subjective experience, Varela, Thompson, and Rosch abandon the focus on objectivity per se. But they stress that this does not lead to the nihilistic abandonment of any kind of judgments of knowledge which seems to haunt the nightmares of many participants in the Science Wars. Rather, they argue that Buddhist traditions have disciplined ways of thinking about that experience. The problem, they say, is not with subjectivity, but with being undisciplined. The goal, then, is being able to generate a kind of cognitive science that is subjective without being arbitrary.
Petit Mal, for example, is a minimalistically engineered, whimsical, elegantly clumsy robot, which interacts physically with the audience and whose chaotic behavior illicits an enormous range of culturally-specific interpretations from its audience.The tenuous relationship between Petit Mal's simple design and the audience's complex interpretation points out the extent to which our perceptions of and judgements about technical artefacts are always already embedded in a cultural environment. "Petit Mal constitutes an Embodied Cultural Agent: an agent whose function is self reflexive, to engage the public in a consideration of the nature of agency itself" 43.
Mateas argues, "AI-based art is not a subfield of AI, nor affiliated with any particular technical school within AI, nor an application of AI. Rather it is a stance or viewpoint from which all of AI is reconstructed" 46. In particular, expressive AI focuses on the expression of human authorial intention through `intelligent' machines, rather than on the generation of autonomous intelligent processes. An explicit commitment of Expressive AI is the analysis and provision of interpretive and authorial affordances, i.e. what sorts of interpretations a technical design or methodology supports, and the `knobs' or `hooks' it provides authors in order to embody their chosen concepts in the machine.
The approach taken in my own work follows Varela, Thompson, and Rosch in asserting that subjective experience, which goes to the heart of what it means to humans to be alive in the world, should be an important component of AI research. I believe that one of the major limitations of current AI research --- the generation of agents that are smart, useful, profitable, but not convincingly alive --- stems from the traditions AI inherits from science and engineering. These traditions tend to discount subjective experience as unreliable; the experience of consciousness, in this tradition, is an illusion overlaying the actual, purely mechanistic workings of our biological silicon. It seems to me no wonder that, if consciousness and the experience of being alive are left out of the methods of AI, the agents we build based on these methods tend to come across as shallow, stimulus-response automatons.
In the reduction of subjective experience to mechanistic explanations, AI is by no means alone. AI is part of a broader set of Western cultural traditions, such as positivist psychiatry and scientific management, which tend to devalue deep, psychological, individual, and subjective explanations in favor of broad, shallow, general, and empirically verifiable models of the human. I do not deny that these theories have their use; but I fear that, if taken as the only model for truth, they leave out important parts of human experience that should not be neglected. I take this as a moral stance, but you do not need to accept this position to see and worry about the symptom of their neglect in AI: the development of agents that are debilitatingly handicapped by what could reasonably accurately, if metaphorically, be termed autism.This belief that science should be understood as one knowledge tradition among others does not imply the rejection of science; it merely places science in the context of other, potentially --- but not always actually --- equally valid ways of knowing. In fact, many if not most scientists themselves understand that science cannot provide all the answers to questions that are important to human beings. This means that, as long as AI attempts to remain purely scientific, it may be leaving out things that are essential to being human.
In Ways of Thinking: The Limits of Rational Thought and Artificial Intelligence, for example, cognitive scientist Méró, while affirming his own scientific stance, comes to the disappointing conclusion that a scientific AI will inevitably fall short of true intelligence.
In his book Mental Models Johnson-Laird says, `Of course there may be aspects of spirituality, morality, and imagination, that cannot be modeled in computer programs. But these faculties will remain forever inexplicable. Any scientific theory of the mind has to treat it as an automaton.' By that attitude science may turn a deaf ear to learning about a lot of interesting and existing things forever, but it cannot do otherwise: radically different reference systems cannot be mixed. (228-229) 47But while the integration of science and the humanities is by no means a straightforward affair, the work already undertaken in this direction by researchers in AI and other traditionally scientific disciplines suggests that Méró's pessimism does not need to be warranted. We do have hope of creating a kind of AI that can mix these `radically different reference systems' to create something like a `subjectivist' craft tradition for technology. Such a practice can address subjective experience while simultaneously respecting its inheritances from scientific traditions. I term these perhaps heterogeneous ways of building technology that include and respect subjective experience `subjective technologies.' My work is one example of a path to subjective technology, achieved through the synthesis of AI and cultural studies, but it is by no means the only possible one.
Because of the great differences between AI and cultural studies, it is inevitable that a synthesis of them will include things unfamiliar to each discipline, and leaves out things that each discipline values. In my approach to this synthesis, I have tried to select what is to be removed and what is to be retained by maintaining two basic principles, one from AI and one from cultural studies: (1) faith in the basic value of concrete technical implementation in complementing more philosophical work, including the belief that the constraints of implementation can reveal knowledge that is difficult to derive from abstract thought; (2) respect for the complexity and richness of human and animal existence in the world, which all of our limited, human ways of knowing, both rational and nonrational, both technical and intuitive, cannot exhaust.
The additions I make to these approaches are based on a broad analysis of attempts to limit or circumscribe human experience. I believe that the major way in which AI and similar sciences unintentionally drain the human life out of their objects of study is through what agent researchers Petta and Trappl satirize as `boxology:' the desire to understand phenomena in the world as tidy black boxes with limited interaction 48. In order to maintain the comfortable illusion that these black boxes sum up all that is important of experience, boxologists are forced to ignore or devalue whatever does not fall into the neat categories that are set up in their system. The result is a view of life that is attractively simple, but with glaring gaps, particularly in places where individual human experience contradicts the established wisdom the categories represent.
The predominant contribution to this tradition of critical technical practices which I try to make is the development of an approach to AI that is, at all levels, fundamentally anti-boxological. At each level, this is done through a contextualizing approach. My approach is based on this heuristic: ``that there is no such thing as relatively independent spheres or circuits'' 49. My approach often feels unusual to technical workers because it is heavily metaphorical; I find metaphorical connections immensely helpful in casting unexpected light on technical problems. I therefore include in the mix anything that is helpful, integrating deep technical knowledge with metaphorical analysis, the reading of machines 50, hermeneutics, theory of narrative, philosophy of science, psychology, animation, medicine, critiques of industrialization, and, in the happy phrasing of Hayes and friends, ``God knows what else.'' The goal is not to observe disciplinary boundaries --- or to transgress them for the sake of it --- but to bring together multiple perspectives that are pertinent to answering the question, ``What are the limitations in the way AI currently understands human experience, and how can those limitations be addressed in new technology?''
Concretely, some of my most recent technical work is based on a tracing out and treating of the consequences of the boxological approach current in AI. I argue that the desire to construct agents in terms of a limited number of independent black boxes leads to a form of schizophrenia, or gradual incoherence in the overall behavior of the agent as more and more of these "black boxes" are combined. This schizophrenia can be traced to atomizing methodologies AI inherits from its roots in industrial culture. The disintegration AI researchers can recognize in their agents, like that felt by the assembly line worker and institutionalized mental patient, is at least in part a result of reducing subjective experience to objective atoms, each taken out of context and therefore out of relationship to one another and to the context of research itself.
This suggests that the problems of schizophrenia can be mitigated by putting the agent back into its sociocultural context, understanding its behavior as implicated in a cycle of human interpretation, on the part of both its builder and those who interact with and judge it. This approach to AI, which sees agents not in a sociocultural vacuum but as a form of communication between human beings, I term "socially situated AI" and is closely related to Mateas's Expressive AI. With this metaphor as a basis, it becomes clear that creating coherence means integrating, not the agent's internally defined code, but the way in which the agent presents itself to human users. This changes the focus in agent-building from primarily a design of the agent alone, with its subsequent interpretation as an afterthought, to including the agent's comprehensibility in the design and construction of agents from the start.
Narrative psychology suggests that agents will be maximally comprehensible as intentional beings if they are structured to provide cues for narrative. I therefore argue that agent behavior should be structured as narrative, in order to make it as easy as possible for users to make coherent sense of agent activity. I implement this narrative structure for behavior using an agent architecture, the Expressivator, that connects formerly disparate behavior into coherent narrative sequences 51.
Why should a humanist care about this development? On the basis of my experience, I believe there are several advantages to using cultural studies as a basis for a practice of AI. The first is that by actually practicing AI, the cultural critic has access to a kind of experiential knowledge of science that is difficult to get otherwise and will deepen his or her theoretical analysis. This increased knowledge is expressed in two ways in my work: (1) analysis of alternative AI as a manifestation of industrial culture, and (2) analysis of the metaphorical basis of alternative AI even into the details of the technology. The second advantage is that working within AI allows cultural theorists to not only criticize its workings, but to actually see changes made in practice on the basis of those criticisms. The Expressivator reflects the cultural studies analysis in the fundamental changes it makes in how an agent is conceived and structured. This brings home at a technical level the idea that agents are not simply beings that exist independently, but have authors and audiences by which and for which they are constructed.
Finally, the most important advantage to such an approach is the potential alteration to the rhetoric of mutual assured destruction that currently seems to be prevalent in interdisciplinary exchanges between cultural studies and science. The most fundamental contribution my work tries to make toward a cease-fire in the Science Wars is in demonstrating that `science criticism' is relevant to and can be embodied in the development of technology, so that there are grounds for the two sides to respect each other, as well as a reason for them to talk. In order to address contemporary experience, we need both sides. My hope is that my work can join other similarly motivated work on whatever side of the interdisciplinary divide to replace the Science Wars with the Science Debates, a sometimes contentious and always invigorating medley of humanist, scientific, and hybrid voices.
But there is no reason why critical technical practices --- practices of technology-building which include a critical perspective --- should be limited to the subfield of AI. In fact, complementary practices have already developed and continue developing in other parts of computer science. These critical perspectives have long played a role in the field of computer-human interaction, for instance. A nice example is Kristina Höök's work, in which she develops new tools for evaluation that analyze the pleasurable quality of the experience the system provides, rather than focusing on its efficiency 53.
In a related vein, critical technical practices, and particularly cultural informatics, may have an enormous advantage in developing poetic technology, technical applications which enrich human life, not by making it more efficient, but by inspiring sensations of magic and wonder. Chris Dodge's "The Bed" is a beautiful example of this kind of technology: it is an environment to allow intimate connection between people who are far from one another. A pillow on the bed heats when the remote participant is there, and vibrates in time with the remote person's heartbeat; a curtain moves in time with his or her breath, and colorful shadows are projected onto it according to the tenor of conversation. The result is a feeling of connection and intimacy, made possible not by optimized functionality but by the emotionally-laden overtones of the meaning of bed, light/dark, shadows, and so on 54.
Certainly, there are still gaps in the work that has been done; in particularly, in AI there has been a heavy emphasis on semiotic, philosophical, and metaphorical analysis, which can be relatively easily "smuggled into" the rhetoric of computing, with a corresponding lack of materialist analysis and work in the political economy of computing. In addition, research in critical technical practices and cultural informatics is generally done under-the-table; research communities are organized by technical application area, not by degree of incorporation of extra-disciplinary viewpoints. If research in this area is to blossom, we will probably need our own mailing lists, workshops, conferences, journals. Coherence of the community may be threatened by the heterogeneity of technical approaches, which after all may require a technically specialized audience.
Critical technical practices are generally thought of as a way of reforming the practice of computer science. A crucial question practicioners of critical technical practices will therefore have to answer is how they understand their relationship to those outside of computer science pursuing similar projects. In particular, new media art practice is often also a critical technical practice, when artists build complex computational systems (i.e. artworks) which are informed by critical reflection on technology and its role in society. The lines between technical practice, artwork, and cultural studies are blurring, and the space between is becoming home to a new interdiscipline. Hopefully, under this pressure the traditions informing the design and development of computational systems will expand, allowing for an altogether different way of looking at technology in society, and allowing for technical artefacts that enrich human experience, rather than reducing it to a quantified, formalized, efficient, and lifeless existence.