La Inteligencia Artificial, La Cognición, y los Video Juegos

(Lo siguiente es la versión ensayo de la charla que ofrecí en el Primer Festival de Video Juegos en Puerto Rico, el 20 de julio de 2013.) 

A través de la historia de los juegos de video, el énfasis primordial, en un contexto evolutivo, ha sido el de mejorar las gráficas. Desde muy temprano se descubrió que el ojo domina en cuestiones de juzgar la calidad de los juegos. Si tomamos un juego de cada uno de los años, desde el 1980 hasta el 2013, uno puede apreciar la tendencia hacia juegos mas llamativos gráficamente.

Una selección de juegos, del 1980 hasta el 2013.

Claro, estos ejemplos son tomados del ámbito profesional, no de desarrolladores independientes. Esta tendencia de mejorar las gráficas ha sido posible por los avances tecnológicos de procesadores y tarjetas de video cada vez mas potentes. Cabe recalcar que los primeros juegos para las masas (juegos como Donkey Kong, PacMan, Mario Bros.), aún cuando sufrían limitaciones en cuestión de tecnología (4K de memoria era enorme para aquél entonces), dedicaban una gran parte de sus códigos para el despliegue de gráficas a la pantalla. Esta limitación y énfasis en gráficas obligaba a los programadores de aquél entonces a ponerse creativos al querer introducir otros elementos a los juegos. Piense, el juego Mario Bros., producido para el Nintendo Entertainment System (el cual corría un procesador MOS 6502 con no mas de 2K para la memoria), pero aún así, logró tener una gran variedad de enemigos (algunos de los cuales exhibían una inteligencia rudimentaria), y de niveles (o en buen castellano “tablas”), y también logró tener música. Esta creatividad al programar es la semilla de muchos algorítmos que hoy en día se conocen como “Generación de Contenido Programático” (o como se conoce en inglés “procedural content generation“), una técnica de la inteligencia artificial para introducir contenido a medida que corre el juego. Es decir, antes de prender el juego ese contenido no existe, y los códigos del juego crean el contenido (por ejemplo enemigos, niveles) dinámicamente.

El Fin de la Revolución Gráfica

Hay quienes que piensan que la revolución gráfica (por darle nombre a este movimiento) se acabará dentro de 10 años. A mi no me gusta hacer ese tipo de predicción, porque la tecnología es verdaderamente impredecible. Lo que sí me siento cómodo en decir es que en algún punto se acabará. L.A. Noire demostró lo que era posible para juegos de consolas. Varios juegos como Grand Theft Auto IV, y Skyrim (que fueron publicados en consolas como también para PC) han disfrutado de mejoras a la calidad de gráficas, impulsadas por comunidades de fanáticos de dichas franquicias. Pero llegará el día donde no se podrá mejorar más. Y entonces vuelvo a la pregunta que motivó esta presentación: ¿cuál es el próximo paso evolutivo?

El Inicio de la Revolución de la Inteligencia Artificial

Como ya saben, mi respuesta a la interrogante es: la inteligencia artificial. El problema es que tenemos que deshacernos del concepto que comúnmente se le atribuye a la inteligencia artificial en contextos interactivos. El problema no es uno de crear enemigos más difíciles para que el jugador sufra derrotas casi injustas; eso sería inteligencia artificial en el sentido mas grosero. Sino, el problema es el de la Ingeniería de la Experiencia – cómo asegurarte de que cada individuo pueda disfrutar de una experiencia interactiva, que cambie y evolucione con la jugadora, y que sea única cada vez que se siente a jugar.

Si son fanáticos de la ciencia ficción de los 80 y los 90, ya se habrán dado cuenta de que lo que estoy describiendo aparece prominentemente en la serie de televisión Star Trek: The Next Generation: la visión del Holodeck. Es muy posible que esta metáfora murió hace 10 años, tal que para aquellos que desconocen, el Holodeck era una computadora única, en la cuál uno podía programar un ambiente interactivo con representación fidedigna de la realidad, solo que era una simulación computarizada sofisticada. Como punto y aparte, mi área de investigación sueña con crear una computadora lo suficientemente avanzada como para crear el Holodeck.

Volviendo a la Ingeniería de la Experiencia – el problema de crear experiencias interactivas, únicas y cambiantes, cae en el dominio de la inteligencia artificial, lo cual provee la motivación para la revolución que se aproxima. Analicemos entonces las experiencias que la Inteligencia Artificial facilita:

El Efecto Catalizador de la Inteligencia Artificial para la Ingeniería de la Experiencia

He identificado cuatro áreas en las cuales pienso que la inteligencia artificial puede únicamente facilitar diferentes experiencias.  No quiero decir que estas son las únicas áreas, pero pienso que es un buen punto de  partida.  Para evitar que este escrito se convierta en una mini-tesis, proveeré enlaces para cada uno de los proyectos de ejemplo.

1. Generación de Contenido Programático –  la programación de algorítmos que generan código para contenido dinámicamente.  Este contenido no existe antes de que comienze el juego como tal, sino que se genera a medida que la jugadora juega.  Ejemplos: PaSSAGE, Galactic Arms Race, Games by ANGELINA
2. Análisis Conductual de Jugadores – el análisis de la conducta de los jugadores, que puede ser interpretada de diversas maneras. La data que se recopila puede ser utilizada para mejorar el juego, como también para generar contenido personalizado.  Ejemplos: Análisis Conductual de los Jugadores en League of Legends
3. Modelaje de Situaciones Sociales Complejas – a medida que los juegos intentan simular aspectos de la vida real, se torna cada vez mas compleja la simulación.  Los algorítmos de la inteligencia artificial son capaces de lidiar con complejidad emergente, y han sido utilizados para crear juegos cada vez más sutiles.  Ejemplos: Façade, Prom Week, Versu
4. Diseño Automatizado de las Mecánicas de los Juegos – el desarrollo de juegos de gran embergadura se torna cada vez mas difícil, dada la demanda para más contenido, y más interacción en los video juegos.  Esto crea una demanda para herramientas que puedan ayudar en el proceso de diseño, y que puedan razonar acerca de las reglas de los juegos, al igual que sobre el contenido que los jugadores enfrentarán.  Ejemplos: Ludocore, Tanagra

En conclusión, estamos empezando la revolución en la inteligencia artificial, demostrado por esta selección de experiencias únicas, las cuales no hubiesen sido posibles, sin avances en técnicas avanzadas.

La Cognición

Hay una comunidad, compuesta de desarrolladores e investigadores científicos de los video juegos, que piensan que el componente mas fundamental de un juego es su capacidad para proveer modos de interacción. En otras palabras, lo importante en un juego es lo que puedes hacer. La Dra. Janet Murray, catedrática de Georgia Tech y una eminencia en el estudio de juegos, proclamó que tu capacidad de intervención en un juego determina tu disfrute del mismo. Ella definió la capacidad de intervención como la potestad de poder actuar en un entorno interactivo y observar los efectos de las acciones; o sea, no basta con poder actuar, sino que también tu debes poder observar algún cambio en tu entorno, presumiblemente causado por ti.

Junto a varios colegas, retamos este punto de vista. Diseñamos un experimento en el cuál varias personas jugaron un juego de texto, similar al juego del antaño Zork. Los partícipes del experimento jugaron una de dos versiones del juego; en la primera, se les presentaba opciones para jugar como es de esperar de un juego de texto, y en la otra versión, se presentaban las mismas opciones, pero con una modificación simple: aún cuando presentaba las mismas opciones, eran falsas – no conducían a nada diferente. Como es de esperar, las personas en la versión modificada reportaron sentirse limitadas a través del juego.

Entonces, tomamos la versión modificada e introducimos otro cambio. En esta segunda ronda de experimentos, las dos versiones del juego producían la misma respuesta – es decir, todas las opciones para jugar resultaban en lo mismo. Lo único diferente es que ahora, en una de las versiones, el juego reconocía la acción, presentando un mensaje personalizado.Mi argumento es que la percepción y la cognición son clave para el desarrollo de experiencias interactivas.

Por ejemplo: tu empezabas el juego, y tenías la opción de escoger entre la espada o el arco y flecha, antes de continuar. Si decidías escoger la espada, lo único diferente en esta versión es que el juego te demostraba un mensaje como el siguiente: “Haz tomado la espada por el mango, y decides salir de tu casa.”

Aún cuando este cambio es relativamente insignificante, los partícipes del experimento reportaron sentirse significativamente más en control de la experiencia interactiva, de lo cual concluimos que no hace falta crear un juego complejo para presentar opciones de juego satisfactorias. Basta con crear la ilusión de control para que los jugadores estén conformes.

Mi Tesis:  Modelaje Cognitivo del Razonamiento Narrativo

Mi tesis intenta realizar el modelaje cognitivo a través de la inteligencia artificial. Me interesa modelar cómo es que uno interpreta los cuentos interactivos, a medida que uno los vive. En particular, yo me enfoco en la memoria como base cognitiva de diversos fenómenos narratológicos. En esencia, mi argumento es que la memoria es fundamental porque afecta tu capacidad de hacer inferencias, y da paso a sensaciones de suspenso, sorpresa, y hasta terror. Considera la escena donde el antagonista de la película Skyfall intenta tomarle la vida a la dirigente del MI6.

Cuando uno ve esa escena, la memoria de los eventos (el hecho de que el antagonista se encuentra de camino a “M” en todo momento que ella habla) y la capacidad de inferencias (el hecho de que uno sabe las intenciones del antagonista y puede extrapolar sus acciones) lleva a uno sentirse ansioso.

Comprensión Narrativa en los Video Juegos

Mi tesis no es en el contexto de las narrativas cinematográficas, sino que es en el contexto de los juegos. El contexto interactivo ofusca los fenómenos narratológicos, ya que los juegos llevan consigo ciertas expectativas (como parte del medio en el cual se realizan) que afectan los procesos de inferencia. Aún pienso que la memoria tiene que ver algo, pero es menos claro cómo. Mi tesis se dedicará a investigar la naturaleza de la interacción entre la cognición y los juegos.

En Resumen,

estamos en medio de una revolución de la inteligencia artificial (IA) en los juegos de video. Mi argumento es que la cognición esta al centro de la experiencia de juego. Mi tema de tesis intenta modelar la cognición con estructuras de datos tomadas de la IA.  La revolución artificial ha comenzado.  Espero que seas parte de ella.

* Si te interesa, puedes ver las transparencias que utilizé en mi charla.

Reply: “The Simulation Dream” by Tynan Sylvester

I was recently recommended a Gamasutra article on “The Simulation Dream,” by Tynan Sylvester.  I wanted to write down a quick response that I’d love to start a discussion around.

Overall, I agree with Sylvester’s article, which essentially echoes that what matters in a game  is what players perceive, as opposed to what the game offers.  In essence, a game may hold great potential (through extensive narrative content, or intricate game mechanics, etc.), but if that potential is never acted upon, then it is, for lack of a better phrase, wasting space.  This also implies, as Sylvester suggests, that you can play with the player’s perception to elicit game content that isn’t actually there, something my colleagues and I have studied in our paper on The Illusion of Agency.  While we didn’t call it that, Sylvester references “Apophenia,” the cognitive bias of perceiving relationships which aren’t necessarily there, as the reason for the primacy of player perception.  Apophenia is not the only cognitive bias relevant to the study of games, but it certainly merits a great deal of attention.  Also, his comment regarding creating story-richness reminds me of some of the problems that the field of Artificial Intelligence faced when studying issues relating to Knowledge Representation.  Sylvester writes that we should “choose the minimum representation that supports the kinds of stories you want to generate,” a question that has beleaguered AI researchers for quite some time (although not necessarily in the context of games).

However, there are some things I disagree with, of which I will name two.

First, the statement that “anything in the Game Model that doesn’t copy into the Player Model is worthless,” to me sounds like too broad of a catch-all.  Perhaps this complaint is due to my desire to be precise, but the statement implies that everything in the Game Model ought to be in the Player Model (because if not, it is not worth putting in).  I think what he’s implying is that things intended (i.e.) to be perceived ought to have feedback to advertise them, otherwise they risk not making it into the person’s mental model of the game, which I think is fair to say.  However, I’m taking a risk in further refining his original thoughts, since he doesn’t explicitly define the Game Model; does he mean the mechanics of the game?  or all the supporting code?  the code that helps the game run smoothly is certainly not worthless, but it may not be perceived in the player’s mind.  Small tangent: part of the Game Model (I presume) includes the game’s AI, which we as developers actively try to shield from the player’s mind in the sense that we do not want them to know how the AI “is smart,” because we want to maintain the willing suspension of disbelief.

Second, the statement that a real complex system will “constantly break the Player Model Principle” (succinctly defined as: “The whole value of a game is in the mental model of itself it projects into the player’s mind.”) to me is a gross over-generalization of complex systems.  While it is true that complex system interactions make it difficult to tease apart causal relations (and transitively, to parse stories), I don’t think it will constantly break the Player Model Principle as long as the interactions with the story are actively mediated by the game itself.  I don’t think I necessarily disagree with Sylvester’s point, but I would rather have the problem of “from all the content I have, let me pick and choose what I think you would like best,” rather than the problem of “let me come up with content that I think you would like best.”  Thus, I say I disagree with him, but what I really want is to avoid discarding complex systems, because I think there is great potential in them for cool stories that we have not even come up with yet.  A computer is excellent at bookkeeping, and it has the capacity to store relationship information between artificial story agents living in a fictional world.  A  human, contrarily, must focus on content with a limited set of characters, because the author herself is a bottleneck in terms of keeping up with intricate social relations and happenings.  Complexity, for lack of a better word, is good.

The Simulation Dream is alive and kicking in me, and I think generous amounts of complexity is the way to do it.  The task is knowing where, when, and how much.

A view from the outside: my experience at the 2013 Communication, Rhetoric, and Digital Media Research Symposium

Being at the intersection of gaming, stories, computation, and cognition (as my blog’s headline suggests), I often have a research identity crisis, which I suspect (although this has yet to be confirmed) is a shared feeling with other researchers in my field.  This inner confusion does have its advantages; like a chameleon, I can float around different kinds of people and find some language in which to converse.  Such was the case at the 2013 Communication, Rhetoric, and Digital Media Research Symposium, where I was scheduled to talk about my research “Computational Models of Narrative and their Relation to Human Action,” as part of a panel on Gaming.  My “language” was cognition, and (I think) it was the most appealing aspect of my research to the community .  Regardless, there were several takeaways from the symposium that I wanted to share:

• Everyone is working on something related to everyone else

Something I perceived, which was most likely affected by my inexperience in CRDM, was the fact that everyone seemed to be working on topics that were relevant to everyone else.  What was remarkable is that seemingly disparate topics shared a common thread (which was often Dr. Carolyn Miller’s landmark paper: Genre as Social Action).  The concept of genre (I learned) is definitely a cross-cutting thing, and has useful taxonomic properties, as well as historical fingerprint qualities.  The fact that this common thread was woven, I think, was probably due in part to the excellent focus of the workshop.

• Support is overwhelming

Despite feeling I was overly technical, and that my presentation had too much jargon, the response was overwhelmingly positive.  I had several people throughout the presentation nod in agreement, had others tell me afterward that the presentation was well done, and even had one professor approach my advisor to congratulate me by proxy.  I’m sure I’m not the first one to say this, but having experienced it first hand, I think the following is worth repeating: communities certainly welcome outside perspectives.  I think it’s worth reaching out.

• Be precise!

Like Dr. Nicholas Taylor said before delivering his talk at the Gaming Panel, “all games researchers apologize before beginning their talks,” as a way of acknowledging that, because we’re all from such diverse fields, there’s bound to collision on some aspect of research.  I did, in fact, apologize for “possibly offending someone with my research.”  While, at the time, I sincerely doubted that my scientific advances would constitute an offense to anybody, it did help a bit.  Someone called out distinction of the virtual v. the real, alluding to the philosophical arguments relating to phenomenology.  All I meant to highlight was the distinction of video games and non-video games.  Specifically, I was talking about the challenge of borrowing non-interactive narrative concepts to analyze an interactive medium.  I should have been more precise.

• Keynote by Dr. David Herman

Dr. David Herman, Distinguished Professor at Ohio State University, was the evening keynote speaker.  His talk was probably the most valuable aspect of the whole symposium, because his research has been highly influential on my own and because we both see narratology as a cognitive science.  In essence, narratives are such a core part of our lives, that we use them for more than just entertainment; we use them for sense-making, for structuring our reality, and for guiding our future action.  These ideas merit their own set of posts, but my ideas aren’t completely formed yet.

All in all, a very fruitful symposium.  I hope I get to invade other types of academic gatherings to gain unique insights going forward!

The Role of Perception in Games

(This is a cross-post from my entry in the Liquid Narrative blog at NC State University)

This upcoming November, my colleague, Stephen Ware, will present a paper at the 2012 International Conference for Interactive Digital Storytelling.  This paper, on which several colleagues and myself worked, is titled “Achieving the Illusion of Agency,” which argues that complex drama management systems (see Roberts et al. for a great survey) are not necessary for dynamically creating appealing interactive narrative experiences.  As long as we can create an illusion of agency (using cheap tricks), it is enough.

I expected that this paper would ruffle some feathers – which is always a plus when you are engaging a community of worthy scholars.  The paper was accepted, for which I am happy.  However, the paper has (at least for me) a hidden agenda that was not picked up on by the reviewers of the paper.  I do not fault them, since the point is implicit, but here I will make it clear:

To find the future of game experiences, we have but one place to look: inward.

On the Value of ‘False’ Choices

One of the reviewers of the paper had this to say:

“It is difficult to address subjective aesthetic experiences such as ‘agency’ within a narrative in user testing,  so it is useful to have this well-designed study.

[…] but they reach very different conclusions than I would, ultimately arguing for the value of false choices.”

It’s an interesting point, which raises a philosophical question: what is a false choice?  That which is truthfully false? or that which is apparently false?

I argue the latter.  Since an interactive narrative is a controlled user experience, the player is not aware of false choices, because she never is told about the manner of the choices.  Given the existence of cognitive economy, the player assumes that what is being stated or presented is true, following Gricean pragmatics about quality of the “conversation” between her and the game.

To follow up, if it is not apparently false, does it matter?

Not to me.  Even if the choice is truly false, it is apparently true unless told otherwise, which is all that matters for her.

And so, to find the future of game experiences, we have but one place to look:  inward.  Perception, a sort of middleware for general cognition, is of utmost importance when creating games.  Ergo, to create experiences beyond what are currently imaginable in games, we need to tackle perception head on.

“You’re just making games.” – The Importance of Marketing in Our Controversial Science

(This is a cross-post from my entry in the Liquid Narrative blog at NC State University)

I have been on the receiving end of the title quote.  Often, I receive it verbatim.  Other times, I receive it in spirit.  As games researchers, we walk a fine line between art and science.  In my short academic career, I have found that justifying our work to scholars of the arts and the humanities is not as difficult as justifying our work to scholars in the sciences; not for lack of scholarly rigor in the arts and humanities, but rather because artists and humanists already know that it is important to look at games for what they represent, as well as their ubiquity and communicative power.  Our peers in the sciences, it seems, need a little more goading.  However, it is not their fault.  It is ours.

I admit, on the surface, it is difficult to imagine how the scientific process fits inside the machinery of video games. Games are primarily known for entertainment, and so, what possible science could there be?  What compounds the problem is that it is very easy to imagine that video games are a waste of a person’s time.  My anecdotal experience is very telling of this:

Exhibit A:  at a conference that was not focused on games, I had the very challenging experience of explaining my research to a community of scientists and non-scientists.  I had the opportunity to engage with some of the brightest minds the world has to offer…who (without fail) asked of my research:  “where is the science?”  I smiled every time, and tried as best I could to explain the complexity and the implications of my work.  Some got it (and were genuinely excited), others didn’t (and diplomatically dismissed the work).  Those who didn’t are especially memorable, for reasons I won’t go into here.

Exhibit B:  when I applied for the Graduate fellowship from the National Science Foundation, I received praise for the general quality of my application.  However, I got one specific bit of feedback that I will never forget:

“his proposed research topic – digital games – may be less critical for the society.”

My gut reaction to these experiences is always the same:  diplomatic anger, followed by personal disappointment.  It is not easy to get a Ph.D. in the first place, and it becomes more difficult to justify its worth when a community of scholars cannot see why games research is real science.  My mentors have often said that it is important to have thick skin and mental toughness for getting a Ph.D.  However, nothing really quite prepares you for a scientific community that routinely reminds you that “your problem is not worth solving.”

It’s easy to say:  “The scientists are bound by the shackles of the old guard.  They’re old, and close-minded.  They have lost touch with what is really important.  They don’t realize that games are a multi-billion dollar industry, eclipsing Hollywood and providing a pillar for the U.S. rebound economy.”  All these comments and many more are whispered in the halls of game research centers, and screamed in the heads of the scientists that study games.  However, I do not fault scientists for being skeptical; a healthy dose of skepticism is necessary for science.  It is our own fault.  I blame ourselves for not knowing enough marketing.  And I don’t mean marketing in terms of buzzwords (adding the terms “crowdsourcing” or “metaspectral” add fluff and will only impress marketers by trade),  I mean marketing as in “communicating science.”  We are not the only ones under this pressure; the government funding agencies have recently come under fire for funding basic science research that has no apparent immediate benefit or application.  This cascades into making video game funding especially hard to come by (who wants to fund a bunch of graduate students to make games?)

We games researchers are not doing enough to communicate the importance of our science, both to our scientific peers and the (much greater) non-scientific community.  And who could blame us?  We already know that it sucks to talk to scientists that look down upon your work.  As junior scientists, we seek experiences that help us grow professionally.  Pungent criticism stunts growth if you’re not prepared to handle it (and junior scientists, myself included, often aren’t).    This leads us to become a recluse of the general community – we prefer hanging out with our own crowd; publishing in blogs, conferences and journals devoted to games research, preparing posters that other games researchers will appreciate, and eventually establishing a network of games researchers.  This has to stop.

Rather than making the critical feedback personal and seeking the relative security of the games research community, I have set myself the goal of improving my science communication, actively seeking ways to engage and publish in other communities and I urge all games scientists to do the same.  The mindset of “they don’t understand and therefore they are close-minded” is not helpful nor productive.  Instead, ask yourself what I ask myself every time I encounter someone critical of my work:  “what am I not communicating that makes my audience think this is trivial or not worth doing?”  This becomes an issue of developing a deep understanding of your work, as well as anticipating potential criticisms, and knowing your audience, challenging aspects of research that are nonetheless do-able.  When someone tells you that you are just developing games, the correct response is:  “It may seem like it, but this is why it’s so much more than that: …”