Combining modeling and gaming for predictive analytics

Abstract

Our most significant security challenges involve people. While human behavior has long been studied, computational modeling of human behavior is early in its development. An inherent challenge in modeling of human behavior is efficient and accurate transfer of knowledge from humans to models, and subsequent retrieval. The simulated real-world environments of games present one avenue for knowledge generation and transfer. In this paper we describe our approach of combining modeling and gaming disciplines to develop predictive capabilities, using formal models to inform game development, and using games to provide data for modeling. We also describe the development of a prototype “Illicit Trafficking Game” that we used as a tool to exercise, evaluate and refine our approach. The resulting predictive capability combines human expertise and actions with computational modeling capabilities, resulting in a predictive capability that may approach the richness and diversity of human behaviors we wish to predict.

Background: modeling human behavior

Many of our most significant security challenges involve people. While human behavior has long been studied, there are recent advances in computational modeling of human behavior. There are a variety of modeling approaches used to predict and understand human behavior. Game theory [1, 2] is used to assess settings involving comingled objectives and outcomes. Bayesian networks [3] are widely used for decision support, and can also represent static, or time averaged, behaviors. System dynamics [4] models represent a time-dynamic structure among a set of entities, and have been developed to support the analysis of business organizations. A range of social phenomena have been modeled using agent based methodologies [5]. They include worker protest [6], cooperation, traffic [7], power systems [8], modeling scientific communities [9] and more.

Background: serious gaming

Serious gaming is a diverse field that deals with the use of games for primary purposes other than entertainment. Within the field of serious gaming, we draw a distinction based on the direction of knowledge transfer a game is designed to facilitate. The most prevalent forms of serious games are those designed to transfer knowledge to players for the purpose of education or training [10]. Another form of serious gaming is that which we refer to as “Analytical Gaming” (AG), and is focused primarily on tapping into the knowledge and ability of players.

The most notable examples of AG in the context of this paper are “wargame” simulation exercises in which participants use a variety of role playing methods to reason over potential outcomes of conflict. These exercises are essentially structured brainstorming activities carried out across a wide range of subject matter and level of detail. For exercises that consider physical outcomes of armed conflict, detailed models may be used to predict expected outcomes (for example, one can model, with high degrees of accuracy, the effectiveness of a particular type of weapon against a particular type of target). For exercises that are more concerned with social/behavioral issues (such as the expected reactions of a population to military actions and higher order effects), the use of computational modeling is less prevalent. While game-inspired modeling approaches (e.g., game theory) may be used in efforts to forecast social/behavioral outcomes, they are not always well integrated into games that involve human players.

Note that in both the case of educational gaming and of analytical gaming, a primary focus is indicated since educational/training games and analytical games are by no means mutually exclusive. To some extent, determining into which of these categories a game falls may be based on the intended player base. If players do not have knowledge of the subject matter in a particular game, it is more likely that the end result of playing the game will be to educate players about the subject matter than to gather new knowledge; however, this does not preclude the possibility of a player devising a clever new strategy (assuming that the game allows players some flexibility in this regard).

Background: an analytical gaming approach

In a previously published study [10], our analysis of 684 role playing games that involve the use of computerized models found that a majority (68%) focused on learning and training, while the remainder that focused on analysis and decision support had varying degrees of flexibility and social interaction. Among games that made use of computerized models, we found tight couplings between game, model, and problem domain. Motivated by these observations, we sought to examine ways in which gaming and modeling could be combined in fundamental ways that are more domain agnostic, and which simultaneously capitalize on the strengths of human players and computational capabilities (which we note include not just modeling, but also more mundane tasks such as movement of data between players).

Our AG approach [11, 12] consists of a methodology for game definition and execution, and a software engine – the Analytical Gaming Framework (AGF) – designed and implemented to support running of scenario-based games. Our AG approach and AGF software were developed with specific objectives in mind, including:

Leverage computational models at the framework level as abstract elements, such that a game implementation may leverage models that are relevant to a particular scenario

Foster creative play by human users

Map game and model data to flexible common formats to facilitate storage of game play histories, and leverage technology to reduce the burden on human participants associated with tracking these data

Support customization of games without requiring software modifications, and allow a “Game Master” (GM) role with the ability to directly manipulate the game scenario.

To meet these objectives, our AG approach includes deliberate identification of applicable models related to the subject domain, roles and objectives of players, and data parameters. Data parameters include parameters for both model (the explicit inputs and outputs of a particular computational model) and game (used to describe the current state of the game environment). The AGF captures player actions and data parameter values in a database, expressed as eXtensible Markup Language (XML) elements. Modern relational database engines have native support for XML data, and combine relational data storage with hierarchical data query functions. As a result, we are able to create a database that can store the entire history (player moves, communications, and model state) of any AGF-compatible game, without prior knowledge of the specific data types utilized in the game and model definitions. This flexibility is a key element, as it enables us to avoid crafting custom data handling code for specific scenarios, making our approach domain-agnostic.

Using models to drive game behavior

One of the most straightforward ways in which models and games may be integrated is the use of models to inform game design, and provide computational support for game execution by driving game behavior. In our prior work, we have experimented with use of two different types of models for this purpose.

System-dynamics

For well-understood systems (natural, mechanical, “process” models, etc.) we can use System-Dynamics models to control elements of the game environment. This requires mapping of the parameters of the model to parameters of the game and determining how player actions affect them. There may not be a 1:1 mapping of model parameters to game parameters, as games must be expressed in terms of things that players can control and/or observe, while inputs and outputs of computational models may deal with lower-level details or values that are driven by higher-order effects of player actions. Thus, an important part of game design when integrating with System-Dynamics models is to examine model parameters from the perspective of the player roles, objectives, and “handles” (items that players may directly influence) to derive the appropriate mapping of game parameters to model parameters and vice versa.

In our prior work, we constructed a prototype game to demonstrate use of System-Dynamics models related to climate change and crop production in a game designed to consider human interactions at a political level [13]. The underlying models were built and run using the STELLA®^a modeling system, which provides a simulation engine for executing models paired with a graphical interface for constructing, parameterizing, and running models as well as visualizing outputs.

Bayesian network

Probabilistic models can also be used to drive game behaviors. In this case, the models do not provide a deterministic outcome, but a range of possibilities. The game engine may then use these probabilities to drive stochastic changes to the game environment.

One example of such a model is the “IED Kill Chain” model [14], a Bayesian Network (BN) model that describes the process by which an attack using an Improvised Explosive Device (IED) is conceived and executed, and relates the actions of the attacker to potential indicators. The particular model in the above reference is based on a standard process model for IED development and deployment. This representation has a couple of advantages: A BN representation supports an evidence-based [15] approach to attaching data. As a stochastic model, a BN model can be empirically critiqued in a manner similar to statistical regression.

Methods for integration of a BN model into a game system include:

Allowing player actions to drive the values of “outcome” model nodes as “occurred” or “not occurred,” and using probabilities calculated at attached “evidence” nodes as a lookup table. The game engine reads the values of evidence nodes, and uses them as thresholds to determine (using a random number generator) whether or not to display an appropriate indicator to players. We describe this method in greater detail in [10].

Allowing player actions to drive the values of evidence nodes, and stochastically generating game events based on the probabilities calculated at outcome nodes of the model.

Using games to generate data for models

Another method of model-game integration is the use of games to inform the modeling process, particularly in the case of behavioral models. A goal of AG is to craft games that are sufficiently accurate and realistic that players will take actions similar to what they would do in a non-game setting.

As mentioned previously, the AGF includes data capture capabilities, recording all player actions during gameplay. These recorded game histories can be used to inform the modeling process via:

Observation of player behaviors at a micro level (individual player actions or small batches of actions), particularly when significant events happen in the game as indicated by changes in the model state: Creative player actions may be evaluated for realism and potential for real-world occurrence, and considered for potential inclusion in the formal game definition (e.g., iterative construction of the game definition, such that future sessions of the game include actions deemed feasible and interesting in previous sessions). If a model is intended to give a complete picture of potential behaviors, then a player action of this sort indicates a gap in the model coverage – by playing the game and engaging the creativity of human players, we identify potential courses of action for inclusion in behavioral models.

Examination of data at a macro level: Each time a game is played, it generates an additional unique game play history. As a library of these histories is built, it may be examined to look at patterns of behaviors. These can be compared to real data if available, to determine if game play activities parallel observed behaviors of people in similar, real scenarios. Another use of these data is calibration of behavioral models; in particular, agent based models (ABMs). If an ABM is designed such that the agents are able to take actions that correspond to the actions provided to players in the AG, the AG history data may then be used to compare the behaviors of computerized players and human players and, if desirable, modify the behavior of the agents to more closely resemble human behaviors.

An illicit trafficking game

In our most recent prototype development, the AG development team worked in conjunction with modeling experts at PNNL to develop an “Illicit Trafficking Game.” The analytical goal of this game is to serve as proof of concept for use of an AGF game as an experimental apparatus in support of efforts to model behaviors and dynamics of network formation for illicit transfer of nuclear materials and capabilities. This game focuses heavily on player communications and financial transactions, and includes roles for players as countries and corporations. In the game setting, one country is tasked with covertly obtaining items related to a nuclear weapons program while others seek to prevent this from happening.

We note that this game was devised as a way of exploring methodologies for domain-agnostic interactions across gaming and modeling disciplines. While illicit trafficking was selected as a relevant example of a problem domain for which analysis is desirable, the game was not primarily intended (in its prototypical form) to facilitate in-depth analyses of the subject matter of illicit trafficking.

The game was focused on understanding mechanisms of nuclear proliferation. The agents in the game included Countries and Companies. Each of these entities was initiated with Commodities and objectives. Some of the countries were proliferators, others non-proliferators. The countries had different levels of export control and different historical allies. The companies had primarily a profit motive, as well as the constraints associated with the particular host country in which they were set.

Commodities – We divided the commodities of interest into three categories with respect to their use in nuclear weapons programs. These were:

General use – items such as pencils, computers, and corn. These are widely used in economies and were considered separately.

Dual use – items that might be used for both nuclear power and for nuclear weapons

Focused use – commodities and technologies that are singularly used in support of nuclear weapons.

Countries – These entities have associated companies, and also have objectives. The objectives can include obtaining a nuclear weapon, denying other countries from obtaining a nuclear weapon, desire for nuclear power, and the need for funds to maintain the populace.

Companies – Companies are associated with a particular country, and can trade with other companies or countries. The primary motivation of a company is to stay in business (avoid bankruptcy). Companies are ‘gifted’ with an ability to produce goods at a particular rate.

Actions – The fundamental actions are communication, trade, taxation/tax payment, and intelligence gathering.

The game typically began with a series of communications. Countries with particular needs for types of commodities, and countries and companies with resources to offer, had to communicate to make their needs and resources known. Additionally, communication was required to set prices. Trade commenced after communication. Even with this simple structure, and a limited number of players (e.g. 4 countries and a comparable number of companies), there was sufficient structure to create trade networks and subterfuge for controlled (dual and focused use) goods.

The model was structured similarly with the game. The player roles, actions and entities in the model are the same as the game. The primary addition required is agent cognition. In the model, we used Bayesian networks to encode decision making procedures in the agents. The networks encoded what to acquire and from whom to acquire it.

The notion of using a game as an “experimental apparatus” (described in [16]) was our motivation for the co-development of the Illicit Trafficking Game and corresponding model. We started with the game definition, using paper prototyping to brainstorm how the game should flow in general. Once the game definition was sufficiently solidified, the player actions became the basis of the initial ABM definition. An unexpected benefit of having the game developed in advance of the model is that having the modelers play the game communicated the requirements and characteristics needed in the model.

Our goal in doing this synchronization of game and model development was to ensure that the game could ultimately generate data against which the model could be compared. Parameters in both the game and the model could vary, and changes in network formation and trafficking could be observed.

Observations

Even with a very simplified and limited game definition, results indicated promise for future large-scale efforts. For example:

The use of AGF for management of game communications and resources reduced the burden on human participants (in particular the GM). Table 1 shows raw volumes of messages sent and player actions completed during game sessions lasting approximately 1.5 hours each. We note that the purpose of these game sessions was constructive rather than evaluative. These numbers were not gathered as part of a formal scientific study of performance measurement, so there were uncontrolled factors that certainly have some bearing (for example, players’ level of familiarity with the game scenario and experience with game mechanics). Still, we can infer from this anecdotal evidence a likely and significant increase in activity volume enabled by computerizing aspects of game play.

Table 1 Data recording volume for manual and computerized games

Manual inspection and analysis of player communication histories showed patterns similar to published examples of real-world scenarios (in particular, one such example includes a technique of sending out multiple requests which end up converging on a single supplier, as described in an example of a country seeking large quantities of valves for a gas centrifuge program [17]).

Particular player behavioral patterns (for example, use of third parties to indirectly acquire goods) were used as guides for behavioral patterns that the ABM model agents should produce. We view this iterative game and model refinement as a key aspect of our AG approach.

In more than one game play iteration, players devised clever ways to obtain resources using methods not anticipated in the game definition. Using the flexibility of the AGF system, the GM was able to grant these player requests and seamlessly integrate them into the game. These player innovations are then the catalyst for circling back with the modeling team to evaluate potential inclusion of specific new actions in the ABM.

Issues

Through the course of developing the Illicit Trafficking Game prototype, we identified the following issues that serve as lessons learned to inform future AG development efforts:

As mentioned previously, the prototypical nature of this particular game and associated development cycle and resources necessitated a simple game design. Recall, however that above we stated that “a goal of AG is to craft games that are sufficiently accurate and realistic that players will take actions similar to what they would do in a non-game setting.” In this particular case, the systems and interactions being examined are quite complex, with social, political and economic factors all coming into play. This complexity mismatch relegates the results of gameplay in this instance to proof-of-concept status.

For similar reasons of resource constraints and time availability, our initial play-testing was often conducted by non-expert players (e.g., team members who were experts in the game process but not necessarily the subject matter of illicit trafficking).

Some software usability issues hampered ability of players to effectively communicate with each other (prompting erroneous sending of “public broadcast” messages that were intended to be private). This is particularly important to note as it relates to expert players – while game development team members may be able to tolerate and work-around mechanical issues, for expert players we want to minimize intrusions into the role-playing experience. In an ideal game situation, players should be able to assume their roles and focus all of their mental energy on the envisioned scenario for maximum immersion, rather than on the mechanics of gameplay which detract from the suspension of disbelief.

Additional model/game interactions and future work

Non-computational models

The term “model” can be very broad in definition and usage. While our discussions above pertain to integration of computational models with analytical games, there are also aspects of game-model interface that can be considered with non-computational models. For example, in the aforementioned Illicit Trafficking game, we made use of a process model that describes the nuclear fuel cycle at a high level as the basis for player objectives. The process model provides a road map with distinct waypoints that the would-be nuclear power must reach.

Knowledge transfer through iterative game development

We previously mentioned value in early and frequent collaboration between game and model development teams. In the particular case of the Illicit Trafficking Game, game play was used as an introductory mechanism to familiarize new members of the modeling team with the subject matter. Having model developers play a game specifically aimed at integration with an ABM helps them get the requirements and “feel” for the model.

In the case of expert players, we also found that using even a simplified prototype game was an effective way of kicking off brainstorming sessions. After playing the game for a period of time (e.g., one hour), an equal or greater time was devoted to roundtable discussion. The framework of the game gave players a sense of the boundaries of the associated model. Actions and factors that players perceived as missing from the game are prime candidates for game and model expansion. The engagement of the expert players (while not formally measured, interest levels were deemed high) also serves to validate that the model is faithful enough to the subject area to garner expert buy-in.

There is potential future work in further formalizing this relationship between game play and model brainstorming (for example, use of tools such as mind maps as guidelines for sketching out and spinning off additional modeling tasks during the post-game brainstorming sessions).

Characterizing and formalizing knowledge transfer to models

The types of knowledge transfer described thus far have included well-defined and automated transfer from models to games (e.g., using models to directly influence game behavior), as well as primarily manual transfer of knowledge from game players to models. There are opportunities to further define— and ultimately provide more automation of—the latter. For example, open questions include:

For System-Dynamics and Bayesian Network models, we can transfer predefined player actions as “evidence” to populate and drive the models. Can we also formalize and automate the transfer of player actions formulated “on the fly” during gameplay (i.e., through negotiation between player and GM of proposed new actions)? Generally, these would require modifications of the structure of models such as adding new nodes. It remains to be seen whether this could be reasonably accomplished in real time during game play, or only as an after-action activity. If added during game play, we posit that such modification would still require careful examination after game play when time constraints are relaxed.

Similarly, while the actions of individual players can be plugged in to existing models easily on a per-instance basis, a more complicated problem is that of aggregating behavioral patterns of multiple players across multiple game instances, and potentially modifying model definitions that are reliant on operating assumptions about potential behaviors. Here again, this task will likely require some degree of human interaction through examination of the data and application of domain-specific expertise. We do not presume that game play data can supplant the modeling process, but that it may augment the modeling process.

Automated players

While our previous discussions with respect to agent based modeling have focused on the use of a role-playing type game to facilitate development of seemingly separate ABMs, the synchronization of game and model definitions could enable those same ABMs to serve as the basis for automated players. These could be plugged into the game alongside human players (for example, to provide the appearance of more actors than we have players), or in a game played entirely by automated agents (in which case, the game environment essentially becomes a Monte Carlo simulation test bed, capitalizing on the inherent data capture of the AGF to record the history of agent actions and interactions).

Future work

We are currently embarking on and proposing new projects that apply the lessons learned during our prototype and proof-of-concept developments to current and meaningful questions about human behavior (such as effects of science and innovation policies on creation and growth of the U.S. scientific workforce, and decision factors that drive students to pursue higher education in science and technology). This ongoing and future work will include formal game-model integration as described in this paper, including all of the aforementioned types of models (System-Dynamics, Bayesian Networks, and Agent Based Models).

Endnote

^aSTELLA is a registered trademark of isee Systems. The STELLA system is described at http://www.iseesystems.com/softwares/Education/StellaSoftware.aspx

Disclaimers

The work describe in this article was performed in consultation with our organization’s Human Subjects Review Board and did not involve experimentation on human subjects.