research paper game development

• Open access
• Published: 09 November 2016

Game development software engineering process life cycle: a systematic review

• Saiqa Aleem   ORCID: orcid.org/0000-0002-3385-0613 1 ,
• Luiz Fernando Capretz 2 &
• Faheem Ahmed 3  

Journal of Software Engineering Research and Development volume  4 , Article number:  6 ( 2016 ) Cite this article

94k Accesses

64 Citations

5 Altmetric

Metrics details

Software game is a kind of application that is used not only for entertainment, but also for serious purposes that can be applicable to different domains such as education, business, and health care. Multidisciplinary nature of the game development processes that combine sound, art, control systems, artificial intelligence (AI), and human factors, makes the software game development practice different from traditional software development. However, the underline software engineering techniques help game development to achieve maintainability, flexibility, lower effort and cost, and better design. The purpose of this study is to assesses the state of the art research on the game development software engineering process and highlight areas that need further consideration by researchers. In the study, we used a systematic literature review methodology based on well-known digital libraries. The largest number of studies have been reported in the production phase of the game development software engineering process life cycle, followed by the pre-production phase. By contrast, the post-production phase has received much less research activity than the pre-production and production phases. The results of this study suggest that the game development software engineering process has many aspects that need further attention from researchers; that especially includes the postproduction phase.

1 Introduction

With the rapid advancement of computer technology, the significance of software engineering in our daily lives is increasing. It affects every aspect of our lives today, including working, living, learning, and education. A new and popular mode of entertainment and an important application of technology are software games, which have become increasingly accepted by people of all ages. In today’s culture, technology is easily accessible and has become more convenient; more and more people like to play games and are also becoming motivated to design their own games. Salen and Zimmerman ( 2003 ) defined “game is a software application in which one or more players make decisions by controlling game objects and resources, in the pursuit of its goal”. Software games are software applications that are installed on hardware devices such as video game consoles, computers, handheld devices, and Personal Digital Assistants (PDAs). Software games have now become a worldwide creative industry, but because of the multidisciplinary activities required, their development is a very complex task.

The multidisciplinary nature of the processes that combine sound, art, control systems, artificial intelligence (AI), and human factors, also makes the software game development practice different from traditional software development. However, despite the high complexity of the software engineering development process, the game industry is making billions of dollars in profit and creating many hours of fun (PWC, 2011–2014 outlook). The software game market throughout the world has grown by over 7–8 % annually and has reached sales of around $5.5 billion in 2015 (SUPERDATA 2015 ). Newzoo Game Market ( 2015 ) has also reported that the world-wide digital game market will reach $113.3 billion by 2018.

Creation of any game involves cross-functional teams including designers, software developers, musicians, script writers, and many others. Also, Entertainment Software Association (ESA) ( 2014 ); 2015 ) reports highlighted the latest trends about the software game industry. Therefore, game development careers have currently become highly challenging, dynamic, creative, and profitable (Liming and Vilorio, 2011 ). The ability to handle complex development tasks and achieve profitability does not happen by chance, but rather a common set of good practices must be adopted to achieve these goals. The game industry can follow the good and proven practices of traditional software engineering, but only a clear understanding of these practices can enhance the complex game development engineering process.

The computer game domain covers a great variety of player modes and genres (Gredler, 1995 ; Gredler, 2003 ; Rieber, 2005 ). The complexity of software games has posed many challenges and issues in software development engineering process because it involves diverse activities in creative arts disciplines (storyboarding, design, refinement of animations, artificial intelligence, video production, scenarios, sounds, marketing, and, finally, sales) in addition to technological and functional requirements (Keith, 2010 ). This inherent diversity leads to a greatly fragmented domain from the perspectives of both underlying theory and design methodology. The software game literature published in recent years has focused mainly on technical issues. Issues of game production, development, and testing reflect only the general software-engineering state of the art. Pressman ( 2001 ) states that a game is a kind of software that entertains its users, but game development software engineering faces many challenges and issues if only a traditional software-development process is followed (Kanode and Haddad, 2009 ; Petrillo et al., 2009 ). Some studies have proposed a Game Development Software Engineering (GDSE) process life cycle that provides guidelines for the game development software engineering process (Hendrick, 2014 ; Blitz game studio, 2014 ; McGrath, 2014 ; Chandler, 2010 ; Ramadan and Widyani 2013 ). However, the proposed GDSE process life cycle development phases do not ensure a quality development process.

A GDSE process is different from a traditional software development engineering process, and all phases of the proposed GDSE process life cycle can be combined into three main phases: pre-production, production, and post-production. The pre-production phase includes testing the feasibility of target game scenarios, including requirements engineering marketing strategies; the production phase involves planning, documentation, and game implementation scenarios with sound and graphics. The last phase post-production involves testing, marketing, and game advertising. Because of high competition and extreme market demand, game development companies sometimes reduce their development process so they can be first to market (Kaitilla, 2014 ). This reduction of the development process definitely affects game quality. Because of these types of complex project-management tasks, the game development software engineering process diverges from traditional software development. Therefore, it becomes important now to investigate the challenges or issues faced by game development organizations in developing good quality games. This systematic literature review is the first step towards identifying the research gaps in the GDSE field.

1.1 Related work

Managing GDSE process life cycle has become a much harder process than anyone could have initially imagined, and because of the fragmented domain, no clear picture of its advancement can be found in the literature. A systematic literature review provides a state of the art examination of an area and raises open research questions in a field, thus saving a great deal of time for those starting research in the field. However, to the best of the authors’ knowledge, no systematic literature review has been reported for GDSE process life cycle. Many researchers have adopted the systematic literature review approach to explore different aspects in software games. Boyle et al. ( 2012 ) conducted a systematic literature review to explore the engagement factor in entertainment games from a player’s perspective. In this study, 55 papers were selected to perform the systematic literature review. The study highlighted the different aspects of engagement factors with entertainment games; these include subjective feelings of enjoyment, physiological responses, motives, game usage, player loyalty, and the impact of playing games on a player’s life. Connolly et al. ( 2012 ) explored 129 papers to report the impacts and outcomes of computer and serious games with respect to engagement and learning by using the systematic literature review approach.

Another study also reported the importance of engagement in digital games by using a systematic literature review approach. Osborne-O’Hagan et al. ( 2014 ) performed a systematic literature review on software development processes for games. A total of 404 studies were analyzed from industry and academia and different software development adoption models used for game development were discussed. The findings of the study were that qualitative studies reported more agile practices than the hybrid approach. The quantitative studies used an almost hybrid approach. We also noted that lightweight agile practices such as Scrum, XP, and Kanban – are suitable where innovation and time to market is important. A risk-driven spiral approach is appropriate for large projects. Only one systematic study was performed related to research on software engineering practices in the computer game domain rather than GDSE process life cycle (Ampatzoglou and Stamelos 2010 ).

This study mainly review the existing evidence in the literature concerning the GDSE process research and suggest areas for further investigation by identifying possible gaps in current research. Furthermore, the aim of this study is to cover the state of the art for the GDSE process life cycle, and to accomplish this, an evidence-based research paradigm has been used. In the software engineering field, possible use of an evidence-based paradigm have been proposed by Dyba et al. ( 2005 ) and Kitchenham et al. (2004). The Systematic Literature Review (SLR) research paradigm constitutes the first step in an evidence-based paradigm research process, and its guidelines for performing systematic research are thoroughly described by Brereton et al. ( 2007 ) and Kitchenham ( 2004 ).

The rest of the paper is organized as follows: Section 2 provides the research background and Section 3 describes the methodology used for the systematic literature review as described by Breton et al. (2007). Section 4 presents the statistics for the primary studies, Section 5 answers various research questions, Section 6 discuss the external threats to validity, and, finally, Section 7 concludes the presentation.

2 Background

In the software development industry, software games are gaining importance because they are not only used for entertainment, but also for serious purposes that can be applicable to different domains such as education, business, and health care. Serious games are designed to have an impact on the target audience similar to entertainment games but they are combined seemingly with a practical dimension too. Both have to be attractive and appealing to a broad target audience (Alvarez & Michaud, 2008 ). Especially for serious games, along with their applicability to different domains, their revenue has also been increasing. Games software earned three times more revenue than any other software product in 2012 (Nayak, 2013 ).

Robin ( 2009 ) defines a development method as a systematized procedure to achieve the goal of producing a working product within budget and on schedule. A number of methodologies used for game development and design (Castillo 2008 ). The first is the waterfall method, which is also commonly used in traditional software development. Unlike game projects, once the pre-production phase is completed, production phase activities are performed in a “waterfall” manner. First, the activities are segregated based on functionalities and assets, and then they are assigned to their respective teams. The requirements team spent a significant amount of time in functionality definition and front-end activities, which implies a late implementation of level and mechanisms (Schwaber & Beedle, 2002 ). However, in the waterfall method, it is difficult to reverse any activity (Flood, 2003 ).

The second development methodology is the agile method that is commonly used for game development. These methods are highly iterative and not documentation-centric. The production phase is divided into small iterations and focusses on the most crucial features. During the beginning phase of each iteration, the whole team meets and sets clear objectives. At the end of each iteration, results are communicated to clients. These methods support different team cycles and dynamics through daily meetings. The most used agile methodologies in game development are extreme programming (XP), rapid prototyping, and Scrum (Godoy & Barbosa, 2010 ).

The unified development process (Kruchten, 2000 ) is another traditional SE method, which focusses more on analyzing requirements and converting them into functional software components. The requirement analysis document includes a definition of the game concept, use cases, and assets definitions (Schwaber & Beedle, 2002 ). The method includes five disciplines: requirements, analysis, design, implementation, and testing. The unified process is based on a philosophy of four key elements: iterative and incremental, use case-driven, architecture-centric, and risk-driven.

Kanode and Haddad ( 2009 ) stated that an important, but incorrect, assumption was made that GDSE follows the waterfall method. More recently, researchers have agreed that it must follow the incremental model (Munassar and Govardhan 2010 ) because it combines the waterfall method with an iterative process. A major concern, reported by Petrillo et al. ( 2009 ), was that very poor development methodologies are commonly used by developers for software creation in the game industry. The GDSE appears as a question in many forms attempting to determine what types of practices are used. However, there is no single answer to this question. Few researchers have explored GDSE practices and then tried to answer questions like the phases of the GDSE process life cycle. Blitz game studios ( 2014 ) proposed six phases for the GDSE process life cycle: Pitch (initial design and game concept), Pre-production (game design document), Main production (implementation of game concepts), Alpha (internal testers), Beta (third-party testers), and the Master phase (game launch). Hendrick ( 2014 ) proposed a five-phase GDSE process life cycle consisting of Prototype (initial design prototype), Pre-production (design document), Production (asset creation, source code, integration aspects), Beta (user feedback), and, finally, the Live phase (ready to play). McGrath ( 2014 ) divided the GDSE process life cycle into six phases: Design (initial design and game design document), Develop/redevelop (game engine development), Evaluate (if not passed, then redevelop), Test (internal testing), Review release (third-party testing), and Release (game launch). Another GDSE process life cycle proposed by Chandler ( 2010 ) consisted of four phases: Pre-production (design document and project planning), Production (technical and artistic), Testing (bug fixing), and, finally, the Post-production phase (post-mortem activities). The latest GDSE process life cycle in 2013 proposed by Ramadan and Widyani ( 2013 ) was based on the four GDSE process life cycles previously described. They proposed six phases: Initiation (rough concept), Pre-production (creation of game design and prototype), Production (formal details, refinement, implementation), Testing (bug reports, refinement testing, change requests), Beta (third-party testers), and Release (public release).

In traditional software engineering, the development phase usually involves activities such as application design and its implementation; the production phase is when the software actually runs and is ready for use. However, in the GDSE process lifecycle, the production phase includes the development process, which is the pre-production phase of the traditional software engineering process, and the production phase of traditional software engineering is actually the post-production phase of the GDSE process life cycle (Bethke, 2003 ). Therefore, the GDSE process life cycle is different from the traditional software engineering process, and many researchers have studied the challenges faced by this domain (Kanode and Haddad, 2009 ). The most prominent observation made in these studies is that to address the challenges faced by the GDSE process life cycle, more rigorous software engineering strategies must be used. Most researchers have explicitly compared the software engineering process with the GDSE process, but none of them has studied complete GDSE process life cycle and research topics under this domain in detail. This study will provide evidence on these topics and their differences from the traditional software engineering process. In this paper, the GDSE process phases were divided into three phases for basic understanding: Preproduction, Production, and Post-production. Efforts were made to classify these further based on studies found in the literature. The primary contribution of this paper is that it is the first SLR that addresses these GDSE process life cycle research topics and highlights the topics that need further attention by researchers.

In this work, the conceptual description of the SLR process presented by Kitchenham ( 2004 ) was used to investigate the research intensity for each phase of the GDSE process life cycle. Conceptually, SLR provides an opportunity for researchers to collect empirical evidence from the existing literature about a formulated research question. Although most authors followed the general SLR guidelines provided by Kitchenham ( 2004 ), there were slight variations in the description and presentation of the conceptual process layout. The generic SLR guidelines stated by Kitchenham ( 2004 ) are further elaborated here, and the overall process is described as a set of activities The research process has been adopted for this study described by Kitchenham and Charters ( 2007 ). There are mainly three phases of the review and the steps associated with each phase are shown in Fig.  1 .

3.1 Planning phase (Step 1–4)

This study started by selecting a topic, at which point the study objectives were also clearly defined and the boundaries of the domain delineated.

3.1.1 Selection of topic and research questions

Selecting a topic for SLR is of crucial importance because many factors such as individual or community interest, research gaps, and research impact contribute to shaping research questions on the topic. Our understanding of the GDSE process life cycle is continuously evolving (Kitchenham et al., 2010 ), and many areas in this field lack generalized evidence. It is critically important for the game industry to identify a quality-driven GDSE process. Several studies have investigated different phases of the GDSE process life cycle, but they do not offer systematic, comprehensive, and thorough methodological research specific to this topic.

In this review, studies from 2000 to 2015 will be explored to answer the following research questions:

Research Question (RQ1): What is the intensity of research activity on the GDSE process life cycle?

RQ2: What topics are being researched in the pre-production, production, and post-production phases?

RQ3: What research approaches are being used by researchers in the software game domain?

RQ4: What empirical research methods are being used in the software game domain?

The number of publications has been identified by the research group to address RQ1. A graphical representation has been used to represent the increase or decrease in the number of publications per year as a measure of research activity. To address RQ2, RQ3, and RQ4, each study selected has been affiliated to a research topic, to a certain approach, and to a specific methodology used for the research. Details of this classification into corresponding categories are discussed in section 3.2.4 .

3.1.2 Review team & protocol establishment

A multidisciplinary team is needed to perform a high-quality scientific SLR. To enhance the thoroughness and minimize the potential bias of a study, an SLR is normally undertaken by more than one reviewer. The SLR team for this review was made up of three people. Two people were designated as principal reviewers (Second expert report by American institute 2011). One person was also selected as the project leader to handle additional administrative tasks such as team communication, points of contact, meeting arrangements and documentation, task assignment and follow-up, and quality assurance. Table  1 details the tasks required for the SLR process and reviewer’s involvement and total time duration.

In order to ensure the review could be replicated and to reduce researcher bias a review protocol and it’s evaluation procedure was developed at step 3 and 4. The final review protocol is discussed in the following sections 3.2.1 to 3.2.4 (Steps 5–9 incl.).

3.2 Conducting phase (Step 5–9)

3.2.1 search strategy.

In the SLR, the search procedure is based on an online search. The search strategy for an SLR is a plan to construct search terms by identifying populations, interventions, and outcomes. Key terms are combined together to created different groups in order to form search strings. Each group comprise of terms that are either different forms of the same word, synonyms, or terms that have similar or related semantic meaning within the domain. Table  2 depicts the followed approach.

In order to retrieve different sets of relevant literature, four groups are designed. The main objective of this grouping is to find the literature that is the intersection of the groups as shown in Fig.  2 .

Selection of relevant studies

The search strategy was implemented by applying the “AND” and “OR”, where the “OR” operator is used within the Group and the “AND” is used between the groups. According to Table  2 , the following search string will capture the structure:

( Group 1: [Software game] OR [Digital game] OR [Video game] OR [Computer game] OR [Online Game] OR [Serious games] OR [Educational Games] OR [Learning Games])

( Group 2: [Development] OR [Advancement] OR [Steps] OR [Evolve] OR [Project])

( Group 3: [Life cycle] OR [Design] OR [Implementation] OR [Requirements Engineering] OR [Testing] OR [Evaluation] OR [Maintenance])

( Group 4: [Process] OR [Progression] OR [Method] OR [Model]).

Therefore, “ Software game development lifecycle process ”, “ Computer game development design process ” and “ video game testing process” are some examples of the search strings and similar way different search strings were formed in order to capture all relevant studies.”

To ensure that all relevant research concerning this area of study was reviewed, journals and conferences from 2000 to 2015 were covered, using as sources IEEE Explorer, ACM Digital Library, Science Direct Elsevier, Taylor & Francis, Google Scholar, and Wiley Publications. If the information required, as indicated on the form shown in Table  3 , was not explicitly present in the potential study, then that paper was peer-reviewed by all team members and, after discussion, validated for correctness. Otherwise, each paper was reviewed by one reviewer. Each study involved some general information and some specific information, as indicated on the form.

3.2.2 Pilot selection & data extraction

The research study selection and data extraction was based on the following coverage criteria:

Inclusion criteria for study

For SLR, articles and research papers from 2000 to 2015 were included, and to evaluate their suitability, the following criteria were analyzed:

The study should be thoroughly reviewed by at least one of the reviewers.

Only the following types of studies were considered: case studies, theoretical papers, and empirical analysis surveys.

The full text of the article should be available.

If any article identifies any challenges and problems in software games, that article is included as a review.

Studies that describe motivation for game application.

Study exclusion criteria

The following criteria were used to determine articles to be excluded:

Articles published on company Web sites.

Articles not relevant to the research questions.

Articles not describing any phase of the game development life cycle.

Study selection

This procedure involved two phases. In the first phase, an initial selection was made on the basis of the inclusion criteria and after reading the title, abstract, and conclusion of each article. In the second phase, if a particular article met the criteria, then the whole article was studied. One hundred forty-eight papers were identified after final selection, as shown in Fig.  3 . Table  4 shows the results found in each data source and Additional file 1 : Appendix A contains a full list of selected publications.

Study selection process

3.2.3 Quality criteria

In this research, quality guidelines were defined based on a quality instrument that was used to assign a quality score to each article as a basis for data analysis and synthesis. The quality instrument consisted of four sections: a main section containing a generic checklist applicable to all studies, and three other sections specific to the type of study.

The checklist was based upon SLR guidelines (Kitchenham, 2004 ) and was derived from Kitchenham ( 2004 ) and Second expert report by American institute (2011). The detailed checklist is shown in Table  5 . Some of the checklist items could be answered by “yes” or “no” and they also included a “partial” option. A value of 1 was assigned to “yes,” 0 to “no,” and 0.5 to “partial”; then the sum of the checklist values was used to assign a quality score to the study to assess document quality.

3.2.4 Data synthesis

For data synthesis the topics, research approaches and methods are classified and their classification details are listed below:

Classification of topics in the GDSE Life Cycle

This section includes a classification of the topics covered by each study with respect to the pre-production, production, and post-production phase issues involved. The 2012 ACM classification system was used for classification, which is the same method used by Cai and Card ( 2008 ). The proposed classification system has been adopted by many journals and conferences specifically for software engineering topics. The same classification was used here to classify the papers under study, and these were further fabricated based on studies found in the GDLC domain. Table  6 presents the selected classification schema.

Research approaches and methods classification

Research articles can be characterized based on their method and approach, as described by Glass et al. ( 2002 ). The main categories for scientific approach are descriptive (a system, tool, or method; a literature review can also be considered as descriptive studies), exploratory (performed where a problem was not clearly defined), and empirical (findings based on observation of its subjects). To evaluate new methods or techniques, three major empirical research methods are used: surveys, case studies, and experiments (Wohlin et al., 2000 ). Table  7 describes the three major empirical research types; Dyba and Dingsoyr ( 2008 ) also used the same type of empirical classification.

The data collected were statistically analyzed as follows:

To address RQ1, the number of studies published per year, whether journal articles or conference publications, and the number of publications on the GDLC hosted by each digital library.

To address RQ2, the major topics of the GDLC that were investigated in the software game domain.

To address RQ3 and RQ4, the research approach or method used by number of studies.

From Section  3.2.4 , data were tabulated and are presented in Additional file 2 : Appendix B.

3.3 Documenting (Step 10–12)

This step of the SLR describe conclusion, possible threats and limitations to the validity of this study. Authors believe that there is a chance that the word game was not part of the title of some studies, but that nevertheless they discussed game development. These studies may, therefore, have been excluded from the primary dataset by the search procedure. There are other threats that are also linked to a systematic literature review such as generalization and subjective evaluation (Shadish et al., 2002 ).

There are limitations to our results, although significant amounts of effort and time was spent to select the papers that were studied. More specifically, our search was limited to the academic databases. It is obvious from the results of RQ1 that developers prefer to submit their work on the blogs or forums. However, posts for different game forums and blogs cannot be included in a systematic literature review because they don’t fulfil the quality criteria used for the selection of papers. In addition, the exclusion of less-known journals and conferences from the Web of Science and the Scopus index might have led to a different dataset.

Another limitation of the study is the exclusion of Human-Computer Interaction (HCI) filed studies. In the phase of screening out, we found studies from HCI field such as (Plass-Oude Boss et al. ( 2010 )) for games but they didn’t focus on software engineering perspective. In short, we didn’t consider studies from HCI because they take non-functional requirements, and usability features into account. These methods help developers to evaluate software and they considered as an integral part of game development. However, due to the limited scope of the study, we excluded studies from HCI field.

Finally, the classification scheme might have altered the results if they were classified by a scheme, such as the waterfall model, instead of the ACM classification scheme. Despite these limitations, the results of our systematic literature review will be useful to game development organizations and developers of software games.

4 Results and Discussion

This section presents the results of statistical analysis of the data set discusses the findings concerning the RQs formulated in Section 3.1 . The characteristics of the data set are tabulated for better understanding. To trace the categories of each mapped study, the interested reader is referred to the Additional file 2 : Appendix B. A total 148 studies were collated and analyzed as part of this review. To identify GDSE process life cycle domain specific characteristics, the findings of this review will be compared to results from similar studies done by Cai and Card ( 2008 ), Glass et al. ( 2002 ), and Dyba and Dingsoyr ( 2008 ).

4.1 RQ1 What is the intensity of research activity on the GDSE process life cycle?

Table  8 clearly shows that GDSE process life cycle research intensity has increased during the last few years. Figure  4 showed an increase in GDSE process life cycle over time. The y -axis represents the number of publications in the form of a fraction and is calculated by taking year (i) ’s number of publications as the numerator and year (0) ’s number of publications as the denominator. From Table  8 , 2007 was taken as year (0) , and the first data point of the graph was calculated for year (1) i.e., 2008. Figure  4 shows the results up to 2015. Years are given on the x- axis.

Increase in GDSE process life cycle research activity

Figure  4 illustrates that during the last few years, research activity in the GDSE process life cycle domain has continuously increased and the number of publications in the GDSE domain has increased at a polynomial growth rate since 2005. During 2013, 2014 and 2015 the drop in research activity is noted. It seems obvious that most of the work related to GDSE research activity was not published on the selected sources for this study. During 2014, most of the research activities were seen on the game development associations/groups web sites, like DIGRA association and Gamastura, or game developers personal blogs.

Moreover, Fig.  5 shows the list of countries most active in GDSE process life cycle topics research. Looking at research activity based on countries, China now dominates GDSE process life cycle research, but its research into the game domain started only in 2010. In four years, China has come to dominate this area of research. Before 2010, the United States and the United Kingdom were dominant.

Research activity per country

Authors from North and South America have played a dominant role since 2004 and are still contributing in this area. Contributors in Europe also started research into the GDSE domain in 2007, but the Asian continent has dominated the GDSE domain since 2010. It can be visualized in Fig.  6 . The most popular venue for GDSE research publication is IEEE; it seems that IEEE accounts for the main bulk of publications (approximately 63 %), followed by Elsevier, Springer, and ACM.

Research activity by continent

4.2 RQ 2: What topics are being researched in the pre-production. Production and post production phase?

This section addresses the identification of main research topics in the GDSE process life cycle domain. Table  9 clearly suggests that most research has been conducted in the production phase, followed by the pre-production phase. On the other hand, the post-production phase has not attracted much research interest. These GDSE process life cycle topics are somewhat different than in software engineering because of two factors: first, the GDSE domain has special needs and priorities, and second, it is a young domain which requires more fundamental research in the area of requirements, development, and coding tools. When the GDSE domain becomes mature, then other areas in the field, like testing and verification, will attract the interest of researchers.

As mentioned earlier in Section 2 , games have specific characteristics, which the conventional software development process cannot completely address. In the past years, research on GDSE process life cycle topics has become more active because, unlike other software products, games provide entertainment and user enjoyment, and developers need to give more importance to these aspects. As a result, research about the pre-production phase has increased. The implementation phase is shorter than in the traditional software implementation process because of the short time to market. This production-phase research intensity has attracted the interest of many researchers, and maximum research activity has been reported because the GDSE domain requires efficient development and coding techniques. McShaffry ( 2003 ) also highlighted the importance of the production phase to counteract poor internal quality. There is much less research activity in the post-production phase than in the pre-production and production phases.

Figure  7 presents the growth of each GDSE process life cycle research topic since 2000. It is apparent that in the pre-production phase, the most researched topic is management of the game development process, followed in this order by production-phase development platforms, programming, and implementation topics. In the post-production phase, the marketing area attracted the largest amount of research interest. The state of the art research is the description of actual primary studies, and, therefore, they are mapped according to the research topics they addressed (Budgen et al., 2008 ). Next, a short description of each GDSE topic is presented along with a full reference list. A full reference list of all the studies included is presented in Additional file 1 : Appendix A.

GDSE process life cycle research topics

4.2.1 Pre-production phase

In the pre-production phase, most of the studies categorized under this topic address management issues during the GDSE process life cycle. The overall management of the game development process combines both an engineering process and creation of artistic assets. Ramadan and Widyani [S1] compared various game development strategies from a management perspective, and most studies like [S3], [S6], [S7], and [S8] have proposed frameworks for game development. Game development guidelines can be followed to manage GDSE process life cycle. The presence of agile practices in the game development processes is also highlighted by some studies. Tschang [S4] and Petrillo et al. [S17] highlighted the issues in the game development process and their differences from traditional software development practices. Management of development-team members and their interaction is critically important in this aspect.

Some studies [S10] and [S11] have provided data analytics and empirical analysis of the game development process and issues of interdisciplinary team involvement. Best management practices in the game development process must consider certain elements such as staying on budget, timing, and producing the desired output. To assess game quality, five usability and quality criteria (functional, internally complete, balanced, fun, and accessible) can be used, but a process maturity model specific to the game development process is still needed to measure these processes for better management and high performance.

Requirements specification

One of the main differences between the traditional software development process and GDSE process life cycle is the requirements phase. The game development process requires consideration of many factors such as emotion, game play, aesthetics, and immersive factors. In four studies, the authors have discussed the requirements engineering perspective to highlight its importance for the whole game-software development process. They discussed emotional factors, language ontology, elicitation, feedback, and emergence [S19], [S20], [S21], and [S22]. In particular, game developers must understand these basic non-functional requirements along with the game play requirements and incorporate them while developing games. The main challenges in requirements identification are a) communication between diverse background stakeholders, b) non-functional requirements incorporation with game play requirements, such as media and technology integration, and c) validation of non-functional requirement such as fun, which is very complex because it is totally dependent on the target audience. Callele et al. [S20] further fabricated a set of requirements based on emotional criteria, game-playing criteria (cognitive factors and mechanics), and sensory requirements (visual, auditory, and haptic). The requirements specification phase must address both the functional and non-functional requirements of game development.

Game system description language

Many description languages are currently used by developers, such as the UML model, agent-based methodologies, and soft-system methodologies. Quanyin et al. [S32] proposed the UML model for mobile games. They performed experiments and reported that it would be a good model for further development of games on the Android operating system. Shaker et al. [S33] extracted features of the Super Mario Brothers game from different levels, frequency sequences of level elements, and statistical design levels. Then, they analyzed the relationship between a player’s experience and the level design parameters of platform games using feature analysis modelling. Tylor et al. [S28] proposed a soft system methodology for initial identification of game concepts in the development process. The proposed approach can be used instead of a popular description language because it provides an overview of the game. Chan and Yuen [S30] and Rodriguez et al. [S31] proposed an ontology knowledge framework for digital game development and serious games modelling using the AOSE methodology. A system description language for games must be both intelligible to human beings and formal enough to support comparison and analysis of players and system behaviors. In addition, it must be production-independent, adequately describe the overall game process, and provide clear guidelines for developers.

Reusability

The existence of reusability of software (Capretz and Lee 1992 ) and development platforms in game development has been reported by some researchers, but to gain its full advantages, commonality and variability analysis must be done in the pre-production phase. This category addresses reuse techniques for game development software (Ahmed and Capretz, 2011 ). Neto et al. [S34] performed a survey that analyzed game development software reuse techniques and their similarity to software product lines. Reuse techniques in game development could reduce cost and time and improve quality and productivity. For reuse techniques, commonality and variability analysis is very important, similar to a software product line. Szegletes and Forstner [S36] proposed a reusable framework for adaptive game development. The architecture of the proposed framework consisted of loosely coupled components for better flexibility. They tested their framework by developing educational games. The requirements of the new game must be well aligned with the reusable components of the previously developed game.

Game design document

The Game Design Document (GDD) is an important deliverable in the pre-production phase. It consists of a coherent description of the basic components, their interrelationships, directions, and a shared vocabulary for efficient development. Westera et al. [S37] addressed the issue of design complexity in serious games by proposing a design framework. Furthermore, Salazar et al. [S38] highlighted the importance of a game design document for game development and provided an analysis of many available game design documents from the literature. They also compared their findings with traditional software requirement specifications and concluded that a poor game design document can lead to poor-quality product, rework, and financial losses in the production and post-production phases. Hsu et al. [S40] pointed out the issues of level determination in games and trade-off decisions about them. They proposed an approach to solve the trade-off decision problem, which is based on a neural network technique and uses a genetic algorithm to perform design optimization. Khanal et al. [S41] presented design research for serious games for mobile platforms, and Cheng et al. [S42] provided design research for integrating GIS spatial query information into serious games. Finally, Ibrahim and Jaafar [S43] and Tang and Hanneghan [S44] worked on a game content model for game design documents. Currently, GDD suffers from formalism and incomplete representation; to address this issue, the formal development of GDD is very important. A comprehensive GDD (focused on the game’s basic design and premises) results in good game quality.

Game prototyping

Game prototyping in the pre-production phase helps the developer to clarify the fundamental mechanics of the final game. Game prototyping in the preproduction phases is considered important because it is used to convey game and play mechanics and also helps in evaluating a game player’s experience. Reyno and Cubel [S49] proposed automatic prototyping for game development based on a model-driven approach. An automatic transformation generates the software prototype code in C++. De Silva et al. [S48] proposed community-driven game prototyping. The developer can approach the well-established community and focus on the technical stuff rather than starting from scratch. They used this approach for massive, multi-player online game development. Guo et al. [S50], Kanev and Sugiyam [S51], and Piesoto et al. [S52] proposed analysis of rapid prototyping for Pranndo’s history-dependent games, 3D interactive computer games, and game development frameworks respectively. Prototypes also help to identify missing functionality, after which developers can easily incorporate quick design changes. Model-driven or rapid-prototyping approaches can be used to develop game prototypes.

Design tools

Game design tools are used to help game developers create descriptions of effects and game events in detail without high-level programming skills. Cho and Lee [S56] and Segundo et al. [S57] proposed an event design tool for rapid game development and claimed that it does not require any kind of programming skill. These tools also enable reuse of existing components and reduce the total time of the game-creation process.

Risk management

In the game development domain, risk management factors do not receive much discussion by researchers. Risk management is very important from a project management point of view. Identifying risk factors in the game development process is also important. In game development, the project manager is the game producer and must bring together management, technical, and aesthetic aspects to create a successful game. The study by Schmalz et al. [S58] is the only study highlighting the issue of risk management in video development projects. They identified two risk factors during the development process: failure of development strategy and absence of the fun factor. In game development, important risk factors can be the development strategy, the fun factor or extent of originality, scheduling, budgeting, and others, but very low priority has been given by game developers to formal analysis of risk factors.

4.2.2 Production phase

Asset creation.

Asset creation in the production phase is the foundation stage where game developers create the various assets and then use them in the game implementation phase. In the production phase, the first step is to create assets for the game. One of these assets is audio creation. Migneco et al. [S63] developed an audio-processing library for game development in Flash. It includes common audio-processing routines and sound-interaction Web games. Minovic et al. [S65] proposed an approach based on the model drive method for user interface development, and Pour et al. [S64] presented a brain computer interface technology that can control a game on a mobile device using EEG Mu rhythms. For audio processing, open-source libraries are available, especially for games. Audio and interface design are examples of game assets.

Storyboard production

Storyboard production is the most important phase of game production; it involves development of game scenarios for level solutions and incorporation of artificial intelligence planning techniques for representing the various features of games through a traditional white board or flow chart. Pizzi et al. [S59] proposed a rational approach that elaborated game-level scenario solutions using knowledge representation and also incorporated AI techniques to explore alternative solutions by direct interaction with generated storyboards. Finally, Anderson [S61] presented a classification of scripting systems for serious and entertainment games, and Cai and Chen [S62] explored scene editor software for game scenes. Their approach was based on the OGRE.Net framework and C++ technology. Various scripting editors based on different technologies are available for game developers to produce storyboards. Some of this software helps to develop and edit scenes at different game levels, and other software helps by generating game levels automatically based on a description.

Development platforms

The studies classified under this category proposed various types of platforms for game development. Development platforms provide a ready-made architecture for server–client connectivity and help developers create games quickly. Open-source development platforms are available, but developers must customize them according to the required functionality. Peres et al. [S69] used a scrum methodology for game development, especially for multiple platforms, and implemented interfaces with social networking Web sites such as Twitter and Facebook. Jieyi et al. [S70] proposed a platform for quick development of mobile 3D games. First, the platform implemented the game template in two environments such as the Nokia series 60 platform and the Symbian OS. The second part of the process involved analysis of the entire game structure and extraction of game parameters for later customization. Finally, the tool could be used for game customization. Lin et al. [S] developed intelligent multimedia mobile games from embedded platforms. The proposed communication protocol was able to control the embedded platform to achieve the game usability and amusement. Mao et al. [S78] presented a logical animation platform for game design and development, and Alers and Barakova [S81] developed a multi-agent platform for an educational children’s game. Suomela et al. [S77] highlighted the important aspects of multi-user application platforms used for rapid game development. Some researchers have proposed a development platform similar to that described above that provides connectivity along with client customization and unnecessary updating of game servers.

Formal language description

Game semantics can be classified under formal language description for programming languages; only two studies were reported under this classification. The formal language description of game semantics provided a way to gain insight into the design of programming languages for game development. Mellies [S99] proposed a denotational prepositional linear logic for asynchronous games, and Calderon and McCusker [S100] presented their analysis of game semantics using coherence spaces. Very little work has been reported in this area, and very few game semantic descriptions of languages have been published.

Programming

Code complexity is increasing, especially in game development, because of the incorporation of complex modules, AI techniques, and a variety of behaviors. The most common programming languages used in game development are object-oriented structured languages such as Java, C, and C++. Studies classified under this category explored the programming aspect of game development. El Rhalibi et al. [S82] proposed a development environment based on Java Web Start and JXTA P2P technologies called Homura and NetHomura. It extends the JME game engine by facilitating content libraries, providing a new interface, and also providing a software suite that supports advanced graphical functionalities within IDE. The other two studies, done by Meng et al. [S84] and Chen and Xu [S85], also explored programming languages such as C++, DirectX, and Web GL and also Web Socket technologies for game development. Three studies by Yang et al. [S87], Yang and Zhang [S88], and Wang and Lu [S89] explored collision detection algorithms from a game logic aspect for software games, proposed A* search, and AI optimization-based algorithms.

Wang et al. [S83] proposed a framework for developing games based on J2ME technology. Zhang et al. [S92] also explored the effects of object-oriented technology on performance, executable file size, and optimization techniques for mobile games and suggested that object-oriented technology should be used with great care because the structured programming in game development is highly competitive. Bartish and Thevathayan [S86] and Fahy and Krewer [S90] analyzed the use of agents, finite state machines, and open-source libraries for the overwhelmingly complex process of multi-platform game development. Optimization techniques can be used with object-oriented programming to avoid unnecessarily redundant classes and inheritance, and to handle performance bottlenecks. These languages can be used across different development environments such as Android, iOS, Windows, and Linux. Researchers have proposed various approaches and tools for efficient game development. The integration of various development artefacts into games can also be done by generative programming, which also helps to achieve efficient development.

Game engine

A game engine is a kind of special software framework that is used in the production phase for creating and developing games. Game engines consist mainly of a combination of core functionalities such as sound, a physics engine or collision detection, AI, scripting, animation, networking, memory management, and scene graphs. Hudlicka [S108] identified a set of requirements for a game engine, including identification of the player’s emotions and the social interactions among game characters. This is the only study that has highlighted the important functionalities that an affective game engine must support. Another study by Wu et al. [S109] focused on game script engine development based on J2ME. It divided script engines into two types. The first type is the high-level script engine that includes packaging and refining of the script engine. The second type, the low-level script engine includes feature packages associated only with API. Four studies [S102], [S105], [S106], and [S107] explored the development of game engines on mobile platforms. Finally, Anderson et al. [S109] proposed a game engine selection tool. Recently, developers have been using previously developed or open-source game engines to economize on the game development process. Various researchers have proposed script-based, design pattern-based, and customizable game engines. In the GDSE process life cycle, game engines automate the game creation process and help a developer to develop a game in a shorter time.

Implementation

The foundations of game theory are used in game development because it is a branch of decision theory that describes interdependent decisions. Most studies in this category described different aspects of game implementation technologies on various types of platforms. They considered improving programming skills, 2D/3D animations and graphics, sound engineering, project management, logic design, story-writing interface design, and AI techniques. Various kinds of game implementation technologies can be found in the literature. Vanhatupa [S117] presented a survey of implementation technologies especially for browser games. The technologies explored in these studies are mainly server applications (application runtime, server-side scripting, and user interface and communication), client applications, databases, and architecture. The same study also described the accessories that can be used for implementation: application platforms, game engines, and various types of plug-ins. Abd El-Sattar [S112] proposed an interactive computer-based game framework for the implementation process. The framework includes steps from design through implementation that are based on game theory foundations and focus mainly on game models, Nash equilibrium, and strategies of play. The proposed framework includes architectural design and specifications, a proposed game overview, a game start-up interface and difficulty scaling, game modelling, the game environment and player control, and a free-style combat system.

Four studies [S113], [S114], [S119] and [S120] focused mainly on a development framework for mobile devices. Su et al. [S96] proposed a framework describing implementation of various main modules such as pressure movement, a thread pool based on the I/O completion port, and a message module. They also claimed that their proposed framework addressed the problems of traditional frameworks such as the single-server exhaustion problem, synchronization, and thread-pooling issues. Jhingut et al. [S114] discussed 3D mobile game implementation technologies from both single-player and multi-player perspectives. They also evaluated two game APIs: MDP 2.0 and M3G API. Finally, Kao et al. [S120] proposed a client framework for mobile devices that used a message-based communication protocol and reserved platform-specific data as much as possible. A few researchers have proposed agent-based frameworks as explored above for effective communication and synchronization between system components.

4.2.3 Post-production phase

Quality assurance.

Process validation plays an important role in assessing game quality. Collection and evaluation of process data from the pre-production phase through to the post-production phase either provide evidence that the overall development process produces a good-quality game as a final product or reveal that it cannot. Only two studies were reported under this classification. Stacey et al. [S122] used a story-telling strategy to assess the game development process. They carried out a two-year case study on a four-person development team. Astrachan et al. [S126] tried to validate the game creation process by analyzing the development process and design decisions made during development. The scope of studies done under this category was limited. The case studies were done for small teams and were limited to only one phase. In the game development process, quality assurance and process validation are critical components, and standard methodologies are lacking. More exploration is needed to provide deeper insights. QA for games needs more research attention because very little work has been reported.

Beta testing

Beta testing in games is used to evaluate overall game functionality using external testers. Beta testing is a kind of first public release for testing purposes by users. Game publishers often find it effective because bugs are identified by users that were missed by developers. If any desired functionality is missing, it must be addressed at this stage. This testing is performed before final game release. Under this classification, only four studies [S127], [S128], [S129], and [S130] were reported. Hable and Platzer [S129] evaluated their proposed development framework for mobile game platforms. Omar et al. [S128] evaluated educational computer games and identified two evaluation techniques: Playability Heuristic for Educational Games (PHEG) for expert evaluators, and Playability Assessment of Educational Games (PAEG) for real-world users. The proposed AHP-based Holistic Online Evaluation System for Educational Computer Games (AHP_HeGES) online evaluation tool can be used in the evaluation process. Very little work was reported in this category.

Heuristic-based testing

Heuristics are a kind of design guideline and can be used as an evaluation tool by game design developers or users. Basically, heuristics can be used in software engineering to test the interface. In games, evaluation must extend beyond the interface because other playability experiences also need evaluation such as the game story, play, and mechanics. Six studies [S132], [S133], [S134], [S146], [S147], and [S148] fell under this classification. Al-Azawi et al. [S132] proposed a heuristic testing-based framework for game development. The proposed framework divides testing by two types of user: experts and real-world users. Experts evaluate playability, game usability, and game quality factors. Users evaluate the game as a positive or negative experience. Omar and Jaafar [S133] and Al-Azawi et al. [S134] proposed a framework for the evaluation phase in the game development process. Heuristic testing can be done during the development process and repeated from the early design phase. It is perfect for game testing because after the game is implemented, if anything goes wrong, it will be too expensive to fix and will affect the project schedule. This topic also needs attention by researchers.

Empirical testing

Empirical testing approaches for the game-testing phase have been explored by only a few researchers. The approaches described by these researchers have focused only on final-product quality and usability. Only two studies were reported under this classification [S135] and [S136]. Escudeiro and Escudeiro [S135] used a Quantitative Evaluation Framework (QEF) to evaluate serious mobile games and reported that QEF frameworks are very important in validating educational games and final-product quality. Choi [S136] analyzed the effectiveness of usability-expert evaluation and testing for game development. Experimental results showed the importance of the validation process in game development. The scope of the studies done under this category was very limited, and other aspects of final-product testing have not been explored by researchers.

Testing tools

Development of testing tools has not been addressed by many researchers. Only one study [S137] was reported under this classification. Cho et al. [S137] proposed testing tools for black-box and scenario-based testing. They used their tool on several online games to verify its effectiveness. Tools for game testing facilitate the testing process. The proposed scope of study was also limited, and available testing tools have focused only on evaluation of online games.

After a game has been developed, the final step is marketing. Marketing of games includes a marketing strategy and a marketing plan. The marketing strategy is directly related to the choice of users and the types of games that are in demand. The marketing plan is something that a publisher can give to a distributor to execute on the publisher’s behalf. Some studies have been done from the perspective of game-user satisfaction that provide the baseline for the factors that game developers must take into account for new game development. Yee et al. [S142] described a game motivation scale based on a three-factor model that can be used to assess game trends. Three studies [S139], [S143], and [S144] empirically investigated the perspective of game-user satisfaction and loyalty. No study in the literature has directly captured a marketing strategy and a marketing plan for games.

4.3 RQ 3: What research approaches are being used by researchers in digital game domain?

Table  10 shows that most GDSE process life cycle studies have used an exploratory research approach. Figure  8 shows a comparison between the three research approaches used in the GDSE process life cycle domain. Figure  9 shows a comparison among the empirical research methods used in the GDSE process life cycle domain. The results suggest that surveys are most frequently used in GDSE domain research.

GDSE process life cycle research approaches

Empirical research approaches

These results were to be expected because the GDSE domain has only been growing since 2005; before 2010 more studies follow the descriptive approach because the field was young. After 2010, more studies have followed the exploratory approach because the domain has been maturing. More specifically, exploratory and descriptive approaches seem now to be equally used in the GDSE process life cycle domain.

4.4 RQ4: What empirical research methods are being used in the software games domain?

Table  11 depicts the results of the RQ4. The experimental empirical method is less used in the GDSE process life cycle domain, as mentioned by Wohlin et al. ( 2000 ), because carrying out formal experiments requires significant experience. The case-study method has also been used infrequently by researchers. The reason for this could be that case studies require project data obtained through various types of observations or measurements, and no research database or repository is available for the GDSE process life cycle domain. Finally, the survey method was more common than the other two methods. This is reasonable because the GDSE domain is still immature and researchers are trying to produce knowledge by questioning game users, experts, and others.

5 Conclusions

The GDSE process proved to be incredibly challenging as game technology including game platforms and engines changes rapidly and coding modules are used very rarely in the another game project. However, recent success of digital game industry enforces further stress along with game development challenges and highlights the need of good practices adoption for game development process. In order to find out the specific area in game development software engineering process for improvement, assessment of process activities needs to be performed. However, due to relatively young history and empirical nature of the field, there has not been any development strategies or set of best practices to carry out game development fully explored. This systematic literature review helps to identify the research gaps in game development life cycle.

The main objective of this research was to provide an insight into the GDSE process life cycle domain because, in the past, researchers have pointed out that it is different from the traditional software development process. To achieve this objective, a systematic literature review was performed, which confirmed the first step of the evidence-based paradigm. The results also confirmed that the GDSE process life cycle domain is different from the traditional software engineering development process and that research activity is growing day by day, attracting the interest of more researchers. This observation provided an evidence for developers they need to look for other important activities on top of software development process. This paper describes the various topics in the GDSE domain and highlights the main research activities related to the GDSE process life cycle. The research topics identified in the GDSE were a combination of different disciplines and together they complete the game development process.

The most heavily researched topics were from the production phase, followed by the pre-production phase. On the other hand, in the post-production phase, less research activity was reported. In the pre-production phase, the management topic accounted for the most publications, whereas in the production phase, the development platform, programming, and the implementation phase attracted the most researchers. The production phase has attracted more research because game developers focus more on implementation and programming because of the limited game-development time period. The post-production phase includes process validation, testing, and marketing topics. Very little research activity was observed in this area because the quality aspect of game development is not yet a mature field. These results highlighted that researcher’s need to pay attention especially in the phase of post-production.

In addition to research topics, more researchers used exploratory research methods; as for empirical research methods, surveys were carried out by more researchers than case studies and experiments. Overall, the findings of this study are important for the development of good-quality digital games. Rapid and continual changes in technology and intense competition not only affect the business, but also have a great impact on development activities. To deal with this strong competition and high pressure, game development organizations and game developers must continually assess their activities and adopt an appropriate evaluation methodology. The result of the study highlighted that use of a proper assessment methodology will help the organization identify its strengths and weaknesses and provide guidance for improvement. However, the fragmented nature of the GDSE process requires a comprehensive evaluation strategy, which has not yet been entirely explored. Finally, this kind of research work provides a baseline for other studies in the GDSE process life cycle domain and highlights research topics that need more attention in this area. The findings of this study will help researchers to identify research gaps in GDSE process life cycle and highlights areas for further research contributions. This study also is a part of a larger project aiming to propose a digital game maturity assessment model (Aleem et al. 2016a ). The identified important dimensions are developer’s perspective (Aleem et al. 2016b ), the consumer, the business (Aleem et al. 2016c ), and the process itself. It also reinforces the assertion that the GDSE process life cycle domain is a complex scientific domain comparable to the software engineering development process, and it needs more attention and consideration of different factors in game development software engineering process.

In short, this study presents a systematic literature review of the GDLC topics. Overall, the findings of this study are important for the development of good-quality digital games because they highlight the areas that needs research attention. The results of this study have shown that the fragmented nature of the GDLC process requires a comprehensive evaluation strategy, which has not yet been entirely explored. Finally, this kind of research work provides a baseline for other studies in the GDLC domain and highlights research topics that need more attention in this area. The findings of this study will also help researchers to identify research gaps in the GDLC and highlight areas for further research contributions.

Abbreviations

Game Design Document

Game Development Software Engineering (GDSE)

Quantitative Evaluation Framework

Systematic Literature Review

Ahmed, F., Capretz, L. F., 2011. A business maturity model of software product line engineering. Information Systems Frontiers, Springer, 13, 4, 543–560, DOI: 10.1007/s10796-010-9230-8

Aleem S, Fernando Capretz L, Ahmed F (2016). A Digital Game Maturity Model (DGMM), Entertainment Computing 17, 55-73. http://dx.doi.org/10.1016/j.entcom.2016.08.004

Aleem S, Capretz LF, Ahmed F (2016a) Critical Success Factors to Improve the Game Development Process from a Developer’s Perspective. J Comput Sci Technol 31(5):925–950

Article   Google Scholar  

Aleem S, Capretz LF, Ahmed F, (2016c). Empirical investigation of key business factors for digital game performance, Entertainment Computing, Vol. 13,pp. -25-36, http://dx.doi.org/ 10.1016/j.entcom.2015.09.001

Alvarez, J. Michaud, L., (2008). Serious Games: Advergaming, Edugaming, Training, and More, IDATE

Ampatzoglou A, Stamelos I (2010) Software engineering research for computer games: a systematic review. J Inf Softw Technol Elsevier 52(9):888–901.

Bethke E (2003). Game Development and Production. Wordware game developer's library. Wordware Pub, Plano. ISBN 978-0-585-44833-6

Blitz game studio, (2014). Project Lifecycle. Retrieved May 1, 2014 from http://www.blitzgamesstudios.com/blitz_academy/game_dev .

Boyle EA, Connolly TM, Hainey T, Boyle JM (2012) Engagement in digital entertainment games: A systematic review. Comput Hum Behav 28:771–780

Brereton P, Kitchenham B, Budgen D, Turner M, Khalil M (2007) Lessons from applying the systematic literature review process within the software engineering domain. J Syst Softw 80(4):571–583

Budgen D, Turner M, Brereton P, Kitchenham B (2008). Using mapping studies in software engineering. In: Proceedings of Psychology of Programming Interest Group (PPIG). Lancaster University, Lancaster. pp. 195–204

Cai KY, Card D (2008) An analysis of topics in software engineering. J Syst Softw 81(6):1051–1058

Capretz LF, Lee PA (1992) Reusability and life cycle issues within an Object-Oriented Design methodology (refereed). In: Ege R, Singh M, Meyer B (eds) Technology of Object-Oriented Languages and Systems. Prentice Hall, Englewood Cliffs, pp 139–150. ISBN 0-13-042441-2

Google Scholar  

Castillo T, Novak J, (2008). Game Development Essentials: Game Level Design. Delmar Cengage Learning. ISBN: 9781401878641

Chandler HM (2010) Game Production Handbook. Johns and Bartletts, Sudbury

Connolly TM, Boyle EA, MacArthur E, Hainey T, Boyle JM (2012) A systematic literature review of empirical evidence on computer games and serious games. Comput Educ 59:661–686

Dyba T, Dingsoyr T (2008) Empirical studies of agile software development: a systematic review. Information and Software Technology 50(9-10):833–859

Dyba T, Kitchenham BA, Jorgensen M (2005) Evidence-based software engineering for practitioners. Software Magazine. IEEE Computer Society 22(1):58–65

Entertainment Software Association (ESA), (2014). Essential facts about the Computer and Video Game Industry. Entertainment Software Association Available at: http://www.theesa.com/wp-content/uploads/2014/10/ESA_EF_2014.pdf . Accessed on 15 Oct 2015.

Entertainment Software Association (ESA), (2015). Essential facts about the Computer and Video Game Industry. Entertainment Software Association. Available at: http://www.theesa.com/wp-content/uploads/2015/04/ESA-Essential-Facts-2015.pdf . Accessed on 15 Oct 2015.

Flood K (2003) Game Unified Process: GameDev., Available at: http://www.gamedev.net/page/resources/_/technical/generalprogramming/game-unified-process-r1940 . Accessed June 12, 2015

Glass RL, Vessey I, Ramesh V (2002) Research in software engineering: an analysis of the literature. Inf Softw Technol 44(8):491–506

Godoy A, Barbosa E F, (2010). Game-Scrum: An approach to agile game development, Proceedings of SBGames 2010 Computing Track (I. S. F. SC, ed.), Sao Carlos, pp. 292–295, November 8–10, pp. 292–295.

Gredler M. E (1995). Designing and evaluating games and simulations. Behavioral Science. Wiley Online Library, 40, 1 (1995), 76–77

Gredler M. E (2003). Games and simulations and their relationship to learning. Handbook of Research on Educational Communications and Technology, Lawrence Erlbaum, Inc: Mahwah, NJ pp. 571–581.

Hendrick A (2014). Project Management for Game Development. Retrieved 20 May 2014, from http://mmotidbits.com/2009/06/

Kaitilla C (2014). How to learn Ouya Gamdev. Retrieved December 20, 2014, from http://gamedevelopment.tutsplus.com/articles/how-to-learn-ouya-gamedev--gamedev-9197 .

Kanode C M., Haddad H M (2009). Software engineering challenges in game development. In Proceedings of the 2009 Sixth International Conference on Information Technology: New Generations, (April 27–29, 2009), 260–265

Keith C (2010) Agile game development with Scrum. Addison-Wesley, Boston

Kitchenham B (2004). Procedures for performing systematic literature reviews. Joint Technical Report. Computer Science Department, Keele University, July 2004, 33 pages.

Kitchenham B, Charters S, (2007). Guidelines for performing systematic literature reviews in software engineering, Software Engineering Group, Keele University and Department of Computer Science, University of Durham, United Kingdom, Technical Report EBSE-2007-01, 2007

Kitchenham, B., Sjoberg, D.I.K., Brereton, P., Budgen, D., Dyba, T., Host, M., Pfahl, D., Runeson, P., 2010. Can we evaluate the quality of software engineering experiments? In Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, 1–8

Kruchten P (2000) The Rational Unified Process: An Introduction, 2nd edn. Addison Wesley Longman, Reading

Liming D, Vilorio D (2011). Work for play: Careers in video game development, Occupational Outlook Quarterly. Available at: http://www.bls.gov/careeroutlook/2011/fall/art01.pdf . Accessed on: 30 Sept 2015.

McGrath J (2014). The game development lifecycle: A theory for the extension of the agile project methodology. http://blog.dopplerinteractive.com/2011_04_01_archive.html . Accessed 1 May 2014

McShaffry M (2003) Game coding complete. Paraglyph Press, AZ, USA

Munassar N, Govardhan A (2010) A Comparison Between Five Models Of Software Engineering. International Journal of Computer Science Issues 7(5):94–101

Nayak M (2013). A look at the $66 billion video-games industry, Reuters, Retrieved June 2013 from http://in.reuters.com/article/2013/06/10/gameshow-e-idINDEE9590DW20130610 . Accessed 12 Sept 2014

Newzoo Game Market Research, 2015. Global Report: U.S. and China take half of $113 bn game market in 2018. Available at: http://www.newzoo.com/insights/us-and-china-take-half-of-113bn-games-market-in-2018/ . Accessed 2 Oct 2015

Osbourne-O'Hagan A, Coleman G, O'Connor RV (2014) Software development processes for games: a systematic literature review. In: 21st European Conference on Systems, Software and Services Process Improvement EuroSPI, Luxembourg, 25-27 June 2014

Petrillo F, Pimenta M, Trindade F, Dietrich C (2009) What went wrong? A survey of problems in game development. Computers in Entertainment. ACM Digit Library 7(1(13)):1–22

Plass-Oude Boss, D., Reuderink, B., Van De Laar, B.L.A., Gurkok, H, Muhl, C., Poel, M., Heylen, D.K.J., Nijholt, A. (2010), Human-Computer Interaction for BCI Games: Usability and User Experience. In Proceedings of the International Conference on CYBERWORLDS, A. Sourin (eds), IEEE Computer Society, Los Alamitos, 277–281

Pressman RS (2001) Software engineering: a practitioner approach, 5th edn. Wiley, New York

PWC global media and entertainment outlook 2011–2014, 2011. Available at http://www.pwc.com/gx/en/global-entertainment-mediaoutlook/territory-segments-digital-forecast-overview.jhtml . Accessed on 28 Jul 2013.

Ramadan R., Widyani Y, (2013). Game development life cycle guidelines. In Proceedings of 5th International Conference on Advanced Computer Science and Information Systems (ICACIS). IEEE Computer Society, Jakarta, Indonesia, (September 28–29, 2013) 95–100.

Rieber LP (2005) Multimedia learning in games, simulations and microworlds. Cambridge Handbook of Multimedia Learning. Cambridge University Press, UK, pp 549–567

Book   Google Scholar  

Robin S, (2009). Introduction to game development, 2nd edition. Charles River Media. ISBN-10: 1584506792

Salen K, Zimmerman E (2003). Rules of Play: Game Design Fundamentals. MIT Press, ACM Digital Library. p. 80. ISBN 0-262-24045-9

Schwaber K, Beedle M (2002) Agile Software Development With Scrum. Prentice-Hall, Upper Saddle River

MATH   Google Scholar  

Shadish WR, Cook TD, Campbell DT (2002). Experimental and Quasi-experimental Designs for Generalized Causal Inference. Houghton Mifflin Company, Boston

SUPERDATA 2015 Digital Good Measurement Blog. Worldwide digital games market. Available at: https://www.superdataresearch.com/blog/us-digital-games-market/ . Accessed 30 Dec 2015.

Wohlin C, Runeson P, Host M, Ohlsson MC, Regnell B, Wesslen A (2000) Experimentation in Software Engineering. Kluwer Academic Publishers, Boston/Dordrecht/London

Book   MATH   Google Scholar  

Download references

Authors’ contributions

SA designed the study and performed the review methodology, collected the data, analyzed the data and drafted the manuscript. LC helped to conceive the study and provided guidance to carry out the quality assessments of paper, reviewed the drafted manuscript and fine-tune the final draft. FA helped in study design, provided guidance to present the analysis and helped to draft the manuscript. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Author information

Authors and affiliations.

College of Technological Innovation, Zayed University, Abu Dhabi, 144534, United Arab Emirates

Saiqa Aleem

Department of Electrical & Computer Engineering, University of Western Ontario, London, ON, N6A 5B9, Canada

Luiz Fernando Capretz

Department of Computing Science, Thompson Rivers University, Kamloops, BC, V2C 0C8, Canada

Faheem Ahmed

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Saiqa Aleem .

Additional files

Additional file 1:.

Appendix A (DOC 114 kb)

Additional file 2:

Appendix B (DOC 313 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Cite this article.

Aleem, S., Capretz, L.F. & Ahmed, F. Game development software engineering process life cycle: a systematic review. J Softw Eng Res Dev 4 , 6 (2016). https://doi.org/10.1186/s40411-016-0032-7

Download citation

Received : 16 March 2016

Accepted : 25 October 2016

Published : 09 November 2016

DOI : https://doi.org/10.1186/s40411-016-0032-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Software Game
• Online game
• Systematic review
• Development software engineerong proces

Deep learning applications in games: a survey from a data perspective

• Published: 04 December 2023
• Volume 53 , pages 31129–31164, ( 2023 )

Cite this article

• Zhipeng Hu 1 ,
• Yu Ding   ORCID: orcid.org/0000-0003-1834-4429 2 ,
• Runze Wu 2 ,
• Lincheng Li 2 ,
• Rongsheng Zhang 2 ,
• Yujing Hu 2 ,
• Feng Qiu 2 ,
• Zhimeng Zhang 2 ,
• Kai Wang 2 ,
• Shiwei Zhao 2 ,
• Yongqiang Zhang 2 ,
• Ji Jiang 2 ,
• Yadong Xi 2 ,
• Jiashu Pu 2 ,
• Wei Zhang 2 ,
• Suzhen Wang 2 ,
• Ke Chen 2 ,
• Tianze Zhou 2 ,
• Jiarui Chen 2 ,
• Yan Song 2 ,
• Tangjie Lv 2 &
• Changjie Fan 2  

720 Accesses

Explore all metrics

This paper presents a comprehensive review of deep learning applications in the video game industry, focusing on how these techniques can be utilized in game development, experience, and operation. As relying on computation techniques, the game world can be viewed as an integration of various complex data. This examines the use of deep learning in processing various types of game data. The paper classifies the game data into asset data, interaction data, and player data, according to their utilization in game development, experience, and operation, respectively. Specifically, this paper discusses deep learning applications in generating asset data such as object images, 3D scenes, avatar models, and facial animations; enhancing interaction data through improved text-based conversations and decision-making behaviors; and analyzing player data for cheat detection and match-making purposes. Although this review may not cover all existing applications of deep learning, it aims to provide a thorough presentation of the current state of deep learning in the gaming industry and its potential to revolutionize game production by reducing costs and improving the overall player experience.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Forecasting Player Behavioral Data and Simulating In-Game Events

GAIM: Game Action Information Mining Framework for Multiplayer Online Card Games (Rummy as Case Study)

Reinforcement Learning in Tower Defense

Data availibility statement.

No datasets were generated or analysed during the current study

https://www.midjourney.com/app/

https://openai.com/blog/chatgpt

https://openai.com/blog/openai-codex

Created by Jason M. Allen using the generative Artificial Intelligence platform Midjourney . The painting became a news story when it won the 2022 Colorado State Fair’s annual fine art competition on 5 September, becoming one of the first AI generated images to win such a prize.

https://civitai.com/

https://github.com/AUTOMATIC1111/stable-diffusion-webui

https://en.wikipedia.org/wiki/Electronic_Arts

https://blog.playstation.com/archive/2018/07/09/how-milestone-created-the-breathtaking-bikers-paradise-that-is-strada-della-forra-in-upcoming-ps4-racer-ride-3

http://www.narakathegame.com/

http://www.rockstargames.com/GTAOnline

https://loomai.com

https://pinscreen.com

https://h.163.com

https://ro.my.games/

https://www.youtube.com/watch?v=5R8xZb6J3r0

https://beta.character.ai/

https://en.wikipedia.org/wiki/FIFA_(video_game_series)

https://en.wikipedia.org/wiki/NBA_2K

https://en.wikipedia.org/wiki/Elden_Ring

https://en.wikipedia.org/wiki/God_of_War_(franchise)

https://en.wikipedia.org/wiki/Grand_Theft_Auto_V

https://en.wikipedia.org/wiki/PUBG:_Battlegrounds

https://en.wikipedia.org/wiki/Minecraft

https://en.wikipedia.org/wiki/Honor_of_Kings

https://help.steampowered.com/en/faqs/view/571A-97DA-70E9-FF74

Adiwardana D, Luong MT, So DR et al (2020) Towards a human-like open-domain chatbot. arXiv preprint arXiv:2001.09977

Ariyurek S, Betin-Can A, Surer E (2019) Automated video game testing using synthetic and humanlike agents. IEEE Trans Games 13(1):50–67

Google Scholar  

Balduzzi D, Garnelo M, Bachrach Y et al (2019) Open-ended learning in symmetric zero-sum games. In: International conference on machine learning, pp 434–443

Bao S, He H, Wang F, et al (2021) Plato-xl: Exploring the large-scale pre-training of dialogue generation. arXiv preprint arXiv:2109.09519

Bergdahl J, Gordillo C, Tollmar K et al (2020) Augmenting automated game testing with deep reinforcement learning. In: IEEE conference on games, pp 600–603

Berner C, Brockman G, Chan B, et al (2019) Dota 2 with large scale deep reinforcement learning. CoRR arXiv:1912.06680

Blanz V, Vetter T (1999) A morphable model for the synthesis of 3d faces. In: 26th Annual conference on computer graphics and interactive techniques, pp 187–194

Booth J, Roussos A, Ponniah A et al (2018) Large scale 3d morphable models. Int J Comput Vis 126(2–4):233–254

MathSciNet   Google Scholar  

Bradley RA, Terry ME (1952) Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika 39(3/4):324–345

Brown K, Hamilton A (2016) Photogrammetry and star wars battlefront. In: GDC 2016: Game developer conference, San Francisco: UBM Tech

Brown N, Sandholm T (2018) Superhuman AI for heads-up no-limit poker: Libratus beats top professionals. Science 359(6374):418–424

Brown N, Sandholm T (2019) Superhuman AI for multiplayer poker. Science 365(6456):885–890

Brown T, Mann B, Ryder N et al (2020) Language models are few-shot learners. Advances in neural information processing systems 33:1877–1901

Browne CB, Powley E, Whitehouse D et al (2012) A survey of monte carlo tree search methods. IEEE Trans Comput Intell AI Games 4(1):1–43

Cao C, Weng Y, Zhou S et al (2013) Facewarehouse: A 3d facial expression database for visual computing. IEEE Trans Visual Comput Graph 20(3):413–425

Chen H, Liu X, Yin D et al (2017) A survey on dialogue systems: Recent advances and new frontiers. Acm Sigkdd Explorations Newsletter 19(2):25–35

Chen K, Wang Y, Zhang SH et al (2021) Mocap-solver: A neural solver for optical motion capture data. ACM Trans Graph (TOG) 40(4):1–11

Chen KT, Liao A, Pao HKK, et al (2008) Game bot detection based on avatar trajectory. In: International conference on entertainment computing, Springer, pp 94–105

Chen Z, Yi D (2017) The game imitation: Deep supervised convolutional networks for quick video game AI. CoRR arXiv:1702.05663

Chen Z, Xue S, Kolen J et al (2017) Eomm: An engagement optimized matchmaking framework. In: International conference on world wide web, pp 1143–1150

Chenghao L, Wang T, Wu C et al (2021) Celebrating diversity in shared multi-agent reinforcement learning. Ad Neural Inf Process Sys 34

Christiano PF, Leike J, Brown T, et al (2017) Deep reinforcement learning from human preferences. Ad Neural Inf Process Sys 30

Cideron G, Pierrot T, Perrin N, et al (2020) Qd-rl: Efficient mixing of quality and diversity in reinforcement learning. CoRR arXiv:2006.08505

Cohen MM, Massaro DW (1993) Modeling coarticulation in synthetic visual speech. In: Models and techniques in computer animation. Springer, pp 139–156

Creswell A, White T, Dumoulin V et al (2018) Generative adversarial networks: An overview. IEEE Signal Proc Mag 35(1):53– 65

Cudeiro D, Bolkart T, Laidlaw C et al (2019) Capture, learning, and synthesis of 3d speaking styles. In: IEEE/CVF conference on computer vision and pattern recognition, pp 10,101–10,111

Cully A, Demiris Y (2017) Quality and diversity optimization: A unifying modular framework. IEEE Trans Evol Comput 22(2):245–259

Dabral R, Mughal MH, Golyanik V, et al (2023) Mofusion: A framework for denoising-diffusion-based motion synthesis. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9760–9770

De Kegel B, Haahr M (2019) Procedural puzzle generation: A survey. IEEE Trans Games 12(1):21–40

Deng Q, Li H, Wang K, et al (2021) Globally optimized matchmaking in online games. In: ACM SIGKDD conference on knowledge discovery; Data mining, pp 2753–2763

Deng Y, Yang J, Xu S, et al (2019) Accurate 3d face reconstruction with weakly-supervised learning: From single image to image set. In: IEEE computer vision and pattern recognition workshops

Ding Y (2021) Full-body animation generation for expressive npcs. In: Game developers conference (GDC)

Dossa RFJ, Lian X, Nomoto H et al (2019) A human-like agent based on a hybrid of reinforcement and imitation learning. In: Proceedings of the 2019 international joint conference on neural networks (IJCNN’19), pp 1–8

Dyulicheva YY, Glazieva AO (2022) Game based learning with artificial intelligence and immersive technologies: an overview. In: Ceur workshop proceedings, pp 146–159

Edwards P, Landreth C, Fiume E et al (2016) Jali: an animator-centric viseme model for expressive lip synchronization. ACM Trans Graph (TOG) 35(4):1–11

Ekman P, Rosenberg E, Editors (1997) What the face reveals: Basic and applied studies of spontaneous expression using the facial action coding system (facs)

Eysenbach B, Gupta A, Ibarz J et al (2018) Diversity is all you need: Learning skills without a reward function. arXiv preprint arXiv:1802.06070

Fan Y, Lin Z, Saito J et al (2022) Faceformer: Speech-driven 3d facial animation with transformers. In: IEEE/CVF conference on computer vision and pattern recognition, pp 18,770–18,780

Galway L, Charles D, Black M (2008) Machine learning in digital games: a survey. Artif Intell Rev 29:123–161

Gao Y, Shi B, Du X et al (2021) Learning diverse policies in moba games via macro-goals. In: Ranzato M, Beygelzimer A, Dauphin Y, et al (eds) Advances in Neural Information Processing Systems, pp 16,171–16,182

Gong L, Feng X, Ye D et al (2020) Optmatch: Optimized matchmaking via modeling the high-order interactions on the arena. In: ACM SIGKDD International conference on knowledge discovery & data mining, pp 2300–2310

Goodfellow I, Pouget-Abadie J, Mirza M et al (2020) Generative adversarial networks. Commun ACM 63(11):139–144

Gorman B, Humphrys M (2007) Imitative learning of combat behaviours in first-person computer games. Proceedings of CGAMES pp 85–90

Gu X, Fan Z, Zhu S, et al (2020) Cascade cost volume for high-resolution multi-view stereo and stereo matching. In: IEEE/CVF Conference on computer vision and pattern recognition, pp 2495–2504

Hannun A, Case C, Casper J et al (2014) Deep speech: Scaling up end-to-end speech recognition. arXiv preprint arXiv:1412.5567

Harmer J, Gisslén L, del Val J et al (2018) Imitation learning with concurrent actions in 3d games. In: IEEE Conference on computational intelligence and games. IEEE, pp 1–8

Herbrich R, Minka T, Graepel T (2007) Trueskill™: a bayesian skill rating system. In: Advances in neural information processing systems, pp 569–576

Hingston P (2009) The 2k botprize. In: Lanzi PL (ed) IEEE Symposium on Computational Intelligence and Games

Ho J, Jain A, Abbeel P (2020) Denoising diffusion probabilistic models. Ad Neural Inf Process Sys 33:6840–6851

Holden D (2018) Robust solving of optical motion capture data by denoising. ACM Trans Graph (TOG) 37(4):1–12

Holden D (2019) A new era of performance capture with machine learning. In: Game developers conference (GDC)

Hong ZW, Shann TY, Su SY, et al (2018) Diversity-driven exploration strategy for deep reinforcement learning. Ad Neural Inf Process Sys 31

Hooshyar D, Yousefi M, Lim H (2018) Data-driven approaches to game player modeling: a systematic literature review. ACM Comput Surv (CSUR) 50(6):1–19

Hu EJ, Wallis P, Allen-Zhu Z et al (2021) Lora: Low-rank adaptation of large language models. In: International conference on learning representations

Huang TK, Lin CJ, Weng R (2004) A generalized bradley-terry model: From group competition to individual skill. Ad Neural Inf Process Sys 17:601–608

Hussein A, Gaber MM, Elyan E et al (2017) Imitation learning: A survey of learning methods. ACM Comput Surv (CSUR) 50(2):1–35

Jensen R, Dahl A, Vogiatzis G et al (2014) Large scale multi-view stereopsis evaluation. In: IEEE conference on computer vision and pattern recognition, pp 406–413

Ji Z, Lu Z, Li H (2014) An information retrieval approach to short text conversation. arXiv preprint arXiv:1408.6988

Justesen N, Bontrager P, Togelius J et al (2019) Deep learning for video game playing. IEEE Trans Games 12(1):1–20

Kang AR, Jeong SH, Mohaisen A et al (2016) Multimodal game bot detection using user behavioral characteristics. SpringerPlus 5(1):1–19

Karras T, Aila T, Laine S et al (2017) Audio-driven facial animation by joint end-to-end learning of pose and emotion. ACM Trans Graph (TOG) 36(4):1–12

Kazhdan M, Hoppe H (2013) Screened poisson surface reconstruction. ACM Trans Graph (ToG) 32(3):1–13

van Kesteren M, Langevoort J, Grootjen F (2009) A step in the right direction: Botdetection in mmorpgs using movement analysis. In: Proc. of the 21st Belgian-Dutch Conference on Artificial Intelligence (BNAIC 2009), pp 129–136

Knapitsch A, Park J, Zhou QY et al (2017) Tanks and temples: Benchmarking large-scale scene reconstruction. ACM Trans Graph (ToG) 36(4):1–13

Kotkov D, Pandey G, Semenov A (2018) Gaming bot detection: a systematic literature review. In: International conference on computational social networks, Springer, pp 247–258

Kwon H, Mohaisen A, Woo J et al (2016) Crime scene reconstruction: Online gold farming network analysis. IEEE Trans Inf Forensics Security 12(3):544–556

Lattas A, Moschoglou S, Gecer B et al (2020) Avatarme: Realistically renderable 3d facial reconstruction. In: IEEE/CVF Conference on computer vision and pattern recognition, pp 760–769

Lee E, Woo J, Kim H et al (2016) You are a game bot!: Uncovering game bots in mmorpgs via self-similarity in the wild. In: Ndss

Lee J, Marsella S (2006) Nonverbal behavior generator for embodied conversational agents. In: International conference on intelligent virtual agents, pp 243–255

Lehman J, Stanley KO (2011) Evolving a diversity of virtual creatures through novelty search and local competition. In: Annual conference on genetic and evolutionary computation, pp 211–218

Lewis JP, Anjyo K, Rhee T et al (2014) Practice and theory of blendshape facial models. Eurographics (State of the Art Reports) 1(8):2

Li J, Koyamada S, Ye Q et al (2020) Suphx: Mastering mahjong with deep reinforcement learning. CoRR arXiv:2003.13590

Li L, Wang S, Zhang Z et al (2021) Write-a-speaker: Text-based emotional and rhythmic talking-head generation. In: Proceedings of the AAAI conference on artificial intelligence, pp 1911– 1920

Li W, Chu X, Su Y et al (2022) Fingformer: Contrastive graph-based finger operation transformer for unsupervised mobile game bot detection. In: ACM web conference, pp 3367–3375

Li X, Wu Z, Meng HM et al (2016) Expressive speech driven talking avatar synthesis with dblstm using limited amount of emotional bimodal data. In: Interspeech, pp 1477–1481

Lin J, Yuan Y, Shao T et al (2020) Towards high-fidelity 3d face reconstruction from in-the-wild images using graph convolutional networks. In: IEEE/CVF conference on computer vision and pattern recognition, pp 5891–5900

Lin J, Yuan Y, Zou Z (2021) Meingame: Create a game character face from a single portrait. In: Proceedings of the AAAI conference on artificial intelligence, pp 311–319

Lin J, Li L, Yuan Y et al (2022) Realistic game avatars auto-creation from single images via three-pathway network. In: 2022 IEEE Conference on Games (CoG), IEEE, pp 33–40

Lin J, Chang J, Liu L et al (2023) Being comes from not-being: Open-vocabulary text-to-motion generation with wordless training. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 23,222–23,231

Lin Z, Lin J, Li L et al (2022) High-quality 3d face reconstruction with affine convolutional networks. In: ACM international conference on multimedia

Liu X, Jia H, Wen Y et al (2021) Towards unifying behavioral and response diversity for open-ended learning in zero-sum games. Ad Neural Inf Process Sys 34:941–952

Loiudice A (2018) How milestone created the breathtaking biker’s paradise that is strada della forra in upcoming ps4 racer ride 3. PlayStation Blog

Loiudice A (2019) A plague tale: Innocence overrun with megascans. Quixel https://quixel.com/blog/2019/5/7/a-plague-tale-innocence-overrun-with-megascans

McDonald J (2018) Robocalypse now: Using deep learning to combat cheating in ’counter-strike: Global offensive’. In: Game Developers Conference (GDC)

Mescheder L, Oechsle M, Niemeyer M et al (2019) Occupancy networks: Learning 3d reconstruction in function space. In: IEEE/CVF conference on computer vision and pattern recognition, pp 4460–4470

Miikkulainen R, Bryant BD, Cornelius R et al (2006) Computational intelligence in games. Principles and practice, Computational intelligence, pp 155–191

Milani S, Kanervisto A, Ramanauskas K et al (2023) Towards solving fuzzy tasks with human feedback: A retrospective of the minerl basalt 2022 competition. arXiv preprint arXiv:2303.13512

Mildenhall B, Srinivasan PP, Tancik M et al (2020) Nerf: Representing scenes as neural radiance fields for view synthesis. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part I, pp 405–421

Minka T, Cleven R, Zaykov Y (2018) Trueskill 2: An improved bayesian skill rating system. Tech Rep

Mou C, Wang X, Xie L et al (2023) T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. arXiv preprint arXiv:2302.08453

Mouret JB, Clune J (2015) Illuminating search spaces by mapping elites. CoRR arXiv:1504.04909

Muñoz-Avila H, Bauckhage C, Bida M et al (2013) Learning and game ai. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik

Noh Y, Jeong S, Kim HK (2021) Trading behind-the-scene: Analysis of online gold farming network in the auction house system. IEEE Trans Games 14(3):423–434

Oh J, Borbora ZH, Sharma D, et al (2013) Bot detection based on social interactions in mmorpgs. In: IEEE international conference on social computing, pp 536–543

Pao HK, Chen KT, Chang HC (2010) Game bot detection via avatar trajectory analysis. IEEE Trans Comput Intell AI Games 2(3):162–175

Park JJ, Florence P, Straub J et al (2019) Deepsdf: Learning continuous signed distance functions for shape representation. In: IEEE/CVF conference on computer vision and pattern recognition, pp 165–174

Park JS, O’Brien JC, Cai CJ et al (2023) Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442

Parker-Holder J, Pacchiano A, Choromanski K et al (2020) Effective diversity in population-based reinforcement learning. CoRR arXiv:2002.00632

Pelling C, Gardner H (2019) Two human-like imitation-learning bots with probabilistic behaviors. In: Proceedings of the 2019 IEEE conference on games (CoG’19), pp 1–7

Perez-Nieves N, Yang Y, Slumbers O et al (2021) Modelling behavioural diversity for learning in open-ended games. In: International conference on machine learning, PMLR, pp 8514–8524

Perolat J, De Vylder B, Hennes D et al (2022) Mastering the game of stratego with model-free multiagent reinforcement learning. Science 378(6623):990–996

Pham HX, Cheung S, Pavlovic V (2017) Speech-driven 3d facial animation with implicit emotional awareness: a deep learning approach. In: IEEE conference on computer vision and pattern recognition workshops, pp 80–88

Poznanski (2014) Visual revolution of the vanishing of ethan carter. The Astronauts https://www.theastronauts.com/2014/03/visual-revolution-vanishing-ethan-carter

Qi X, Pu J, Zhao S et al (2022) A gnn-enhanced game bot detection model for mmorpgs. In: Pacific-Asia Conference on Knowledge Discovery and Data Mining, Springer, pp 316–327

Raab S, Leibovitch I, Li P et al (2023) Modi: Unconditional motion synthesis from diverse data. In: IEEE/CVF conference on computer vision and pattern recognition, pp 13,873–13,883

Radford A, Kim JW, Hallacy C et al (2021) Learning transferable visual models from natural language supervision. In: International conference on machine learning, PMLR, pp 8748–8763

Ramesh A, Dhariwal P, Nichol A et al (2022) Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125

Richard A, Zollhöfer M, Wen Y et al (2021) Meshtalk: 3d face animation from speech using cross-modality disentanglement. In: IEEE/CVF International conference on computer vision, pp 1173–1182

Risi S, Preuss M (2020) From chess and atari to starcraft and beyond: How game ai is driving the world of ai. KI-Künstliche Intelligenz 34:7–17

Rombach R, Blattmann A, Lorenz D et al (2022) High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10,684–10,695

Ruiz N, Li Y, Jampani V et al (2022) Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. arXiv preprint arXiv:2208.12242

Rusu AA, Colmenarejo SG, Gulcehre C et al (2015) Policy distillation. CoRR abs/1511.06295 arXiv:1511.06295

Sadoughi N, Busso C (2018) Expressive speech-driven lip movements with multitask learning. In: IEEE International conference on automatic face & gesture recognition, pp 409–415

Sadoughi N, Busso C (2019) Speech-driven expressive talking lips with conditional sequential generative adversarial networks. IEEE Transactions on Affective Computing 12(4):1031–1044

Schönberger JL, Zheng E, Frahm JM et al (2016) Pixelwise view selection for unstructured multi-view stereo. In: European conference on computer vision, Springer, pp 501–518

Sherstinsky A (2020) Fundamentals of recurrent neural network (rnn) and long short-term memory (lstm) network. Physica D: Nonlinear Phenomena 404(132):306

Shi T, Yuan Y, Fan C et al (2019) Face-to-parameter translation for game character auto-creation. In: IEEE/CVF International conference on computer vision, pp 161–170

Shi T, Zou Z, Shi Z et al (2020) Neural rendering for game character auto-creation. IEEE Trans Patt Anal Mach Intell 44(3):1489–1502

Shi T, Zou Z, Song X et al (2020) Neutral face game character auto-creation via pokerface-gan. In: ACM International conference on multimedia, pp 3201–3209

Shi T, Zuo Z, Yuan Y et al (2020) Fast and robust face-to-parameter translation for game character auto-creation. In: AAAI Conference on artificial intelligence, pp 1733–1740

Shuster K, Xu J, Komeili M et al (2022) Blenderbot 3: a deployed conversational agent that continually learns to responsibly engage. arXiv preprint arXiv:2208.03188

Silver D, Huang A, Maddison CJ et al (2016) Mastering the game of go with deep neural networks and tree search. Nature 529(7587):484–489

Silver D, Hubert T, Schrittwieser J et al (2017) Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv preprint arXiv:1712.01815

Song Y, Wang J, Lukasiewicz T et al (2018) Diversity-driven extensible hierarchical reinforcement learning. CoRR arXiv:1811.04324

Starke S, Zhao Y, Komura T et al (2020) Local motion phases for learning multi-contact character movements. ACM Trans Graph (TOG) 39(4):54–1

Statham N, Jacob J, Fridenfalk M (2020) Photogrammetry for game environments 2014-2019: What happened since the vanishing of ethan carter. In: DIGRA 2020, Tampere, Finland, June 2–6, 2020

Su Y, Yao D, Chu X et al (2022) Few-shot learning for trajectory-based mobile game cheating detection. In: ACM SIGKDD Conference on knowledge discovery and data mining, pp 3941–3949

Sutton RS, Barto AG (2018) Reinforcement learning: An introduction. MIT press

Takahashi T, Tanaka K, Oka N (2018) Adaptive mixed-initiative dialog motivates a game player to talk with an npc. In: International conference on human-agent interaction, pp 153–160

Tao J, Xu J, Gong L et al (2018) Nguard: A game bot detection framework for netease mmorpgs. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 811–820

Tao J, Lin J, Zhang S et al (2019) Mvan: Multi-view attention networks for real money trading detection in online games. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 2536–2546

Tao J, Xiong Y, Zhao S et al (2020) Xai-driven explainable multi-view game cheating detection. In: 2020 IEEE Conference on Games (CoG), IEEE, pp 144–151

Tao J, Xiong Y, Zhao S et al (2023) Explainable ai for cheating detection and churn prediction in online games. IEEE Trans Games 15(2):242–251

Taylor S, Kim T, Yue Y et al (2017) A deep learning approach for generalized speech animation. ACM Trans Graph (TOG) 36(4):1–11

Thawonmas R, Kashifuji Y, Chen KT (2008) Detection of mmorpg bots based on behavior analysis. In: Proceedings of the 2008 international conference on advances in computer entertainment technology, pp 91–94

Thoppilan R, De Freitas D, Hall J et al (2022) Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239

Thurstone LL (1927) A law of comparative judgment. Psychol Rev 34(4):273

Touvron H, Lavril T, Izacard G et al (2023) Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

Vaswani A, Shazeer N, Parmar N et al (2017) Attention is all you need. Ad Neural Inf Process Sys 30

Vicini D, Speierer S, Jakob W (2022) Differentiable signed distance function rendering. ACM Trans Graph (TOG) 41(4):1–18

Vinyals O, Le Q (2015) A neural conversational model. arXiv preprint arXiv:1506.05869

Vinyals O, Babuschkin I, Czarnecki WM et al (2019) Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature 575(7782):350–354

Von Neumann J, Morgenstern O (2007) Theory of games and economic behavior. In: Theory of games and economic behavior. Princeton university press

Wang G, Xie Y, Jiang Y et al (2023) Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291

Wang K, Li H, Gong L et al (2020) Match tracing: A unified framework for real-time win prediction and quantifiable performance evaluation. In: Proceedings of the 29th ACM International conference on information & knowledge management, pp 2781–2788

Wang L, Hu K, Bai L et al (2023) Multi-scale control signal-aware transformer for motion synthesis without phase. arXiv preprint arXiv:2303.01685

Wang X, Song J, Qi P et al (2021) SCC: an efficient deep reinforcement learning agent mastering the game of starcraft II. In: the 38th International conference on machine learning, pp 10,905–10,915

Wang Y, Xue K, Qian C (2022) Evolutionary diversity optimization with clustering-based selection for reinforcement learning. In: Proceedings of the 10th international conference on learning representations (ICLR 2022). OpenReview.net

Ward I (2019) Call of duty: Modern warfare. PC, PS4, Xbox One

Whitley D (2001) An overview of evolutionary algorithms: practical issues and common pitfalls. Inf Softw Technol 43(14):817–831

Woo J, Kim HK (2012) Survey and research direction on online game security. In: Proceedings of the workshop at SIGGRAPH Asia, pp 19–25

Woo K, Kwon H, Kim Hc et al (2011) What can free money tell us on the virtual black market? In: Proceedings of the ACM SIGCOMM 2011 conference, pp 392–393

Wright M, Vorobeychik Y (2015) Mechanism design for team formation. In: AAAI Conference on artificial intelligence, pp 1050–1056

Wu Y, Chen Y, Xie X et al (2020) Regression testing of massively multiplayer online role-playing games. In: Proceedings of the 2020 IEEE international conference on software maintenance and evolution (ICSE 2020), IEEE, pp 692–696

Wurman PR, Barrett S, Kawamoto K et al (2022) Outracing champion gran turismo drivers with deep reinforcement learning. Nature 602(7896):223–228

Xing J, Xia M, Zhang Y et al (2023) Codetalker: Speech-driven 3d facial animation with discrete motion prior. arXiv preprint arXiv:2301.02379

Xu J, Luo Y, Tao J et al (2020) Nguard+ an attention-based game bot detection framework via player behavior sequences. ACM Trans Knowl Discov Data (TKDD) 14(6):1–24

Yannakakis GN, Togelius J (2018) Artificial intelligence and games. Springer

Yao Y, Luo Z, Li S et al (2018) Mvsnet: Depth inference for unstructured multi-view stereo. In: European conference on computer vision (ECCV), pp 767–783

Yariv L, Kasten Y, Moran D et al (2020) Multiview neural surface reconstruction by disentangling geometry and appearance. Ad Neural Inf Process Sys 33:2492–2502

Yariv L, Gu J, Kasten Y et al (2021) Volume rendering of neural implicit surfaces. Ad Neural Inf Process Sys 34:4805–4815

Yin QY, Yang J, Huang KQ et al (2023) Ai in human-computer gaming: Techniques, challenges and opportunities. Machine intelligence research pp 1–19

Zanetti S, Rhalibi AE (2004) Machine learning techniques for FPS in Q3. In: ACM SIGCHI International conference on advances in computer entertainment technology, pp 239–244

Zhang K, Riegler G, Snavely N et al (2020) Nerf++: Analyzing and improving neural radiance fields. arXiv preprint arXiv:2010.07492

Zhang L, Agrawala M (2023) Adding conditional control to text-to-image diffusion models. arXiv preprint arXiv:2302.05543

Zhang Y, Sun S, Galley M et al (2019) Dialogpt: Large-scale generative pre-training for conversational response generation. arXiv preprint arXiv:1911.00536

Zhao R, Li W, Hu Z et al (2023) Zero-shot text-to-parameter translation for game character auto-creation. arXiv preprint arXiv:2303.01311

Zhao S, Fang J, Zhao S et al (2022) T-detector: A trajectory based pre-trained model for game bot detection in mmorpgs. In: IEEE International conference on data engineering, pp 992–1003

Zheng Y, Xie X, Su T et al (2019) Wuji: Automatic online combat game testing using evolutionary deep reinforcement learning. In: International conference on automated software engineering, IEEE, pp 772–784

Zhou Y, Xu Z, Landreth C et al (2018) Visemenet: Audio-driven animator-centric speech animation. ACM Trans Graph (TOG) 37(4):1–10

Zhou Z, Fu W, Zhang B et al (2022) Continuously discovering novel strategies via reward-switching policy optimization. In: International conference on learning representations. OpenReview.net

Download references

Acknowledgements

We would like to thank Jiajun Bu and Weixun Wang for their generous and helpful discussions, as well as their constructive suggestions.

This work is supported by the Key Research and Development Program of Zhejiang Province (No. 2022C01011).

Author information

Authors and affiliations.

Zhejiang University, Hangzhou, 310058, China

Netease Fuxi AI Lab, Hangzhou, 310052, China

Yu Ding, Runze Wu, Lincheng Li, Rongsheng Zhang, Yujing Hu, Feng Qiu, Zhimeng Zhang, Kai Wang, Shiwei Zhao, Yongqiang Zhang, Ji Jiang, Yadong Xi, Jiashu Pu, Wei Zhang, Suzhen Wang, Ke Chen, Tianze Zhou, Jiarui Chen, Yan Song, Tangjie Lv & Changjie Fan

You can also search for this author in PubMed   Google Scholar

Corresponding authors

Correspondence to Yu Ding or Tangjie Lv .

Ethics declarations

Conflict of interest.

All authors declare that they have no conflicts of interest.

Ethics approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Hu, Z., Ding, Y., Wu, R. et al. Deep learning applications in games: a survey from a data perspective. Appl Intell 53 , 31129–31164 (2023). https://doi.org/10.1007/s10489-023-05094-2

Download citation

Accepted : 07 October 2023

Published : 04 December 2023

Issue Date : December 2023

DOI : https://doi.org/10.1007/s10489-023-05094-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Deep learning
• Interaction
• Virtual character
• Facial expression
• Animation production
• Intelligent NPCs
• Game industry
• Image processing
• Decision making
• Natural language processing
• User profile
• Find a journal
• Publish with us
• Track your research

The MIT Game Lab has a long history of innovative research that spans game culture to design practice. Below are some highlights of our work. See specific pages in the pull down menu for more detailed information on some of them.

Games & Colonialism

2017-: mikael jakobsson (co-pi), mary flanagan (co-pi).

What does the history of colonialism-themed board games look like, and what can it tell us about the situation today? What does it mean to present these historical moments in such a lavish form and then let these artifacts serve as centerpieces to gather around for social interaction at board game cafes, meetups, and conventions? This greater project includes Playing Oppression , a forthcoming book to be published by MIT Press; Orderly Adventures, in which we play and analyze games with colonialist themes; and Creating Counter-Colonial Games, a series of workshops to prototype games through cultural engagement with people affected by the colonialist endeavor.

Diversity and Inclusion in Esports and Gaming

2015-: t.l. taylor.

Launched in 2015, AnyKey was co-founded by Dr. T.L. Taylor and Dr. Morgan Romine (with support from Intel and ESL) with the goal of building a more inclusive and accessible esports world for all. Since that inception, AnyKey has become the leading advocacy organization for inclusion and diversity in competitive gaming & live streaming. It now operates as a non-profit and Dr. Taylor has transitioned from being the Director of Research to Chair of the Advisory Board.

Playful Augmented Reality Audio Design Exploration

2018-2019: mikael jakobsson & philip tan.

The focus of this project was to explore the potential of audio augmented reality (AR) technology through design research methodology, particularly exploratory prototyping. Going into this, we understood that location-based audio AR allows the potential for telling stories using the players lived world, through innovative use of the affordances of mobile phone devices, particularly GPS. We also considered audio AR as a means of playing with sound and music. Utilizing the accelerometers of the Bose AR glasses and connected mobile device, body movement can be linked to the players’ own music collection or a music generation engine.

Our work culminated in the discovery of what we are calling locomotion-based gameplay, a modification to the assumptions that occur when considering location- based gameplay. From our explorative work, locomotion-based gameplay arises from the affordances and limitations of current audio AR technology. It considers a person’s movement through space as important, more so than their precise location. Locomotion also implies whole body movement through gestures including the nod of a head and the tap of a toe, not just the vector of movement on a map. These gestures are ephemeral and contain multiple meanings dependent on context and mood. We believe more work in discovering this style of gameplay would be fruitful, for purposes of art and entertainment, for education and tourism, and other currently unforeseen use cases.

Intimate Worlds: Reading for Intimate Affects in Contemporary Video Games

2016-2018: kaelan doyle-myerscough (s.m., comparative media studies, 2018).

When we think of pleasures to be found in video games, we often talk about power, control, agency, and fun. But to center these pleasures is to privilege certain stories, players, actions and possibility spaces. This thesis uses the framework of intimacy to closely examine three games for their capacity to create pleasure in vulnerability, the loss of control, dependence on others, and precarity.

Drawing from Deleuzian affect theory and feminist, queer and posthuman theorists, I read for intimate affects in the formal, aesthetic, proprioceptive and structural elements of Overwatch , The Last Guardian and The Legend of Zelda: Breath of the Wild . Ultimately, I argue two points: that video games have a unique capacity to generate intimate affects, and that my games of choice push us to rethink our assumptions about what constitutes intimacy more broadly.

When All You Have is a Banhammer: The Social and Communicative Work of Volunteer Moderators

2016-2018: claudia lo (s.m., comparative media studies, 2018).

The popular understanding of moderation online is that moderation is inherently reactive, where moderators see and then react to content generated by users, typically by removing it; in order to understand the work already being performed by moderators, we need to expand our understanding of what that work entails. Drawing upon interviews, participant observation, and my own experiences as a volunteer community moderator on Reddit, I propose that a significant portion of work performed by volunteer moderators is social and communicative in nature. Even the chosen case studies of large-scale esports events on Twitch, where the most visible and intense tasks given to volunteer moderators consists of reacting and removing user-generated chat messages, exposes faults in the reactive model of moderation. A better appreciation of the full scope of moderation work will be vital in guiding future research, design, and development efforts in this field.

Recasting Player Two

2016-2017: mikael jakobsson, claudia lo, kaelan doyle myerscough, richard eberhardt & dozens of game designers from near and far.

The game development industry is currently on a mission to include “non-gamers” in local co-op games. Within the development community and among players, these games are said to have a “girlfriend mode.” Developers often cast player one as an expert player in their own image, while player two is a projection of antiquated gender stereotypes who has less agency and control over their play experience. This type of interaction would be better described as mansplaining in motion. This project consists of a series of workshops with participants from the game development community, where we not just discuss and spread awareness of what is problematic with current games and development practices, but work together in creating better alternatives.

OpenRelativity

2012-2016: gerd kortemeyer, philip tan, zach sherin, ryan cheu, & steven schirra.

OpenRelativity is an open-source toolkit to simulate effects of special relativity by varying the speed of light, developed to help people create, test, and share experiments to explore the effects of special relativity. Developed by the MIT Game Lab, it contains open-source code for public use with the free and paid versions of the Unity engine. The toolkit was developed during the creation of the game A Slower Speed of Light.

Gender and Systems of Warm Interaction in Digital Games

2014-2016: kyrie caldwell (s.m., comparative media studies, 2016).

This thesis considers the ways in which digital game mechanics (interactive inputs) contribute to games’ worldbuilding. In particular, this work is concerned with the replication and reinforcement of problematic gender roles through game mechanics that express positive (“warm”) interactions between characters, namely healing, protection, and building relationships. Characters who are women and girls are often associated with physical weakness, nature-based magic, and nurturing (or absent) personalities, whereas characters who are men and boys often protect women through physical combat, heal through medical means, and keep an emotional distance from others. Relationships built through game mechanics rely on one-sided agency and potential that renders lovers and friends as characters who exist to support the player character in achieving the primary goals of the game. Even warm interactions in games carry negative, even potentially violent and oppressive, representations and that there is thusly a need for design interventions on the mechanical level to mitigate violence in game worlds and the reinforcement of negative real world stereotypes.

E-sports Broadcasting

2014-2015: jesse sell (s.m., comparative media studies, 2015).

Situating e-sports broadcasting within the larger sports media industrial complex, discussing e-sportscasters, and investigating the economics behind the growing e-sports industry. E-sports, often referred to as competitive or professional gaming, stands as a prime example of the merger of work and play. A growing body of literature has started focusing on this pastime turned profession. As more professionals enter the scene and audiences continue to grow, e-sports broadcasters look towards older models of broadcasting to inform their own style. This reapplication of former conventions stands in contrast to the trends in the larger sports media trajectory. E-sports broadcasting is largely informed by traditional sports broadcasting, yet remains unable to fully capture the success of the global sports industry. On-air talent, once informed solely by traditional sportscasters are now looking to their fellow e-sportscasters to create something new. Revenue streams which form the foundation of the sports industry are making their way into e-sports but not in the way that one might expect.

MIT Overseer: Improving Observer Experience in Starcraft 2

2013-2015: philip tan & nick mohr.

The MIT Overseer project aims to provide casters with real-time graphics to help them tell the story of a game while it is in progress. We are trying out several different ways of displaying what happened in the past of a single game and anticipating what might happen in the near future.

Subversive Game Design and Meaningful Conflict

2012-2013: konstantin mitgutsch & steven schirra.

Movers & Shakers is used as a research tool to explore how a social component influences experiences in serious games. In addition subversive game design elements are implemented in the game to foster the players’ thinking process and to get them out of unquestioned routines. In the game the players are challenged to give up their prior egoistic goals to reach their common goal – to save the world. In a nutshell, the game shifts from a competitive to a collaborational gameplay – once the players start communicating.

Playstyle Motivation Explorations

2012-2013: todd harper.

Across game genres and communities, there are as many styles of play as there are players, from the highly competitive “powergamer” to the MMO fan who’s content to just take in the scenery and everything in between. Fugue is a game that asks: what are some of the motivations behind these styles? Do players reflect themselves — or a desired projection of the self — through playstyle? Or does the shape and context of the game itself direct such decisions? In order to explore these questions, we created a small, controlled gamespace that gives players an opportunity to express themselves via play.

Procedural Puzzles as a Design Tool for Games

2011-2013: alec thomson, clara fernández-vara.

Puzzledice is a set of tools and programming libraries for procedurally generating puzzles for a wide variety of games. These tools, developed by Alec Thomson at the MIT Game Lab from 2011-2013, are the result of multiple iterations of research and were used to develop Stranded in Singapore during the 2011 summer session of the Singapore-MIT GAMBIT Game Lab. Puzzledice is the result of research into how general purpose procedural puzzles can be used as a tool by game designers. These tools were designed to meet the following three goals: Solvability, Generality, and Usability.

Televisual Sports Videogames

2012-2013: abe stein (s.m., comparative media studies, 2013).

Over the three decade long history of sports videogame development, design conventions have lead to the emergence of a new sports game genre: the televisual sports videogames. These games, which usually simulate major professional or college sports, look and sound like television, and they use televised sports as a reference point for players. This thesis takes a critical look at how these televisual sports videogames are situated in the broader sports media industrial complex of North America, while also considering how the televisual design of these games is meaningful for fans of sports. Specifically, the text looks at how sports videogames reflect or reinforce dominant ideologies of hegemonic sports culture. Building on critical theories in sports studies, and through critical close readings of videogame texts, this thesis explores the relationship between sports television production, and sports videogames, with a focus on features that are found in both. Features such as introductory sequences, audio commentary, in-game advertising, news tickers, and instant replay are all commonly found in both sports television and sports videogames.

Purposeful Games for Social Change

2011-2012: konstantin mitgutsch & narda alvarado.

“ Purposeful Games for Social Change ” is a list of serious games designed to foster social change/justice or to raise awareness. This list was created in order to create the Purposeful Games Framework , a tool used to assess the cohesiveness in design of serious games.

Singapore-MIT GAMBIT Game Lab

The Singapore-MIT GAMBIT Game Lab was a six-year research initiative that addressed important challenges faced by the global digital game research community and industry, with a core focus on identifying and solving research problems using a multi-disciplinary approach that can be applied by Singapore’s digital game industry. The Singapore-MIT GAMBIT Game Lab focused on building collaborations between Singapore institutions of higher learning and several MIT departments to accomplish both research and development.

Research topics explored included artificial intelligence, game design, computer graphics and animation, character design, procedurally generated content, interactive fiction, narrative design, and video game production. Game prototypes were made for these research topics during the GAMBIT summer internship program, many of which won international recognition at festivals like IndieCade and the Independent Games Festivals held at GDC and GDC China, as well as academic conferences such as Meaningful Play and Foundations of Digital Games.

Game Development Research

By Henrik Engström

About the book

Digital games have become a ubiquitous part of our society. In many countries, game development is a substantial and important industry. Academic institutions provide programmes aimed at preparing students for careers in game development. Over the past 20 years, there has been great interest in game research. However, very few studies address game development. Instead, most studies have focused on: serious applications of games; analysis of games and players; or, social aspects of playing.

This book provides an overview of the scattered academic landscape of game development research. It highlights studies from a wide range of disciplines and raises arguments for game development to be understood as a complex activity that inherently includes elements of science, engineering, design and art. The consequences of this complexity need to be taken into account by research and/or academic programmes that have a disciplinary focus. There is otherwise the risk that the true nature of game development will not be understood.

Bibliographic Information

Engström, H. (2020) Game Development Research . The University of Skövde, ISBN 978-91-984918-7-6 (print), 978-91-984918-8-3 (digital), https://urn.kb.se/resolve?urn=urn:nbn:se:his:diva-19248 .

This book was created within the Game Hub Scandinavia 2.0 project . Project id: NYPS20201849. EU Interreg Öresund-Kattegat-Skagerrak .

About the Author

Henrik Engström is a professor at the University of Skövde. He holds a PhD in Computer Science from the University of Exeter and has conducted game-related research since 2001. His research focuses on the game development process and, in particular, its entangled, multidisciplinary nature. In a research context, Henrik has served as project manager, producer and developer in a number of game projects.

• © 2020 Henrik Engström. All rights reserved.
• Design: HTML5 UP

video game development Recently Published Documents

Total documents.

• Latest Documents
• Most Cited Documents
• Contributed Authors
• Related Sources
• Related Keywords

Towards a “Filipino” Video Game: Teaching Filipino Culture and Identity for Video Game Development

This paper uses the author’s experiences of teaching the Filipino module of a multidisciplinary video game development class as a case study in teaching Filipino culture and identity as an element of video game development. A preliminary definition of “Filipino video game” as having Filipino narratives and subject matter, made by Filipino video game developers, and catering to a Filipino audience, is proposed. The realities and limitations of video game development and the video game market in the Philippines is also discussed to show how the dominance of Western video game industry, in terms of the dominance of outsource work for Filipino video game developers and the dominance of non-Filipino video games played by Filipino players, has hindered the development of original Filipino video games. Using four Filipino video games as primary texts discussed in class, students were exposed to Filipinomade video games, and shown how these games use Filipino history, culture, and politics as source material for their narrative and design. Issues of how video games can be used to selfexoticization, and the use of propaganda is discussed, and also how video games can be used to confront and reimagine Filipinoness. The paper ends with a discussion of a student-made game titled Alibatas, a game that aims to teach baybayin, a neglected native writing system in the Philippines as a demonstration of how students can make a Filipino video game. The paper then shows the importance of student-made games, and the role that the academe plays in the critical understanding of Filipino video games, and in defining Filipino culture and identity.

Leaper: Video Game Development on Android Devices with Cross-Device Interaction

Holistic data-driven requirements elicitation in the big data era.

AbstractDigital transformation stimulates continuous generation of large amounts of digital data, both in organizations and in society at large. As a consequence, there have been growing efforts in the Requirements Engineering community to consider digital data as sources for requirements acquisition, in addition to human stakeholders. The volume, velocity and variety of the data make requirements discovery increasingly dynamic, but also unstructured and complex, which current elicitation methods are unable to consider and manage in a systematic and efficient manner. We propose a framework, in the form of a conceptual metamodel and a method, for continuous and automated acquisition, analysis and aggregation of heterogeneous digital sources that aims to support data-driven requirements elicitation and management. The usability of the framework is partially validated by an in-depth case study from the business sector of video game development.

Video Game Development in India: A Cultural and Creative Industry Embracing Regional Cultural Heritage(s)

Game development and production practices are complex and highly reflected processes—worldwide. This explorative article discusses video game development as a cultural and creative industry in India, including the industry’s history and introducing recent trends which indicate profound transformations—the use and implementation of Indian cultural heritage in game settings. In the rather short history of Indian game development as compared to other countries—a significant number of games made in India first were produced around 2010—the industry has already lived through big changes and challenges. This article aims at introducing Indian game development and argues that especially independent (so-called indie) game studios in their search for their own, region-specific game development and stand-alone characteristics for Indian games increasingly turn to what they perceive as their own cultural heritage, including, for example, elements from history, art (music, dance, dress styles, and others), and architecture.

Developer Credit: Para-Industrial Hierarchies of In-Game Credit Attribution in the Video Game Industry

Developer credit has been a contested issue in the video game industry since the 1970–80s, when Atari prevented its programmers from publicly claiming authorship for games they had developed. The negotiations over what constitutes a noteworthy contribution to video game development are ongoing and play out in the unregulated space of in-game credits. Here, some creators get top billing akin to film and television credits, while others struggle to be recognized for their work. By analyzing in-game credits of 100 contemporary games published between 2016 and 2020 and representing four major sectors of video game production (AAA, AA, indie, and freemium games as service), I identify recurrent patterns, such as opening credits, order, role descriptions (or lack thereof), and systematic credit omission, that both reinforce and subvert the notion of core development roles and above-the-line/below-the-line divisions.

Teach me to play, gamer! Imitative learning in computer games via linguistic description of complex phenomena and decision trees

Abstract In this article, we present a new machine learning model by imitation based on the linguistic description of complex phenomena. The idea consists of, first, capturing the behaviour of human players by creating a computational perception network based on the execution traces of the games and, second, representing it using fuzzy logic (linguistic variables and if-then rules). From this knowledge, a set of data (dataset) is automatically created to generate a learning model based on decision trees. This model will be used later to automatically control the movements of a bot. The result is an artificial agent that mimics the human player. We have implemented, tested and evaluated this technology. The results obtained are interesting and promising, showing that this method can be a good alternative to design and implement the behaviour of intelligent agents in video game development.

Making players care: The ambivalent cultural politics of care and video games

The relationship between care and video games is fraught. While the medium has the potential to allow players to meaningfully express and receive care, the cultural rhetorics that connect video games to care are often problematic. Even among game designers and scholars committed to social justice, some view care with hope and others with concern. Here, we identify and unpack these tensions, which we refer to as the ambivalent cultural politics of care, and illustrate them through three case studies. First, we discuss “tend-and-befriend games,” coined by Brie Code, which we read through feminist theorists Sarah Sharma and Sara Ahmed. Second, we address “empathy games” and the worrisome implication that games by marginalized people must make privileged players care. Lastly, we turn to issues of care in video game development. We discuss Telltale Games’ The Walking Dead series (2012–18) and strikingly care-less fan responses to recent employee layoffs.

Is Agile Not Agile Enough? A Study on How Agile is Applied and Misapplied in the Video Game Development Industry

Self-making and game making in the future of work.

Paid work has been a keystone of morality, normativity, sociality, and identity in capitalist societies. However, as the future of work is ushered in by technological unemployment, flexibilization, and precarity, researchers have to contend with what has been called the post-work society. The cultural industry of video game development provides a vantage into this future of work because it has been dominated since its inception by a vast field of informal creators and intermediaries, some of whom are paid for their activities while the vast majority are not. This chapter argues that gaming hobbies are exemplars of a conceptual shift in productive leisure not just as a mediating category in industrial capitalism but a mediating stage towards post-work.

Introduction: Why Game Production Matters?

In the introduction, the editors of this collection argue for the importance of game production studies at a point when the public awareness about the production context of video games has, arguably, never been higher. With so many accounts of video game development permeating player and developer communities, the task of game production studies is to uncover the economic, cultural, and political structures that influence the final form of games by applying rigorous research methods. While the field of game studies has developed quickly in the past two decades, the study of the video game industry and different modes of video game production have been mostly dismissed by game studies scholars and requires more attention.

Export Citation Format

Share document.

• University of Michigan Library
• Research Guides

Video Game Development

• Databases and Journals
• Book Resources
• Other Periodicals
• Internet Resources
• Twine Resources

Game Design and Development Journals

These are just a few academic journals and other sources that publish articles and reports in the field of computer and video game design and development. The section below on "General Databases for Video Game Design Articles" will point you toward places to search for more related articles.

• The Computer Games Journal The Computer Games Journal is a worldwide, peer-reviewed publication providing knowledgeable, well-written articles from academics and practitioners that are relevant to the games industry. It aims to encourage and promote research into games development and the games industry as a whole.
• EAI Endorsed Transactions on Serious Games EAI Endorsed Transactions on Serious Games is an open access, peer-reviewed scholarly journal focused on serious games applications, serious games technology, serious games analytics, user experience design and more.
• Entertainment Computing Entertainment Computing publishes original, peer-reviewed research articles in all aspects of digital entertainment, new media, entertainment computing, gaming, robotics, toys and applications among researchers, engineers, social scientists, artists and practitioners.
• IEEE Transactions on Games The IEEE Transactions on Games (T-G) publishes original high-quality articles covering scientific, technical, and engineering aspects of games.
• International Journal of Computer Games Technology The overall aim of the International Journal of Computer Games Technology is to bring together both the... more... less... research and development aspects of games technology covering the whole range of entertainment computing and interactive digital media.

General Databases for Video Game Design Articles

These databases include links to several full text articles and journals pertaining to computer and video game design and development, including programming, graphics, sound, and other related subjects.

• IEEE Xplore This link opens in a new window Provides full-text access to IEEE transactions, journals, magazines, conference proceedings, standards and online courses, plus related e-books.
• INSPEC This link opens in a new window Covers the world-wide literature (mainly journal articles and conference proceeding papers) in astronomy, physics, electronics and electrical engineering, computers and control, and information technology.
• Scopus This link opens in a new window International multi-disciplinary indexing & abstracting database for scientific, medical, technical, and social sciences.
• Web of Science This link opens in a new window Covering 12,000+ scholarly journals, plus selected books and published conference proceedings in all academic disciplines, the Web of Science Core Collection combines seven citation indexes which permit searching for articles that cite a known author or work, as well as searching by subject, author, journal, and author address.

Recommended Search Terms for Databases

Video games

Videogames (one word)

Electronic games

Computer games

Internet games

Online games

Game Design/Programming

Computer games AND design

Video games AND design

Computer games AND programming

Computer games AND development

Video games AND development

Computer games AND graphics

Video games AND graphics

Searching in Databases

A Generative Programming Approach for Game Development

Ieee account.

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.

Video games can have a positive impact on children—if designed with the right features, says new study

New research finds that digital games can contribute to the well-being of children, but game producers must design games to support positive outcomes.

Video games can contribute to and support the well-being of children if they are designed with the needs of children in mind, according to new research from UNICEF Innocenti and collaborating institutions, including New York University.

The report, the second in the Responsible Innovation in Technology for Children (RITEC) project, establishes that digital gaming companies and game designers can and should support the well-being of children through the games they produce and demonstrates that digital play has a particularly positive impact on children’s autonomy, competence, creativity, and identity when it responds to their deep interests, needs and desires. The project was co-founded by UNICEF and the LEGO Group and is funded by the LEGO Foundation.

NYU Steinhardt Professor Jan L. Plass led one of three studies appearing in the report. To understand the effect of digital play on children’s well-being, Plass and his colleagues implemented a 10-week intervention involving digital games including Lego Builder’s Journey and Rocket League with 255 children ages 8 through 12 years old. Among their findings, US children with a greater need for belonging relayed more positive feelings toward their social and parental relationships and an improved sense of autonomy. In Chile, children reported greater autonomy and improved parental relationships. In South Africa, the digital play generally supported well-being for all children.

“Many parents are concerned about screen time for their children, and this is the first study investigating the effect of digital play on well-being for this age group in the US, Chile, and South Africa,” said Plass, founding director of the Consortium for Research and Evaluation of Advanced Technology in Education (CREATE) Lab at NYU.

In addition to the experimental research led by Plass, observational research was conducted in the homes of 50 families over a 14-month period with children ages 6 through 12 in Australia, Cyprus, South Africa, and the UK. Lab-based research measured heart rate, eye tracking, facial expressions, and galvanic skin response (changes in sweat gland activity) of 69 children, ages 7 though 12, playing digital games in Australia.

The three studies found that games can support children’s senses of autonomy, competence, creativity, and identity, as well as help them regulate emotions and build relationships. But in order to support one or more of these aspects of well-being, games should consider certain features. For example, to support children’s sense of autonomy, a game could put them in control, allow them to make decisions about gameplay and encourage them to develop their own strategies to progress. Or to support creativity, a game could allow children to freely explore and solve problems or create their own characters or narratives.

“For decades, people have often assumed that playing video games is somehow bad for children, undermining their well-being. But our new study paints a far more complex picture—one in which these games can actually contribute to children’s well-being and positively support them as they grow up,” said Bo Viktor Nylund, director of UNICEF Innocenti.

Anna Rafferty, vice president of Digital Consumer Engagement, the LEGO Group, said: “This exciting research from UNICEF and leading academics shows that safe and inclusive digital play can have a profoundly positive impact on children’s lives. We’re proud to be partnering with like-minded organisations to understand how digital experiences can be designed in a way that puts children’s well-being first. These findings will empower responsible businesses to create a digital future where children are safe, nurtured and equipped to thrive.”

“The finding that digital play can enhance child well-being and meet children’s psychological needs, such as the need for connections to others, is especially important at a time when we are concerned about children’s mental health,” added Plass.

Said Nylund: “This research helps us understand not only how video games can impact the well-being of children, but also helps the producers and designers of these games understand what elements they can include to support children. We hope they will consider these findings as they design the games our children will be playing in the future.”

Also included in the RITEC project is the Joan Ganz Cooney Center; the Young & Resilient Research Centre at Western Sydney University; the Graduate Center, City University of New York; the University of Sheffield; and the Australian Centre of Excellence for the Digital Child.

The findings will be followed later this year by the launch of a guide to assist businesses to incorporate these findings into the games they design.

Press Contact

Suggestions or feedback?

MIT News | Massachusetts Institute of Technology

• Machine learning
• Social justice
• Black holes
• Classes and programs

Departments

• Aeronautics and Astronautics
• Brain and Cognitive Sciences
• Architecture
• Political Science
• Mechanical Engineering

Centers, Labs, & Programs

• Abdul Latif Jameel Poverty Action Lab (J-PAL)
• Picower Institute for Learning and Memory
• Lincoln Laboratory
• School of Architecture + Planning
• School of Engineering
• School of Humanities, Arts, and Social Sciences
• Sloan School of Management
• School of Science
• MIT Schwarzman College of Computing

Using ideas from game theory to improve the reliability of language models

Press contact :.

Previous image Next image

Imagine you and a friend are playing a game where your goal is to communicate secret messages to each other using only cryptic sentences. Your friend's job is to guess the secret message behind your sentences. Sometimes, you give clues directly, and other times, your friend has to guess the message by asking yes-or-no questions about the clues you've given. The challenge is that both of you want to make sure you're understanding each other correctly and agreeing on the secret message.

MIT Computer Science and Artificial Intelligence Laboratory (CSAIL) researchers have created a similar "game" to help improve how AI understands and generates text. It is known as a “consensus game” and it involves two parts of an AI system — one part tries to generate sentences (like giving clues), and the other part tries to understand and evaluate those sentences (like guessing the secret message).

The researchers discovered that by treating this interaction as a game, where both parts of the AI work together under specific rules to agree on the right message, they could significantly improve the AI's ability to give correct and coherent answers to questions. They tested this new game-like approach on a variety of tasks, such as reading comprehension, solving math problems, and carrying on conversations, and found that it helped the AI perform better across the board.

Traditionally, large language models answer one of two ways: generating answers directly from the model (generative querying) or using the model to score a set of predefined answers (discriminative querying), which can lead to differing and sometimes incompatible results. With the generative approach, "Who is the president of the United States?" might yield a straightforward answer like "Joe Biden." However, a discriminative query could incorrectly dispute this fact when evaluating the same answer, such as "Barack Obama."

So, how do we reconcile mutually incompatible scoring procedures to achieve coherent, efficient predictions? 

"Imagine a new way to help language models understand and generate text, like a game. We've developed a training-free, game-theoretic method that treats the whole process as a complex game of clues and signals, where a generator tries to send the right message to a discriminator using natural language. Instead of chess pieces, they're using words and sentences," says Athul Jacob, an MIT PhD student in electrical engineering and computer science and CSAIL affiliate. "Our way to navigate this game is finding the 'approximate equilibria,' leading to a new decoding algorithm called 'equilibrium ranking.' It's a pretty exciting demonstration of how bringing game-theoretic strategies into the mix can tackle some big challenges in making language models more reliable and consistent."

When tested across many tasks, like reading comprehension, commonsense reasoning, math problem-solving, and dialogue, the team's algorithm consistently improved how well these models performed. Using the ER algorithm with the LLaMA-7B model even outshone the results from much larger models. "Given that they are already competitive, that people have been working on it for a while, but the level of improvements we saw being able to outperform a model that's 10 times the size was a pleasant surprise," says Jacob. 

"Diplomacy," a strategic board game set in pre-World War I Europe, where players negotiate alliances, betray friends, and conquer territories without the use of dice — relying purely on skill, strategy, and interpersonal manipulation — recently had a second coming. In November 2022, computer scientists, including Jacob, developed “Cicero,” an AI agent that achieves human-level capabilities in the mixed-motive seven-player game, which requires the same aforementioned skills, but with natural language. The math behind this partially inspired the Consensus Game. 

While the history of AI agents long predates when OpenAI's software entered the chat in November 2022, it's well documented that they can still cosplay as your well-meaning, yet pathological friend. 

The consensus game system reaches equilibrium as an agreement, ensuring accuracy and fidelity to the model's original insights. To achieve this, the method iteratively adjusts the interactions between the generative and discriminative components until they reach a consensus on an answer that accurately reflects reality and aligns with their initial beliefs. This approach effectively bridges the gap between the two querying methods. 

In practice, implementing the consensus game approach to language model querying, especially for question-answering tasks, does involve significant computational challenges. For example, when using datasets like MMLU, which have thousands of questions and multiple-choice answers, the model must apply the mechanism to each query. Then, it must reach a consensus between the generative and discriminative components for every question and its possible answers. 

The system did struggle with a grade school right of passage: math word problems. It couldn't generate wrong answers, which is a critical component of understanding the process of coming up with the right one. 

“The last few years have seen really impressive progress in both strategic decision-making and language generation from AI systems, but we’re just starting to figure out how to put the two together. Equilibrium ranking is a first step in this direction, but I think there’s a lot we’ll be able to do to scale this up to more complex problems,” says Jacob.   

An avenue of future work involves enhancing the base model by integrating the outputs of the current method. This is particularly promising since it can yield more factual and consistent answers across various tasks, including factuality and open-ended generation. The potential for such a method to significantly improve the base model's performance is high, which could result in more reliable and factual outputs from ChatGPT and similar language models that people use daily. 

"Even though modern language models, such as ChatGPT and Gemini, have led to solving various tasks through chat interfaces, the statistical decoding process that generates a response from such models has remained unchanged for decades," says Google Research Scientist Ahmad Beirami, who was not involved in the work. "The proposal by the MIT researchers is an innovative game-theoretic framework for decoding from language models through solving the equilibrium of a consensus game. The significant performance gains reported in the research paper are promising, opening the door to a potential paradigm shift in language model decoding that may fuel a flurry of new applications."

Jacob wrote the paper with MIT-IBM Watson Lab researcher Yikang Shen and MIT Department of Electrical Engineering and Computer Science assistant professors Gabriele Farina and Jacob Andreas, who is also a CSAIL member. They presented their work at the International Conference on Learning Representations (ICLR) earlier this month, where it was highlighted as a "spotlight paper." The research also received a “best paper award” at the NeurIPS R0-FoMo Workshop in December 2023.

Share this news article on:

Press mentions, quanta magazine.

MIT researchers have developed a new procedure that uses game theory to improve the accuracy and consistency of large language models (LLMs), reports Steve Nadis for Quanta Magazine . “The new work, which uses games to improve AI, stands in contrast to past approaches, which measured an AI program’s success via its mastery of games,” explains Nadis. 

Previous item Next item

Related Links

• Article: "Game Theory Can Make AI More Correct and Efficient"
• Jacob Andreas
• Athul Paul Jacob
• Language & Intelligence @ MIT
• Computer Science and Artificial Intelligence Laboratory (CSAIL)
• Department of Electrical Engineering and Computer Science
• MIT-IBM Watson AI Lab

Related Topics

• Computer science and technology
• Artificial intelligence
• Human-computer interaction
• Natural language processing
• Game theory
• Electrical Engineering & Computer Science (eecs)

Related Articles

Reasoning and reliability in AI

Explained: Generative AI

Synthetic imagery sets new bar in AI training efficiency

Simulating discrimination in virtual reality

More mit news.

Janabel Xia: Algorithms, dance rhythms, and the drive to succeed

Read full story →

Jonathan Byrnes, MIT Center for Transportation and Logistics senior lecturer and visionary in supply chain management, dies at 75

Researchers develop a detector for continuously monitoring toxic gases

The beauty of biology

Navigating longevity with industry leaders at MIT AgeLab PLAN Forum

Jeong Min Park earns 2024 Schmidt Science Fellowship

• More news on MIT News homepage →

Massachusetts Institute of Technology 77 Massachusetts Avenue, Cambridge, MA, USA

• Map (opens in new window)
• Events (opens in new window)
• People (opens in new window)
• Careers (opens in new window)
• Accessibility
• Social Media Hub
• MIT on Facebook
• MIT on YouTube
• MIT on Instagram

Haifeng Xu Wins Best Paper Award at Leading AI Conference for Pioneering Research on Mechanism Design for LLMs

As this year’s Web Conference is under way, pioneering research work by Assistant Professor of Computer Science and Data Science Haifeng Xu and his collaborators has been announced as the winner for their prestigious Best Paper Award.

Xu’s paper, entitled “ Mechanism Design for Large Language Models ,” was selected from amongst 2008 submissions.

This paper lays out a newly developed method to aggregate language generations from multiple self-interested LLM agents into a single text generation. It does so by accounting for these LLM agents’ self-interests in an incentive-compatible way. As summarized in the meta review, “the review team unanimously finds the paper novel, well-executed, and … has potential to be a landmark paper sparking a new line of research linking LLMs and mechanism design.”

This paper is a joint work with Google Researchers. The technology Xu and his team developed has been tested on Google’s LLM model Bard and Xu reports that it performs very well. According to Xu, the nice (and often very rare) combination of both strong theoretical development and real-world implementation on Bard is probably a key reason for the paper to be named the Best Paper.

Congratulations, Haifeng!

This article was originally published by the Data Science Institute.

Related News

Fred Chong Receives Quantrell Award for Excellence in Teaching

Unveiling Attention Receipts: Tangible Reflections on Digital Consumption

NASA to Launch UChicago Undergraduates’ Satellite

University of Chicago Computer Science Researchers To Present Ten Papers at CHI 2024

Two UChicago MPCS Students Win the Apple Swift Student Challenge

How Artificial Intelligence Can Transform U.S. Energy Infrastructure

Community Data Fellow Stephania Tello Zamudio helps broaden internet access for Illinois residents

Two UChicago CS Students Awarded NSF Graduate Research Fellowship

Non-Unital Noise Adds a New Wrinkle to the Quantum Supremacy Debate

The Science of Computer Security: An Interview with Grant Ho, Assistant Professor in Computer Science

Four Students Receive Honorable Mention in CRA Undergraduate Research Awards

Navigating the Intersection of Technology and Public Policy: The Journey of Ranya Sharma at UChicago

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Sign up for alerts
• 13 May 2024

'Einstein’s death shattered Bose — he tore off an unpublished research paper in grief'

You can also search for this author in PubMed   Google Scholar

Partha Ghose. Credit: Partha Ghose

Nature India : What inspired you to embark on a career in particle physics?

Partha Ghose : In 1961, I went to Imperial College London wanting to decipher what lies at the deepest core of matter. This was when physicists had already begun smashing atoms in colliders, churning out new transient particles to unveil the secrets of matter.

I was lucky to learn advanced physics from stalwarts in particle physics like Abdus Salam, P. T. Matthews and T. W. B. Kibble. In 1963, an opportunity to work briefly at CERN further spurred my interest in particle physics.

NI : How did you meet S. N. Bose?

PG : It was in mid-1963 that I met Bose, quite by chance. The famous Bengali pote Bishnu Dey, a relative of ours and Bose’s close friend, took me to meet Bose. They were immersed in discussion, and Bose suddenly turned to me and asked, ''Would you like to work with me?''. It was an offer I couldn’t refuse. He arranged everything very quickly and within days I joined his research group at the Saha Institute of Nuclear Physics in Calcutta as a junior scientist. I worked on particle physics, mainly on broken SU(3) symmetry which was in vogue at the time.

NI : How did Bose start interacting with Einstein and how was their relationship?

PG : Occasionally, Bose would reminisce about his interactions with Einstein. One day when I went to meet him at his residence, he started talking about the historic paper he sent to Einstein on 4 June 1924 along with a letter .

Bose told me that his deduction of the phase space factor in Planck’s law resulted in a factor of 4π instead of 8π. He went on to propose that the missing factor of 2 was due to the photon spin which could take only two values. In his letter back to Bose, “the old man” [Einstein] had crossed this portion out and said it was not necessary to talk about spin since the factor of 2 comes from the two states of polarisation of light.”

Bose said to me, ”I can understand a spinning particle, but what is the meaning of the polarization of a particle?” I asked him, "Sir, when the photon spin was eventually discovered, why didn’t you tell Einstein that you had already worked it out in 1924?” “How does it matter who discovered it,” he quipped. “It was eventually discovered, wasn’t it?”

In a second paper, which also Einstein translated into German and got published in Zeitschrift fur Physik , Bose proposed a probability law for interactions between matter and radiation. According to Einstein, it was inappropriate. He added a comment to the paper giving some reasons for his disagreement with Bose.

The first paper with Einstein’s strong endorsement made Bose famous. He moved to Paris on a two-year sabbatical from Dhaka University, worked in Maurice de Broglie’s and Marie Curie’s labs and arrived in Berlin in 1925 to finally meet Einstein. They discussed several issues including Bose’s new hypothesis of probabilistic interactions, but Einstein stuck to his point.

Despite their differences, Bose regarded Einstein as his master in physics. On 18 April 1955, Einstein died. The news shocked him into silence. He was writing a paper and was looking forward to discussing it with Einstein at a forthcoming conference in Switzerland to celebrate fifty years of Special Relativity. Bose tore that paper into shreds.

NI : Many say that S. N. Bose missed out on a Nobel Prize for physics.

PG : He deserved the prize for his seminal contribution to quantum theory. It led to the classification of particles into bosons and fermions and the prediction and discovery of Bose-Einstein condensates. Besides, his theories helped us understand superconductivity and superfluidity. Bose’s theories and insights shaped the works of many physicists. Some went on to win Nobel Prizes. But Bose, despite being recommended several times, was never considered for the prize.

NI : Apart from physics, you learned music, and played first-class cricket. Do you think science helps enrich music?

PG : Science can help explain music. The best example of this in India is Sir C. V. Raman who had a keen ear for Indian classical music. He could detect five harmonics in the 'mridangam' and the 'tabla' sounds. He did some experiments with Indian drums and circular membranes with central loads. He sprinkled white powder on them to see the patterns of vibrations formed as he kept changing the loads and the manner of striking. This led to a new understanding of the generation of harmonics in stretched membranes. His research in musical instruments earned him the Fellowship of the Royal Society of London even before he got the Nobel Prize for his work on light scattering.

NI : S. N. Bose advocated popularizing science in Bengali. Is it easier to communicate science through one’s mother tongue?

PG : Science is based on logic and requires precise language for its expression and understanding. Non-native speakers find it difficult to grasp the nuances of scientific terms in English. They often acquire wrong notions when they read science in English.

The language in which one dreams is one's mother tongue. Science can therefore take root and flourish in a country only when its citizens start dreaming about science in their mother tongue.

NI : What is your advice for young Indian students who want to pursue a career in physics?

PG : I will quote Bose’s last advice to me. ''Don't jump onto foreign bandwagons. Try to understand things in your way and say something new.''

Bose read the works of all leading quantum theorists of his time, including Einstein, with a critical mind, identified their shortcomings and went on to propose revolutionary new statistics. These days I see an undue rush to publish papers in reputed foreign journals, increase citations, and get quick recognition and promotion. This leads to derivative science.

doi: https://doi.org/10.1038/d44151-024-00054-2

Postdoc in CRISPR Meta-Analytics and AI for Therapeutic Target Discovery and Priotisation (OT Grant)

APPLICATION CLOSING DATE: 14/06/2024 Human Technopole (HT) is a new interdisciplinary life science research institute created and supported by the...

Human Technopole

Research Associate - Metabolism

Houston, Texas (US)

Baylor College of Medicine (BCM)

Postdoc Fellowships

Train with world-renowned cancer researchers at NIH? Consider joining the Center for Cancer Research (CCR) at the National Cancer Institute

Bethesda, Maryland

NIH National Cancer Institute (NCI)

Faculty Recruitment, Westlake University School of Medicine

Faculty positions are open at four distinct ranks: Assistant Professor, Associate Professor, Full Professor, and Chair Professor.

Hangzhou, Zhejiang, China

Westlake University

PhD/master's Candidate

PhD/master's Candidate    Graduate School of Frontier Science Initiative, Kanazawa University is seeking candidates for PhD and master's students i...

Kanazawa University

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies