Introduction

One technique identified by researchers to improve engagement, influence learners’ attitudes and behaviors, and improve learning outcomes, is gamification (Landers, 2014). Gamification has been defined as “the use of game design elements in non-game contexts" (Deterding et al, 2011). In 2013, gamification was described as the “hottest business buzz-word” (McCormick, 2013).   But a year later, Clancy (2014) suggested that it “looks like that whole 'gamification' thing is over”. Burke (2014) explained in his study that the reason that organizations were turning away from the concept of gamification was that not many were getting it right.

Landers (2019) proposed that the root of the problem was not that the technique was ineffective but that the understanding of the process was not correct. Given the similarity between the construct of gamification, serious games, and games, a large number of researchers and practitioners were using the terms interchangeably. This has also led to a lack of consensus on the effectiveness of the process of gamification. The current study attempts to provide some levels of clarity by systematically reviewing literature in the area of gamified learning. The review also takes into account the context of the study, the methodology used, and the variables of the study. The purpose of the study is to provide a coherent and concise view of studies carried out in the area of gamified learning, over the last five years.

Background

As mentioned earlier, the literature and practice of gamification followed a faddish trend. The number of studies and practitioners talking about the concept peaked in 2012, post which there was a decline in interest. This time around, the concept is again garnering interest but at a slower pace with much more stress being put on the understanding of the term and exploring the impact of gamification. Landers (2019) explains how gamification has been misunderstood over the years. While he encourages the borrowing of taxonomy from the construct of games, he also stresses the need to differentiate between the related concepts. 

The most popular definition of gamification was provided by Deterding et. al. (2011). They define it as “the use of game design elements in non-game contexts" (Deterding et al, 2011). The definition is key to understanding the concept and to being able to differentiate it from the related concepts like serious games or games in general. The first part of the definition discusses “game elements”. This implies that there are certain building blocks to games that can be arrived at by deconstructing any game. These blocks are generally found in all games and can be viewed separately from the game. These elements include the story, rules, chance, competition, collaboration, etc. (Kapp, 2014).

The second part of the definition talks about “Using” these game elements. Thus, there is an implied process of application. This also refers to the idea that gamification is a process and not an outcome. According to Kapp (2014), gamification may be considered as a continuum and the extreme right end of the continuum is a game.

The third part of the definition refers to non-game contexts. Gamification involves taking routine daily activities and transforming them by applying the different design techniques so that the final result is more game-like but the objective or learning from the activity remains the same. Werbach (2013) explains that gamification is “more than just rewards and points” but involves engaging and motivating people in a “gameful way”.

The similarity between the constructs of “gamification”, “serious games” and “games” have led to the terms being used interchangeably, although the constructs are considerably different. Games are defined as “voluntary activities, obviously separate from real life, creating an imaginary world that may or may not have any relation to real life and that absorbs the player’s full attention” (Michael & Chen, 2005, p.19). Serious games refer to all games that are created for any purpose, other than entertainment, while gamification is the process of making non-game activities more game-like. However serious games and gamification are very closely related, as the objective in both cases is to improve learning outcomes, gamification implies the application of one or more gaming elements to an existing instructional design and refers more to the process and not the actual instruction. Gamification aims at improving learning outcomes by altering learning behavior and attitude. In other words, the learner may learn from a serious game but not directly from gamification (Landers, 2015).

Gamification and Learning Outcomes

Learning outcomes are typically measured by the change in cognitive, affective, or skill capacities (Kraiger, Ford & Salas, 1993; Burke & Hutchins, 2007). Training reaction is one popular outcome that is usually the target of gamification interventions. Reactions are post-training opinions regarding the training program, including affective reactions, perceptions of the training’s utility, and difficulty in justifying the effort required to perform the training well (Kirkpatrick, 1959; Warr & Bunce, 1995). Armstrong (2015) used a static group comparison to evaluate the reaction to gamified training. The test group showed significantly, more positive reactions, in comparison to the control group.

Gamification has also been found to improve learning. MacKinnon et. al. (2015) used a pre-test, post-test control group design to explore how gamification improved learning and learner motivation. Learner motivation refers to the participants’ desire to learn the contents of the program (Noe, 1986). Their findings suggest that gamification led to a significant increase in learning outcomes. The use of game elements was also found to be helpful in bringing about the desired changes in employee behavior (Suh & Wagner, 2017; O’Neill et. al., 2018). Suh and Wagner (2017) used surveys to evaluate the impact of gamification on the perceived hedonic value of an Enterprise Collaboration System or ECS. Their study suggested that the application of game elements to the ECS would increase the perceived value of the system and encourage employees to participate in knowledge sharing.

Sargent (2017) explored how gamification improved engagement in online learning programs. Learner engagement refers to the level of investment in the learning process (Newman, 1992). The findings suggested increased engagement in the group that participated in the gamified module. Sargent (2017) also found that the test group had higher levels of motivation to participate in the program, implying that the use of game elements increased participant motivation.

Theoretical background

Lander’s (2014) Theory of Gamified Learning, is one of the few theoretical frameworks that are specific to gamification. Landers (2014), built on the Input-Process-Output model (Garris et. al., 2002). The IPO models suggest that the process of training is driven by the instructional material and that the material is expected to trigger a learning cycle. This process of visiting and revisiting the material is expected to enhance the learning process and improve learner outcomes. This is where gamification is expected to contribute by encouraging the desired behavior in the learner. As suggested by Tay (2010), while using the game elements, one must remember that the use of the elements is to trigger changes in the learner behaviors and attitudes that would result in improved learning outcomes.  Whitton Moseley (2014) proposed that the use of game elements is likely to increase learner motivation, which in turn is likely to improve the outcomes of the learning process. The Theory of Gamified Learning draws from the above studies.

Fig 1. Theory of Gamified Learning

There are five propositions of the theory. The first two propositions talk about the role of the learner and the instructional content, and the last three are related to the role of gamification. The first proposition provides a direct relationship between the instructional material and the learning outcomes. Empirical evidence for the same can be found in the work of Arthur et. al. (2003) and Seidel and Shavelson (2007).  The second explains the relationship between learner attitudes and behaviors and the learning outcomes. The learner’s behaviors and attitudes are expected to drive the learning process and thereby impact the learning outcomes. Paas et. al. (2005) found that the change in knowledge levels was directly proportionate to the amount of effort that the leaner put into the activity. Zhao & Kuh, (2004) also found that learner engagement was key to the learning process and would directly impact the learning outcomes.

The next three propositions are directly related to the application of game elements. The third proposition of the theory suggests that the application of the game elements would impact the attitudes and behaviors of the learners. This proposition finds support in the context of serious games (Wilson et. al., 2009). The fourth proposition of the theory is that the use of game elements would moderate the direct relationship between the instructional content and learning. This proposition further highlights the fact that the game elements can only modify the relationship and the original material is still paramount in deciding the effectiveness of the learning process.

The last proposition refers to the direct role of gamification in influencing learning outcomes. Evidence for the same was found in the study by Landers and Callan (2011). The researchers used gamification to encourage certain behaviors and found that the change in behavior improved the academic performance of the students. This mediating role is often considered the primary role of gamification in the learning process (Hamari, Koivisto, & Sarsa, 2014).

Thus, studies in gamification have primarily attempted to identify the different variables that can be moderated or mediated by the use of gamification. While most studies have used the Training Evaluation model suggested by Kirkpatrick (1959), others have attempted to evaluate the impact of gamification on motivation, engagement, and other attitudes and behaviors.

Methodology

The current study is a systematic review of literature in the area of gamified learning. The objective of the current study is to provide a snapshot of the research done in the area of gamified learning over the last five years. This includes identification of the contexts of the studies, understanding the methodology and variables used for the study, and a discussion of the outcomes of the reviewed studies. Through this review of literature, the researchers aim at providing a snapshot of research in the area of gamification and identifying avenues for future research.

The ProQuest databases (ProQuest ABI, ProQuest Theses and ProQuest Research Library) were chosen for the study. The search string used was ‘Gamification and Learning’ and an additional filter of the time period was used.  Only the studies published in the last five years were included in the current study. The search provided 3,107 results, which included research articles, theses, dissertations, and articles in trade journals. An additional filter of language, to filter out regional papers, returned 2,942 valid results. The sourced articles were reviewed and only the papers related to ‘Gamification’ were retained. Papers on related topics like ‘Serious Games’, ‘Games’ and ‘Simulations’ were excluded.

The remaining studies were reviewed and further categorized as Theoretical and Empirical Studies. Further classification of the Empirical Studies was carried out, to identify Quantitative and Qualitative Studies. This was done to facilitate the analysis. The review process has been represented diagrammatically in Figure 2.

Fig. 2. Review Process

Findings

The current section presents the results of each step of the review process. The categorization of the studies, at each step of the review, has also been presented in the tables under each section.

Studies identified

The number of papers that were shortlisted at the different steps has been provided in Table 1.

Table 1. No. of studies shortlisted at each step

While the initial search revealed around 3000 studies and 2942 in English, a review of the manuscripts suggested that only 184 of the studies were actually related to the construct of gamification. Of these, 64 were quantitative in nature and were carried forward to the next step. The quantitative studies were retained since the results were unambiguous and easy to interpret. Qualitative studies would be more useful in gaining a deeper understanding of different aspects of the learning process.

Context of the study

The studies were conducted in a large number of contexts. For the purpose of the current study, the contexts were categorized as School, Higher Education, Organizational and Social. The categorization was done based on the level of learning that the researcher was exploring. The category of Social included those studies that were looking at learning of socially important messages or generic courses related to personal growth. Those that targeted learning in the academic or organizational context were placed in the respective categories. Table 2 presents the categorization of studies.

Table 2. Categorisation based on Context

Among the 64 studies shortlisted in the previous step, 39 were carried out in the academic context (School and Higher Education), 10 in the organizational training context, and 15 in a social learning context.

Research Design

Table 3 provides the categorization of the studies based on the methodology that was adopted. The analysis was restricted to the quantitative studies as the results of these studies could be compared. The pre-test post-test control group design, Solomon four-group design, and Post-test control group design are the designs that are considered truly experimental. Among the reviewed studies, only seven studies used a truly experimental design, while the majority used a one-shot design or a static group comparison. Thus, these studies did not evaluate the knowledge levels of the participants before the intervention. Ten studies used the traditional survey method. These studies mainly evaluated the perception towards Gamification.

Table 3. Categorisation based on Methodology

Dependent or outcome variables

The next step in the analysis was conducted to identify the dependent variables in the studies. Table 4 provides the dependent variables studied and the number of papers that had chosen each variable. Since the focus of these studies was to evaluate the impact of gamification in the learning process, the most commonly chosen dependent variable was learning. Learning implies a change in knowledge levels. The other explored dependent variables include learner motivation and learner engagement.

Table 4. Categorisation based on Dependent Variables

Study Outcomes

Table 5 presents the studies, categorized on the basis of their outcomes. Those studies that found a statistically significant relationship between gamification and the dependent variables formed the first category and those that failed to find a significant relationship formed the second one. A majority of the studies found that Gamification has a significantly positive impact on the learning process.

Table 5. Categorisation based on Study Outcome

Statistically Significant Relations

For each dependent variable that was studied in the context of gamified learning, the number of studies that found a statistically significant relationship are highlighted in Table 6. Gamification was found to have a significant impact on learning in 23 of the 36 studies that evaluated this relation. Similarly, 11 of the 15 studies that evaluated the impact of gamification on learner engagement, found that gamification resulted in an increase in learner engagement. All the studies that evaluated the relationship between gamification and motivation found that learner motivation was higher in the case of a gamified module. Reaction to a gamified module was also found to be significantly higher in 5 of the 6 studies that evaluated this relationship.

Table 6. Categorisation based on Statistically Significant Relationships

Proposed Model of Consequences

Based on the above tables, the researchers proposed a model of consequences of gamified learning. Figure 3 presents a model of proposed outcomes or consequences of gamification. Gamification was found to positively impact all seven variables.

Fig. 3. Consequences of Gamification

Discussion

Research in the area of gamified learning has been on a rise. While the number of studies into the concept has been increasing, there are still a large number of concerns amongst researchers. The main concern is that since the concept of gamification is similar to that of serious games and games, researchers and practitioners have borrowed concepts and terms and adapted them to suit the construct of gamification. This borrowing has led to a large amount of similarity between the concepts and the boundaries appear to be blurring, especially in practice (Landers, 2015).

Before setting out on exploring the impact of gamification, the researchers used the current study to understand the existing literature in the area of gamification. The objective was to understand the contexts, methodology, and outcomes of the existing studies. The initial search revealed over 3000 articles in the area of gamification, with over 2900 in the English language. An analysis of the articles revealed that only 184 of these were actually about the concept of gamification, while the remaining explored related constructs. Finally, the researchers shortlisted only the empirical studies that were quantitative in nature for the final analysis. This was done because in the case of quantitative studies the results can be easily identified as significant or not, while qualitative studies allow room for interpretation.

The number of quantitative studies found was 64. Of these, 47 found statistically significant results between the use of Gamification and changes in the dependent variable. The studies by MacKinnon et. al. (2015) and Nand et. al. (2019) were unique in that they used true experimental methodology to conduct the study. Both studies used a pre-test, post-test control group design for their experiment and evaluated how the use of gamification would improve learning and learner engagement. While the study by Nand (2019) was carried out in an academic context, MacKinnon (2015) studied the impact of gamification in an organizational context.

Most of the researchers opted for a One-Shot or One-Group, pre-test, post-test design. While attempting to evaluate perception, researchers opted for the use of surveys in their study. In the organizational context, Suh and Wagner (2017) used surveys to understand if gamification would increase knowledge sharing in the organization. They evaluated the impact of gamification on the perceived hedonic value of an Enterprise Collaboration System or ECS. The findings of their study suggested that the logical application of game elements to the ECS would increase the perceived value of the system and encourage employees to participate in knowledge sharing.

In the organizational context, seven studies found statistically significant outcomes of gamification. Gamification was found to influence learning in the organization (Trimblett, 2016; ONeill, 2018; MacKinnon et. al., 2015; Andriamiarisoa, 2018). Secondly, Gamification was found to facilitate behavioral training. It was found that the use of game elements helped in bringing about the desired changes in employee behavior (Suh & Wagner, 2017; O’Neill et. al., 2018). Gamification was also found to increase learner engagement (Sargent, 2017; Trimblett, 2016). While Trimblett (2016) explored the use of Gamification in a mandatory course on Jurisprudence for Nursing Staff in Canada, Sargent (2017) explored how Gamification improved Engagement in online corporate training programs. The evaluation was done by comparing the results of a control group and a test group. The findings suggested increased engagement in the group that participated in the Gamified module. Sargent (2017) also found that the test group had higher levels of motivation to participate in the program, implying that the use of game elements increased participant motivation

Lastly, Gamification was also found to improve learner reaction to the training (Armstrong, 2015). In their study, they used a static group comparison to evaluate the reaction to training. The test group showed more positive reactions, in comparison to the control group. However, the post-test results of both groups did not differ. This suggested that while the learning in both groups  was probably not significantly different, the level of satisfaction of the participants of the gamified module, was significantly higher. Thus, most studies suggested that gamification would improve reaction to a learning experience, result in positive behavioral changes and increase learner engagement and motivation. However, the impact of gamification on learning or knowledge levels is still not clear.

The current study summarises the results of studies in the area of gamification; carried out in the last 5 years. The study also highlights areas that need to be explored. Firstly, the lack of studies using a true experimental design suggests that in order to establish the causal relationship, there is a need for more studies that are experimental in nature. The number of studies in the organizational context is also not sufficient. Given the growing presence of technology in the organization, there is a need to explore the role of Gamification in the context of Organizational Training. Additionally, studies in the organizational context must also take into consideration the TETEM. TETEM or Technology-Enhanced Training Effectiveness Model suggests that the effectiveness of the Gamified Modules would be dependent on two sets of characteristics; personal and organizational. Personal characteristics include the trainees’ attitude towards game-based learning and experience with video games. The organizational characteristics refer to the organizational climate and perceived supervisory support (Landers and Armstrong, 2015). There is also a need to identify other environmental factors that could moderate the effectiveness of a gamified module. The identification of the environmental factors would help practitioners design their interventions better.

Limitations

The current study has some limitations. The first limitation, as in all review papers, is with regards to the search terms used and the journals accessed. The existence of a possible publication bias, with journals preferring to publish articles with positive results over others, would impact the results. Secondly, this article only draws from the reviewed literature and the validity of the study relies the quality of the papers. To overcome the same, the researchers have only included studies that were published in reputed journals. Lastly, the focus has been on the quantitative studies, as the probability of misinterpretation of the results is low. However, the researchers acknowledge that qualitative studies may provide more insights.

Conclusion

Prior to attempting to gather empirical evidence for the impact of gamification, the researchers thought it appropriate to conduct a systematic review of the literature. The systematic review of the literature provided some interesting insights into the studies that have been conducted in the last five years. The number of studies that have looked at gamification in organizational learning is relatively low in number. The majority of the studies have evaluated the effectiveness of gamification in the classroom. The rigor in the studies also appears to be low. While the quasi-experimental, static group testing, and one-shot experiments are acceptable methodologies, the validity of true experimental methods would be higher. Thus, while theoretically gamification is expected to enhance a learning experience and improve learning outcomes, empirical evidence supporting the same is still sparse.

Declarations

• Availability of data and material: Not applicable
• Funding: The study has received no external funding
• The authors have no competing interests to declare

Conflict of interest

The authors declare no potential conflict of interest regarding the publication of this work. In addition, the ethical issues including plagiarism, informed consent, misconduct, data fabrication and, or falsification, double publication and, or submission, and redundancy have been completely witnessed by the authors.

Funding

 The author(s) received no financial support for the research, authorship, and/or publication of this article.