back

Open Badges: A systematic Review of Peer-Reviewed Published Literature (2011-2015)

Tharindu R. Liyanagunawardena [tharindu@ucem.ac.uk], Sandra Scalzavara [s.scalzavara@ucem.ac.uk], University College of Estate Management, Horizons, 60 Queens Road, Reading RG1 4BS [https://www.ucem.ac.uk], Shirley A. Williams [shirley.williams@reading.ac.uk], University of Reading, Whiteknights, Reading, [www.reading.ac.uk], United Kingdom

Abstract

Open badges are a digital representation of skills or accomplishments recorded in a visual symbol that is embedded with verifiable data and evidence. They are created following a defined open standard, so that they can be shared online. Open badges have gained popularity around the world in recent years and have become a standard feature of many learning management systems. This paper presents a systematic literature review of the published open badges literature from 2011 to 2015. Through database searches, searching the internet and chaining from known sources, 135 relevant peer-reviewed papers were identified from a corpus of 247 publications for this review. The authors believe this to be the first effort to systematically review literature relating to open badges. The review categorised publications while also providing quantitative analysis of publications according to publication type, year of publication and contributors. After assessing the literature suggestions for future research directions are presented, based on under-represented areas.

Abstract in French

Les badges ouverts, sont une représentation numérique de compétences ou de realisations, enregistrés dans un symbole visuel intégré à des données vérifiables et à des preuves. Ils sont créés en suivant une norme ouverte définie, afin qu'ils puissent être partagés en ligne. Les badges ouverts ont gagné en popularité a travers le monde ces dernières années et sont devenus une caractéristique standard de nombreux systèmes de gestion de l'apprentissage. Cet article présente une revue systématique de la littérature des badges ouverts publiés de 2011 à 2015. Grâce à des recherches de bases de données, en recherchant l’Internet et à la mise en chaîne de sources connues, 135 documents pertinents évalués par des pairs ont été identifiés à partir d'un corpus de 247 publications pour cette revue. Les auteurs pensent que cette revue systématiquement de la la littérature relative aux badges ouverts est une première. L'évaluation a catégorisé les publications tout en fournissant une analyse quantitative des publications selon le type de publication, l'année de publication et les contributeurs. Après avoir évalué la literature, des suggestions pour orienter la recherche dans le future sont présentés en fonction des zones sous-représentées.

Abstract in Portuguese

Badges abertas são uma representação digital de habilidades ou conquistas registradas através de um símbolo visual embutido com dados e provas verificáveis. Elas são criadas seguindo um padrão aberto definido, para que elas possam ser compartilhadas virtualmente. As badges abertas ganharam popularidade em todo o mundo nos últimos anos e se tornaram uma característica padrão de muitos sistemas de gerenciamento de aprendizagem. Este artigo apresenta uma revisão sistemática da literatura pubicada entre 2011 a 2015 referentes as badges abertas. Através de buscas de banco de dados, pesquisa na internet e encadeamento de fontes conhecidas, 135 documentos relevantes que foram revisados por pares foram identificados a partir de um banco de 247 publicações sobre este assunto. Os autores acreditam que este seja o primeiro esforço para revisar sistematicamente a literatura relacionada as badges abertas. A revisão categorizou as publicações, além de fornecer análises quantitativas de publicações de acordo com o tipo de publicação, ano de publicação e contribuintes. Depois de avaliar as sugestões da literatura, futuras orientações de pesquisa são apresentadas com uma base em áreas sub-representadas.

Keywords: open badges; online learning; digital badges; badges

Introduction

Open badges are a fairly recent addition to the online learning landscape although badges (in general) have been in existence for many years. For example, merit badges have been used in the Scouts movement internationally since 1910 (International Scouting Collectors Association, n.d.). Digital badges are used similar to physical merit badges, to visually represent a skill or achievement. Badges have long been used successfully to set goals and motivate, and represent and communicate achievements and success (Knight & Casilli, 2012).

Open badges take the concept of digital badges further by following an open standard to store meta data about the badge as well as the skills and evidence that were demonstrated to earn the badge in the digital badge image file (details in the next section). Earners of open badges can collect/combine various badges they have earned from different issuers to present a holistic view of their learning journey. These badges can then be shared in online profiles. The open badge standard (Mozilla Foundation, 2016) provides a widely-used framework for demonstrating achievements and skills.

Despite a worldwide interest in open badging (www.badgetheworld.org), there are limited reviews of the published literature and Gibson et al (2015) suggested it may be premature for a comprehensive literature review on digital badges. The aim of this paper is to systematically review the literature to identify relevant work that can inform decisions to be made when adopting open badges. Search techniques for papers related to badges are considered and a corpus of papers identified; a grounded research approach is then presented from which a classification of the works emerges.

Background

Learners no longer learn in one learning environment be it physical or digital. Contemporary learners engage in various networked, distributed learning spaces that span across time and space to provide multiple pathways for lifelong learning (Knight & Casilli, 2012). Many learners like to share their success and accomplishments on social media. In this “connected learning ecology” badges can bridge contexts (or different learning spaces) by capturing and making the learning more portable and impactful (The Mozilla Foundation et al, 2012; p.3).

Badges are used for various reasons, most commonly as rewards or motivators. Antin and Churchill (2011) show that in the social media context, badges have five social psychological functions, namely: goal setting, instruction, reputation, status/affirmation, and group identification. Badges are increasingly used in websites where the community content creation and moderation is of importance.

Digital Badges

Digital badges are used to reward users for accomplishing tasks – for example, achieving a certain level in a game or writing a set number of reviews. These badges are normally used internally and cannot be exported elsewhere. For example, the popular travel-related content review site TripAdvisor.com issues various categories of badges to its users (Tripadvisor, United Kingdom, n.d). Different levels of Reviewer badges are issued to users who contribute a certain number of reviews (New Reviewer, Reviewer, Senior Reviewer) and so on. Additionally, the TripAdvisor user community can rate reviewers by awarding them with helpful votes denoting how helpful a review was. Receiving a set number of helpful votes qualifies a reviewer to achieve different levels of Helpful votes badges.

These digital badges enhance the profile of the user within the environment where the badges are being awarded. However, as a concept it does not go far enough to bring together the journey and/or experience of the user across a variety of online settings.

Open Badges

Open badges were created by Mozilla in 2011 with funding support from the MacArthur Foundation, one of the largest independent foundations in the United States. The open badges technical specification that was an outcome of this effort defines the requirements for what an open badge should represent for both issuers and earners (Mozilla Foundation, 2016). These open standards enable interoperability between badging systems, facilitating badge use by both issuers and learners. Open badges can be stored and managed in a personal badge repository – for example, Mozilla Backpack (Mozilla Foundation, 2017). This allows the recipient to bring together the badges they have earned in various environments, blurring the boundaries and overcoming the limitations of digital badges. The ability to manage and tailor badges for display in a badge repository gives the recipient more control over the ways open badges could be used. Open badges enable learners to evidence their accomplishments.

Method

Data Collection

Different methods are used by researchers to identify appropriate papers for literature reviews (Ellis, 1989; Ellis & Haugan, 1997): Methods include searching (databases and/or search engines) and chaining from known research papers. In a systematic review, it is important that the approach to identifying papers are described and justified, and that the method presented can be re-applied by others (Fink, 2010; Liyanagunawardena, Adams, & Williams, 2013). The use of search engines such as Google in academic research is controversial, however it has gained popularity with academics (Jamali & Asadi, 2010; Nicholas et al, 2014), researchers using Google as a discovery service usually limit themselves to studying only the first few items, for example Namei and Young (2015) only consider the first 10 items as does Brophy and Bawden (2005). For a systematic review, it is important to know what cut-off point is used in a search.

The aim of this study was to find and then analyse academic literature on open badges, to facilitate an understanding of developing research areas, methods applied in research, and any topics with limited published research.

Papers deemed relevant were identified through a series of search efforts, using a method based on the approaches used in other systematic reviews (Gao, Luo, & Zhang, 2012; Williams, Terras, & Warwick, 2013; Liyanagunawardena, Adams & Williams, 2013; Liyanagunawardena & Williams, 2016). Given the aim of this work is to identify relevant papers that can inform decisions to be made when adopting open badges, papers were pooled if their primary focus was to: explore badging, implications for education, report on empirical evidence.

Searches

First, the search terms and boundaries to be used in the literature search were established. The terms selected were:

  • Open badges;
  • Open badge.

However, it was identified that some authors used badging and so a third search term was added:

  • Open badging.

Where possible the search was limited to titles and abstracts to reduce the workload when manually filtering irrelevant papers.

The search term open badg* was used to search databases for relevant articles (open badg* search term uses the wildcard * so that terms open badge, open badges or open badging will be captured). The Scopus database search returned 27 entries while the Web of Science database returned 12 entries (December 17, 2015). Together the two databases returned 30 distinct entries.

The same search term on Google Scholar resulted in 985 results (December 18, 2015). Google Scholar advanced search returned 2,250 results for the search term open badge OR open badges OR open badging. To reduce the number of papers to examine at this first level collection, searches were done for each of the terms separately in title section of the papers. These searches were carried out on December 23, 2015 (with patents and citations excluded). The search term open badges returned 38 results; open badge eight results; and open badging five results. Out of these 47 distinct entries were identified. After eliminating the duplicates, the searches identified 64 distinct entries.

Content pages of conference proceedings (returned in the searches) were scanned to identify any other relevant publications, resulting in the addition of eight papers and one proposal for a research competition, resulting in a total of 73 entries. It should also be noted that the presentations returned by the searches which did not have an accompanying peer-reviewed papers were excluded from the corpus.

Next, the authors used the Google (Web) search engine to identify relevant work (January 14, 2016) and in these searches, only the first fifty items of the results were considered. The results were examined for relevant publications and this added three items to the list of publications.

Figure 1

Figure 1. First phase of searching

Chaining

Next, the method of chaining from known sources was applied:

  • Author search;
  • Reference search.

The author search was conducted for all authors in the corpus to find their other relevant papers. Google Scholar searches using the author name followed by “open badge” was used for this. In this search, only the first page of results for each author was scanned for any other relevant publication titles.

The reference section of each corpus paper was scanned to find any other relevant publication titles. If an entry seemed relevant at a first glance, it was allocated a number and included in the list.

The process was repeated for each new item identified. The Open Badges in Higher Education website referred to in a paper was a further useful resource for finding more publications that had not been discovered by the searches. The full list of publications, totalling 247, collected following the method described above is available as Appendix 1.

Elimination

The authors then had to discount publications from this list for various reasons to identify relevant and accessible papers.

Through their academic links, authors had access to four different university libraries providing access to most major publications. However, there were still publications that the authors were not able to access. Where papers were not accessible, authors of those publications were contacted directly via email and social media (mainly Twitter) to request an author final version of the publication. In most instances, this was successful. However, some of the publications had to be discarded from the list because of access limitations (see Appendix 1). Where only abstracts/extended abstracts were accessible and where these did not carry sufficient information, they were discounted. Workshop presentations, working papers and editorials as well as monologs and blogs were rejected as these are not peer reviewed.

On examination, some papers were found not be about open badges, although they used the words “open” and “badge” and these were eliminated. Papers published in languages other than English were discounted. Names of conference/workshop proceedings that were returned as items in the database search were also eliminated; however, as described above, relevant papers from those proceedings were included.

Where possible, conference websites were consulted to ascertain that the papers had been peer reviewed. Extended abstracts were discarded where only full conference papers were said to have been peer reviewed.

Articles were discarded if badging was a minor strand of the discussion. Where two papers by the same authors shared a large amount of common material, only one was carried forward.

Analysis

Elimination

On detailed review, it was identified that there were still some papers that were not relevant to this study and should be eliminated.

It was evident that some publications were not on open badges rather they referred to digital badging experiences where the badges were specific to the environment they were offered in and did not adhere to the open standards. Nine papers were rejected (see Appendix 1). In three instances, it was not possible to clarify whether the badges awarded were digital badges or open badges. In these instances, it was decided that on balance the publications should be retained.

A summary of publication numbers considered in this analysis is provided in Table 1.

Table 1:  Publication numbers in the study

Table 1

Quantitative data

The total number of publications per year on open badges has increased steadily from 2011 to 2015 (note that the literature search was conducted from December 2015 to January 2016). Altogether this analysis drew on 53 journal papers, 76 conference papers, five chapters in edited books and a research report. The distribution of publication types by year of publication is depicted in Figure 2.

Figure 2

Figure 2. Open badge publications by year

The papers considered in this review were authored/co-authored by 178 authors. With six publications, Professor Daniel Hickey from Indiana University was the top contributor to open badges literature in this review, closely followed by Samuel Abramovich with five. Adele Botha and Ari Korhonen both had four publications each. 18 other authors had authored/co-authored three articles on open badges.

Emerging categories and themes

  1. The first two authors reviewed some 10 papers independently and used free format text to describe the paper types. Then the team met in a brainstorming session and identified that papers appeared to belong to one of two categories:
    • theoretical – in general these papers address the issue of badges but did not report implementation;
    • practical – most of these papers contained some information about badging projects.
  2. Then the authors conducted further review of the papers. During this review, it emerged that amongst the practical category there were some papers which described the design of badging projects but had not reported the implementation. Therefore, the practical category identified above was subdivided:
    • design projects;
    • evidence.
  3. Following the final review of all papers it emerged that the evidence category could be further divided into:
    • evidence of badge issuing;
    • other evidence – such as reports of interviews and meta-analysis of projects.
  4. Finally, all papers were revisited and assigned to a category. Additionally, at this point, the authors made notes about the themes within papers. In a final brainstorming session, these were refined to:
    • badge variety (levels, types, user generated);
    • perception (student, employer);
    • supplementing badges and badging in schools.

A summary of the number of publications classified in each category is given in Table 2. Figure 3 depicts the publication categories by year.

Table 2:  Number of publications in each category

Table 2

Figure 3

Figure 3. Paper numbers in each category by year of publication

Publications categorised under design projects, evidence of badge issuing and other evidence were further analysed and summarised (Appendix 2). With (at least) 31 publications describing implementations in the US and one in Canada, North America with (at least) 32 implementations topped the analysis as being the main contributor to the open badge literature. The UK had nine publications with empirical evidence relating to open badges. Finland (six), Australia (five) and South Africa and Singapore had three each. There were some 28 publications relating to open badging projects in Europe. No projects were reported from the South American continent; however, the authors acknowledge that there may be such papers in non-English formats. The spread of projects/publications seemed to correlate with the open badging initiatives and grants (such as Badges for Lifelong Learning by the MacArthur Foundation, US and Open Badge Factory Project Consortium funded by Finnish Funding Agency for Innovation, Finland) that had been a catalyst in the adaptation and spread of open badging. Note that when considering a project location, authors only considered the location of the institute where badges were implemented. When the location of the project implementation institution could not be inferred, it was left blank in the dataset (Appendix 2).

Discussion

Badge variety

Badge levels

In the literature, badges are classified in to various categories. For example, activity-based badges, grade-based badges, hierarchical badges, composite badges and outcome based badges (Põldoja & Laanpere, 2014). In this analysis, the authors identified that most badge implementations had several levels: lower level, project level and course level (Randall, Harrison, & West, 2013); bronze, silver and gold (Cucchiara, Giglio, Persico, & Raffaghelli, 2014); beginner, intermediate and advanced (Kappes & Betro, 2015). Most commonly badges were designed to allow students to achieve a lower level badge and to then work towards higher level badges.

Positive, Neutral and Negative Badges

Most articles in this review presented badge systems that rewarded positive behaviour such as participation or an achievement. However, two articles (Charleer et al., 2013; Santos et al., 2013b) presented a case study of awarding negative badges (along with positive and neutral badges) to discourage unintended behaviour. For example, Charleer et al (2013) describe three types of badges: positive, negative and neutral and three levels of achievements: bronze, silver and gold. Student activity is tracked through digital traces, and inactivity is associated with a negative badge. Other negative badges mentioned included the “troll” badge and “I don’t want to talk to you no more” badge.

Neither paper presents an analysis of the impact of negative badges. At face value, it looks illogical to offer negative badges as the students are unlikely to accept or display a negative badge. However, Santos et al. (2013b) present the results of perceived usefulness of badges as categorised by students. Surprisingly, the badges with negative connotations were rated the most important. The reasoning was not clarified but it could be that the existence of negative badges motivated students to avoid behaviour that could lead to a negative badge.

In the rating of badges, students also considered group badges to be more important than badges that could be earned individually. Santos et al. (2013b) further reported that the students did not add these badges to their Backpacks, and they propose further work on identifying badges students would like to share through such systems. It is unlikely that a student would want to add a badge to their backpack unless it adds value to their CV beyond the college. Thus, raising an important point to be considered in badge design “does the badge add value beyond the point of earning?”.

User Generated Badges

Badges are normally created by the site administrators and/or course creators and offered to participants. However, in SAPO Campus (Portugal) platform’s open badge implementation supported user-generated badges to “break up with the traditional top–bottom perspective for the creation and attribution of badges” (Santos, Pedro, Almeida, & Aresta, 2013a). The first level of users in the SAPO Campus platform are institutions while the next level of users are teachers and learners of those institutions. The SAPO Campus platform provides a Badge Constructor Tool that can be used by both first and second level of users: that is, institutions can create and offer badges as well as teachers or a group of students. It also allows badge recipients to accept, reject or make the badge visible only inside the group. This was the only study that presented a case where students on a course could create their own badges. As badges can be created by any one and awarded to anyone, it was important that the platform had the functionality to reject badges. For example, as seen previously, had someone created a negative badge and offered in The SAPO Campus platform, the recipient can reject the badge. On the other hand, it does empower the grassroots to acknowledge good work via awarding badges and could be valued by educators if the badge was offered by participants. Only preliminary data from the study was presented and the user generated badges is an area that needs more research about benefits and weaknesses of such badges.

Badge perceptions

Student perception

In one study, students have revealed (Mewburn, Freund, & Rutherford, 2014) that they wanted the badges to look “official” and not “like something my kid would bring home from the daycare centre”; another comment was: “We don’t want it to look like it was fun being at ANU [Australian National University]! We want it to look like it was hard work”. Students also felt that it was important not to “dilute” the significance of a badge by issuing too many (Glover & Latif, 2013). Badges are widely shared on social media. If they are to be taken seriously, for example by a potential employer, it is reasonable that students expect the badges to have an official or institutional look and feel to them. However, there is a lack of research on what sort of badges students have included in their badge collections (for example Backpack) or shared in their online profile/CV to better understand this phenomenon as already raised by Santos et al. (2013b).

Employer perspective

The authors observed that only three publications (Halavais, Kwon, Havener, & Striker, 2014; Walker, Lee, & Lonn, 2015; Raish & Rimland, 2016) presented evidence of employer views on badges. Of these publications, Raish and Rimland (2016) have conducted a nationwide survey of employers in the US (188 human resource personnel) about the competencies they would value and the potential worth of badges to a community outside academia. Only 5% of employers indicated that they would not be interested in digital badges. Raish and Rimland (2016) argue that a student earning digital badges beyond what is required to fulfil the coursework is a clear and visual indication to the employers of their learning beyond the classroom. Nevertheless, the study showed that 62% of respondents wanted to learn more about badges. It shows the need for increasing awareness to improve adoption rate.

This again ties in with the student perception of badges and the use of badges in ones’ profile. Unless badges are valued by potential employers, there is very little incentive for badges to be used beyond the point of earning it.

Supplementing badges

The case studies presented in the literature show that in most instances badges were used on their own to recognise an achievement. However, rarely, badges were supplemented with wearable, physical badges and/or other perks.

Rughiniş and Matei (2013) present two badge architectures that they have implemented at University Politehnica of Bucharest: The Cisco Networking Academy medals and the Local Networks Course Hit List. In the former they awarded three (bronze, silver and gold) medals as rewards for final GPA (Grade Point Average) greater than 75%, 85% and 95% respectively along with rewards for other class activities such as involvement in in-class discussions or forums. The Hit List awarded a limited number of medals to the top 10% achievers on selected activities. Each medal consisted of a mention in the online ‘Hit List’ and a metallic pin badge conferred publicly.

Two publications about a school-based badging project (Wardrip, Abramovich, Bathgate, & Kim, 2014; Wardrip, Abramovich, Kim, & Bathgate, 2016) discuss the badging system implemented in an Independent Jewish school in Southern United States.  In the first year of implementation, students could choose to earn one out of four badges: information literacy, collaboration, acceptance, and empowered learning. Upon completion, students were rewarded for their success and the rewards included: ceremonies where students received an indicator of their accomplishment (a wearable badge); non-tangible rewards, which included in-school privileges such as unsupervised computer time and/or out-of-school privileges such as fieldtrip on a subject related to their badge (Wardrip, Abramovich, Kim, & Bathgate, 2016). In a similar setting, Wardrip, Abramovich, Bathgate, and Kim (2014) describe a wearable badge that can be worn around a student’s neck to publicly recognise their work. This badging system was co-developed by faculty, staff and students at the school in partnership with Global Kids, Inc. In this publication, non-tangible rewards associated with earning a badge are referred to as power-ups. These included additional in-school privileges such as unsupervised computer time. The badge earners were also awarded an exclusively catered lunch and a field trip on a subject related to their badge; for example, the students who earned the Information Literacy Badge were rewarded with a trip to the local Google office.

Harbeson and Rice (2015) present the implementation of a technology training programme using open badges, gamification and a points-based rewards system at the Carol Grotnes Belk Library and Information Commons at Appalachian State University. In this system, for each badge earned, the staff member would also accumulate a set number of points, allocated according to the difficulty of the badge, which can then be redeemed for various prizes as a part of a tiered system. The rewards system was set in six tiers: Tier I (5 points) - US$5 Bookstore Gift Card; Tier II (10 points) Tablet Carrying case; Tier III (20 points) US $25 Amazon Gift Card; Tier IV (40 points) Sony Headphones; Tier V (60 points) US$75 Amazon Gift Card; and Tier VI (80 points) Amazon Kindle Paperwhite. An internal grant had been used to fund the rewards system. Despite the appeal of this badging system for the participants, continued funding and scalability of such a badging system raise sustainability questions.

A South African project presented by Botha, Herselman, and Ford (2014) and Botha et al (2014) offered physical rewards. 166 school teachers from 11 schools participated in this project and 26 badges were offered. An Earn as you Learn system was adopted in this context whereby teachers and schools could earn technology add-ons as they progressed. For example, when a school has achieved 80% of five badges per participant they earn a projector; 80% of eight badges earn a school Mobikits (a set of 20 tablets and chargers); 80% of 11 badges earn full tablet integration into the school (including intranet link, solar charging solutions and storage facilities).

In this project the digital equipment is offered to a school based on whether the participants achieve the set standard of knowledge demonstrating the capability of using technology. This is a unique project that demonstrated the use of digital badges to ensure the use of technology once the equipment is provided to the schools, which could have otherwise been left unused due to the lack of knowledge/skills. In fact, this model could be used to distribute limited resources in resource constrained environments to improve the use of new resources and to deliver the sense of ownership to the recipients.

Badging in schools

School-based badging projects were discussed in several publications. The SamEx project was discussed in three publications (Boticki, Seow, Looi, & Baksa, 2014; Boticki, Baksa, Seow, & Looi, 2014; Boticki, Baksa, Seow, & Looi, 2015). SamEx is a mobile learning system that supports self-directed and collaborative learning activities for primary school children in Singapore. SamEx allows collecting, storing, accessing multimedia artefacts and users can give feedback for contributions. Observing pupils’ badge-earning behaviour, Boticki et al. (2015) categorise them as: Badge Hunters, Sharers, Dodgers and Explorers. Dodgers are not at all interested in earning badges while Badge Hunters are only interested in earning the badge; hence posting large amounts of low-quality content. Sharers are interested in sharing content while earning badges and their contributions are of higher quality. They respond well to teacher-directed activity but do not use the system on their own unlike the Explorers. Boticki, Baksa, Seow, and Looi (2014) thus show that badges can only encourage Badge Hunters and Sharers. However, it is likely that Badge Hunters will stop their participation when they earn the desired level of badges. This experience shows the need to consider such user behaviours in designing badge systems.

Limitations

Our work considered a systematic consideration of the existing peer-reviewed open badges literature. Other materials such as blog posts and reports are not included in this review on grounds similar to those posited by Liyanagunawardena, Adams, and Williams (2013); they argue that the inclusion of such resources in a systematic review is contentious because such work can often be highly subjective and has rarely been critically reviewed by peers.

Articles published in languages other than English were not considered for this review. The authors are also aware that the search terms used are only relevant to the English language

When searching for an identified author’s other publications, results returned by Google Scholar search engine’s first page were used. Where any other publications by the same author had not been referenced in one of the papers or had not returned in the first results page of Google Scholar search, these publications may not have been picked up. However, the possibility of this is mitigated given the use of complementary methods (database search, Google search, Google Scholar search, author search and reference list search) used to source open badges literature.

Further work

Few research projects presented open badge use with complementary physical badges, perks and/or awards. It would be an interesting avenue to explore a comparison study of online only and those with supplementary badges. Identifying the types of badges learners (and/or employers) value and would add to their badge collections would also be a useful area to explore.

Expanding the literature search to other languages and considering emerging terms such as micro-credentials which have appeared linked to badges since 2016 would be another interesting avenue to explore (Ifenthaler, Bellin-Mularski, & Mah, 2016).

User generated badges is an area where there is very little empirical research; hence this would be a worthwhile area to explore.

Conclusion

Open badges have an increasing uptake, especially with the initiatives like Badges for Lifelong Learning and Open Badge Factory Project Consortium. In this systematic literature review the authors have brought together the scattered literature on open badges to create an evidence-based review that can inform decisions to be made when adopting open badges.

Badge designs have mostly adapted hierarchical structure of badges that mostly used three levels in their designs. However, there is not enough empirical evidence to advise on one over the other.

The large majority of empirical evidence show the use of positive reinforcement badges except for one case study where negative and neutral badges were awarded. There is not sufficient evidence to suggest any (dis)advantage of using negative badges though they seem illogical given that students do not have to accept offered negative badges. However, according to student feedback from the case study, perceived usefulness of negative badges were the most important to them, which could suggest that if there were negative badges offered, students would do their best to not qualify for such badge.

It is vital to understand how badges are perceived by students and other stakeholders for badges to be an engaging motivational tool. Even then, as Boticki et al. (2015) has shown, only Badge Hunters and Sharers can be motivated by badges. Therefore, it is important that the badging systems are designed with these lessons from the literature in mind.

References

  1. Antin, J., & Churchill, E. F. (2011). Badges in social media: A social psychological perspective. CHI 2011 Gamification Workshop Proceedings. Vancouver: ACM.
  2. Botha, A., Herselman, M., & Ford, M. (2014). Gamification beyond badges. Proceedings of IST-Africa Conference, 1-10. IEEE. doi: 10.1109/ISTAFRICA.2014.6880651
  3. Botha, A., Salerno, C., Niemand, M., Ouma, S., & Makitla, I. (2014). Disconnected electronic badges in resource constrained environments: A use case from the rural Nciba district in the Eastern Cape. Proceedings of the Second International Conference on Advances in Computing, Communication and Information Technology (CCIT), 202-207. doi: 10.15224/978-1-63248-051-4-138
  4. Boticki, I., Baksa, J., Seow, P., & Looi, C.-K. (2014). We are going to the ZOO! Virtual Badges in Formal out-of-school 1: 1 Learning Journey with Smartphones. In Liu, C.-C. et al. (Eds.), Proceedings of the 22nd International Conference on Computers in Education (pp. 710-715). Japan: Asia-Pacific Society for Computers in Education.
  5. Boticki, I., Baksa, J., Seow, P., & Looi, C.-K. (2015). Usage of a mobile social learning platform with virtual badges in primary school. Computers & Education, 86, 120-136. doi: http://dx.doi.org/10.1016/j.compedu.2015.02.015
  6. Boticki, I., Seow, P., Looi, C.-K., & Baksa, J. (2014). How can Badges be Used in Seamless Mobile Learning. Proceedings of the Bristol Ideas in Mobile Learning 2014 conference. Retrieved from https://figshare.com/articles/Proceedings_Vol1_BIIML/1047193
  7. Brophy, J., & Bawden, D. (2005). Is Google enough? Comparison of an internet search engine with academic library resources. Aslib Proceedings, 57(6), 498-512.
  8. Charleer, S., Klerkx, J., Odriozola, S., Luis, J., & Duval, E. (2013). Improving awareness and reflection through collaborative, interactive visualizations of badges. ARTEL13: Proceedings of the 3rd Workshop on Awareness and Reflection in Technology-Enhanced Learning, 69-81. Retrieved from http://ceur-ws.org/Vol-1103/paper5.pdf
  9. Cucchiara, S., Giglio, A., Persico, D., & Raffaghelli, J. E. (2014). Supporting self-regulated learning through digital badges: A case study. Proceedings of International Conference on Web-Based Learning,133-142. doi: 10.1007/978-3-319-13296-9_15
  10. Ellis, D. (1989). A behavioural approach to information retrieval system design. Journal of Documentation, 45, 171-212.
  11. Ellis, D., & Haugan, M. (1997). Modelling the information seeking patterns of engineers and research scientists in an industrial environment. Journal of Documentation, 53, 384-403.
  12. Fink, A. (2010). Conducting research literature reviews: From Internet to paper (3rd ed.). Thousand Oaks, California: Sage.
  13. Gao, F., Luo, T., & Zhang, K. (2012). Tweeting for learning: A critical analysis of research on microblogging in education published in 2008-2011. British Journal of Educational Technology, 43(5), 783-801.
  14. Gibson, D., Ostashewski, N., Flintoff, K., Grant, S., & Knight, E. (2015). Digital badges in education. Education and Information Technologies, 20(2), 403-410. doi: http://dx.doi.org/10.1007/s10639-013-9291-7
  15. Glover, I., & Latif, F. (2013). Investigating Perceptions and Potential of Open Badges in Formal Higher Education. In J. Herrington, A. Couros, & V. Irvine (Eds.) Proceedings of EdMedia: World Conference on Educational Media and Technology (pp. 1398-1402).
  16. Halavais, A., Kwon, K.H., Havener, S., & Striker, J. (2014). What a Badge Is Worth: The Acceptability of Badges as Indicators of Experience. 2014 Annual meeting of the American Educational Research Association.
  17. Harbeson, C., & Rice, S. (2015). Training Library Staff with Badges and Gamification. In B. L Eden (Ed.), Enhancing Teaching and Learning in the 21st Century Academic Library (pp. 121-134).
  18. Ifenthaler, D., Bellin-Mularski, N, & Mah, D (Eds.) (2016). Foundation of Digital Badges and Micro-Credentials. Springer.
  19. International Scouting Collectors Association. (n.d.). Collecting Merit Badges: The ISCA Getting Started Collecting Series. Retrieved from http://www.scouttrader.org/collecting/meritbadge.pdf
  20. Jamali, H. R., & Asadi. S. (2010). Google and the scholar: the role of Google in scientists’ information-seeking behaviour. Online information review, 34(2), 282-294.
  21. Kappes, S., & Betro, V. C. (2015). Using Mozilla badges to certify XSEDE users and promote training. Proceedings of the 2015 XSEDE Conference: Scientific Advancements Enabled by Enhanced Cyberinfrastructure. ACM. doi: 10.1145/2792745.2792759
  22. Knight, E., & Casilli, C. (2012). Mozilla Open Badges. In D. G. Oblinger (Ed.). Game Changers: Education and Information Technologies (pp. 279-284). EDUCAUSE.
  23. Liyanagunawardena, T. R., Adams, A. A., & Williams, S. A. (2013). MOOCs: a systematic study of the published literature. International Review of Research in Open and Distributed Learning, 14(3), 202-227.
  24. Liyanagunawardena, T. R., & Williams, S. A. (2016). Elderly learners and Massive Open Online Courses: a review. Interactive Journal of Medical Research, 5(1), e1. ISSN 1929-073X.
  25. Mewburn, I., Freund, K., & Rutherford, E. (2014). Badge trouble: piloting open badge at the Australian National University. ASCILITE 2014. Dunedin, New Zealand. Retrieved from http://ascilite.org/conferences/dunedin2014/files/concisepapers/233-Mewburn.pdf
  26. Mozilla Foundation, The and Peer 2 Peer University in collaboration with The MacArthur Foundation (2012). Open Badges for Lifelong Learning. Retrieved from https://wiki.mozilla.org/images/b/b1/OpenBadges-Working-Paper_092011.pdf
  27. Mozilla Foundation (2016). About Open Badges. Retrieved from https://openbadges.org/about/
  28. Mozilla Foundation. (2017). Backpack. Retrieved from https://backpack.openbadges.org/backpack/welcome
  29. Nicholas, D., Clark, D., Jamali, H. R., & Watkinson, A. (2014). Log Usage Analysis: What it Discloses about Use, Information Seeking and Trustworthiness. International Journal of Knowledge Content Development & Technology, 4(1), 23-37. doi: 10.5865/IJKCT.2014.4.1.023
  30. Namei, E., & Young, C. A. (2015). Measuring our relevancy: Comparing results in a web-scale discovery tool, Google and Google Scholar. Proceedings of ACRL ‘15, Portland, Oregon, USA, 522-535.
  31. Põldoja, H., & Laanpere, M. (2014). Exploring the potential of open badges in blog-based university courses. Proceedings of the International Conference on Web-Based Learning, 172-178. Springer International Publishing.
  32. Raish, V., & Rimland, E. (2016). Employer Perceptions of Critical Information Literacy Skills and Digital Badges. College & Research Libraries, 77(1), 87-113.
  33. Randall, D. L., Harrison, J. B., & West, R. E. (2013). Giving credit where credit is due: Designing open badges for a technology integration course. TechTrends, 57(6), 88-95.
  34. Rughiniş, R., & Matei, S. (2013). Digital badges: Signposts and claims of achievement. Proceedings of the International Conference on Human-Computer Interaction, 84-88, Berlin: Springer.
  35. Santos, C., Pedro, L., Almeida, S., & Aresta, M. (2013a). Decentralized badges in educational contexts: the integration of open badges in SAPO campus, eLearning Papers, 35. Retrieved from https://www.openeducationeuropa.eu/sites/default/files/asset/From-field_35_4.pdf
  36. Santos, J. L., Charleer, S., Parra, G., Klerkx, J., Duval, E., & Verbert, K. (2013b). Evaluating the use of open badges in an open learning environment. Proceedings of the European Conference on Technology Enhanced Learning, 314-327. Berlin: Springer.
  37. Tripadvisor, United Kingdom (n.d). Trip Collective Badges: What they are and how to receive them. Retrieved from https://www.tripadvisor.co.uk/TripCollectiveBadges
  38. Walker, A. M., Lee, F., & Lonn, S. (2015). Scaffolds: Experimenting with student-driven open badging in an iSchool context. iConference 2015 Proceedings. Retrieved from http://hdl.handle.net/2027.42/110790
  39. Wardrip, P. S., Abramovich, S., Bathgate, M., & Kim, Y. J. (2014). A School-Based Badging System and Interest-Based Learning: An exploratory case study. International Journal of Learning and Media. Retrieved from http://www.acsu.buffalo.edu/~samuelab/IJLM_Badge_Paper.pdf
  40. Wardrip, P. S., Abramovich, S., Kim, Y. J., & Bathgate, M. (2016). Taking badges to school: A school-based badge system and its impact on participating teachers. Computers & Education, 95, 239-253. doi: 10.1016/j.compedu.2016.01.008
  41. Williams, S., Terras, M., & Warwick, C. (2013). What people study when they study Twitter: Classifying Twitter related academic papers. Journal of Documentation, 69(3), 384-410.

Appendices

Appendix 1 – List of All Identified Publications

https://drive.google.com/file/d/0B60U5wToyeSNWXg4dE16bUxodnM/view?usp=sharing

Appendix 2 – Summary of Open Badges Projects in the Literature

https://drive.google.com/file/d/0B60U5wToyeSNWjdITE03ek1TQWs/view?usp=sharing

Acknowledgement

The authors would like to thank Ms Katalin Hanniker for her help with proofreading the paper.

 

Tags

e-learning, distance learning, distance education, online learning, higher education, DE, blended learning, ICT, information and communication technology, internet, collaborative learning, learning management system, MOOC, interaction, LMS,

Current issue on De Gruyter Online

– electronic content hosting and distribution platform

EURODL is indexed by ERIC

– the Education Resources Information Center, the world's largest digital library of education literature

EURODL is indexed by DOAJ

– the Directory of Open Access Journals

EURODL is indexed by Cabells

– the Cabell's Directories

EURODL is indexed by EBSCO

– the EBSCO Publishing – EBSCOhost Online Research Databases

For new referees

If you would like to referee articles for EURODL, please write to the Chief Editor Ulrich Bernath, including a brief CV and your area of interest.