Perceived Credibility of Scientific Content on Facebook by Grade-School Students: The Effect of Popularity

Arnon Hershkovitz [arnon.hershkovitz@gmail.com], Nagat Badarneh [nagatb@mail.tau.ac.il], Tel Aviv University, 30 Haim Levanon St., Tel Aviv 6997801 [https://www.tau.ac.il], Israel

Abstract

In this quantitative study (N = 303), we explore children’s and teen’s credibility assessment of scientific information on Facebook, and its associations with content popularity. By manipulating content familiarity and popularity, we find interesting differences between age-levels. While popularity did not play an important role in the 12th-graders’ credibility evaluation (who were mostly affected by content familiarity), it was prominent for the 6th-graders (where high-popularity compensated for non-familiarity); the 9th-graders, as an intermediate age-level, demonstrate an equal influence of both familiarity and popularity.

Abstract in Hebrew

מחקר כמותי זה (N=303), אנו בוחנים כיצד ילדים ונערים מעריכים אמינות מידע מדעי המפורסם בפייסבוק, בפרט בהתייחס לפופולריות המידע. באמצעות מניפולציה על מידת ההיכרות עם התוכן ועל מידת הפופולריות שלו, אנו מוצאים הבדלים בין-גילאיים מעניינים. בעוד מידת הפופולריות לא שיחקה תפקיד משמעותי בהערכת האמינות בקרב תלמידי כיתות י"ב (שהיו מושפעים בעיקר ממידת ההיכרות עם התוכן), היא היתה משמעותית מאוד בקרב תלמידי כיתות ו' (ופיצתה על אי-היכרות עם התוכן); תלמידי כיתות ט' היוו שכבת ביניים, אצלם נרשמה השפעה דומה הן של מידת ההיכרות עם התוכן והן של מידת הפופולריות.

Abstract in Spanish

En este estudio cuantitativo (N = 303) exploramos la evaluación de credibilidad por parte de niños y adolescentes sobre la información científica en Facebook y su asociación con la popularidad del contenido. Manipulando la familiaridad y popularidad del contenido, observamos diferencias interesantes entre los niveles de edad. Si bien la popularidad no jugó un papel importante en la evaluación de la credibilidad por parte de los estudiantes de 12º grado (quienes se vieron afectados principalmente por la familiaridad con el contenido), fue prominente para los estudiantes de 6º grado (donde la alta popularidad compensó la falta de familiaridad); los estudiantes de 9º grado, con un nivel de edad intermedio, demuestran una influencia similar de familiaridad y popularidad.

Keywords: credibility evaluation, content popularity, social networking sites, Facebook, science communication.

Introduction

The credibility attained by information consumers to a given message – to put it simply, the extent to which they believe the message – is key to its comprehension and acceptance (Heesacker, Petty, & Cacioppo, 1983). However, assessing credibility is a demanding cognitive task, and as people are constantly exposed online to huge amounts of information streams in a plethora of digital channels, it is unreasonable to expect the implementation of a thorough credibility assessment process for every piece of information found online. Rather, people often rely on various cues and on some cognitive heuristics to determine whether they think a piece of content is true or not, in particular in cases in which they are not familiar with the subject matter (Fogg, 2003; Metzger, 2007; Petty & Cacioppo, 1986; Sundar, 2008).

This holds even more true for younger audiences, as a large body of literature argues that children and youth barely assess credibility (cf. Gasser, Cortesi, Malik, & Lee, 2012). Instead, they heavily rely on peripheral cues, like polarity of comments and image types (Kang, Höllerer, & O’Donovan, 2015; O’Reilly & Marx, 2011; Subramaniam et al., 2015).

Social networking sites (SNS) – like Facebook, Instagram, or YouTube – are amongst the most popular websites for young Internet users (Lenhart, 2015; The Associated Press-NORC Center for Public Affairs Research, 2016). Notably, these platforms may serve as catalysts for learning, while users interacting with streams of knowledge in new ways (Buchem, 2011; Haugsbakken & Langseth, 2014; Kop, 2012). As these platforms are characterized by a very high activity volume, which translates into a very high volume of shared and consumed content, it is of great importance to explore the ways young audiences perceive credibility of scientific information posted on SNS.

Originally aimed for social interactions, SNS have developed into major channels of news consumption (Hermida, Fletcher, Korell, & Logan, 2012; Kwak, Lee, Park, & Moon, 2010; Shearer & Gottfried, 2017; Tandoc & Johnson, 2016). However, unlike traditional news venues (e.g., newspapers or television), sharing news on SNS can be easily done by any user, which makes credibility assessment more challenging on these platforms than ever before (Bucchi, 2017; Sharon, Ryder, Osborne, Laslo, & Swirski, 2017), especially for children, who dissociate message and source (Gasser et al., 2012).

Usually, content posted on SNS is enriched by additional information, which is automatically added by the website, specifically details about the user who posted it and measures of popularity. Both these types of information have clear social orientation, at either the personal or the aggregated level, and both of them may impact the users’ perceived credibility of the content (Hayat & Hershkovitz, in press; Hayat, Hershkovitz, & Azran, in press). Content popularity (updated to the moment) may be measured by one or more means; for example, a Facebook user can clearly see the number of Likes every Post got, how many Comments did the network users add to it, and how many have Shared it. One of the heuristics related to this added information is the bandwagon heuristics, according to which “if others think that this is a good story, then I should think so too” (Sundar, 2008; p.83). Indeed, recent studies demonstrate this effect in practice, in the context of social media (Granjon & Benedic, 2017; Huang, 2015; Lin, 2016; Waddell, 2015).

Nevertheless, there is still lack of evidence regarding the effects of popularity of scientific content shared via SNS on credibility among children and teens. Exploring this is the main purpose of the current paper.

Methodology

Design

In order to explore the associations between content popularity and its perceived credibility among children and teens, we designed an experimental study, taking a between-subject approach. The experiment compared perceived credibility of scientific content that was posted on Facebook in four conditions, based on two independent variables: familiarity with the content (Familiar/Unfamiliar), and popularity (High vs. Low popularity), among 6th-, 9th-, and 12th-grade students.

Population

310 students from a few public schools in Arab towns at the north of Israel participated in this study, of whom 104 (34%) were at 6th-grade, 104 (34%) were at 9th-grade, and 102 (33%) were at 12th-grade. Data was collected in schools, during May-June 2016.

Research Groups (Conditions) and Procedure

As mentioned above, the research included a comparison of four groups, differed by the characteristics of the Facebook-post presented to them, as manipulated by two variables. First, we manipulated the familiarity of the content. Participants were exposed to either a familiar content (“Every person’s fingerprint is unique, not even identical twins have matching fingerprints”) or an unfamiliar one (“When mosquitos bite you, they also urinate on you”). Posts containing these contents were taken as-is from a Facebook page that posts daily scientific facts. (This page is in Arabic, and as Arabic is the mother tongue of our participants, the whole questionnaire was written in Arabic).

To put it clear, whether these facts are true or not has nothing to do with their perceived credibility. For validating the familiarity/unfamiliarity of the chosen content, one of the items in the questionnaire was: “I have knowledge on this topic” (6-point Likert scale). Indeed, mean rank for the familiar status was 5.6 (SD = 1.0, N = 155), compared with a mean rank of 1.1 (SD = 0.4, N = 155) for the unfamiliar status. This difference is statistically significant, with t(190.0) = 50.3, at p < 0.001; since Levene’s test was significant, with F = 62.7, at p < 0.001, we did not assume equal variances.

Second, we manipulated popularity of content, with low-popularity posts have 2 or 3 Likes, and high-popularity posts have 7,530 or 8,250 Likes. These were edited by us and were not the original number of Likes the posts had had.

Besides the post that was presented to them, participants were asked to answer on a few questions and to rank a few items, as is detailed below.

Measures

Dependent Variable

Perceived Credibility

To measure this variable in a simple way that will be understandable to young children, we decided to simply ask them about the level of perceived credibility they put in this status. The corresponding question was: “To what extent do you believe to the content of this status?” (4‑point Likert scale).

Background Variables

Demographics

Age and Gender were reported by all participants (N = 310). Age ranged 11-18 (M = 15.0, SD = 2.4). Overall, there were 158 (51%) female student, and 152 (49%) male students.

Facebook Usage

Participants were asked to report whether they had a Facebook account (Yes/No). Only 7 students (2.3%) did not have a Facebook account, therefore we decided to omit them from the analysis. The latter were asked to report how many years had they used Facebook (M = 2.4, SD = 1.3, N = 303), how many hours per day (approximately) they were using it (M = 1.5, SD = 0.7, N = 303), and how many Facebook-friends did they have (M = 248.5, SD = 0.4, N = 303).

In order to assess the level of activeness of using the site, participants were asked to rank, on a 4-poin Likert scale, the frequency of using Facebook for each of 13 activities (reading my Wall, writing posts on my Wall, sharing others’ posts, uploading photos, sharing others’ photos, tagging photos, reading in groups, writing in groups, joining event pages, hitting Like to others’ posts, commenting on others’ posts, chatting, playing). Then, an activity intensity was calculated as the mean of these items. Overall, it has an average of 2.91 (SD = 0.4, N = 303).

Findings

In the rest of this paper, we will refer to these four conditions as follows: Familiar-High, Familiar-Low, Unfamiliar-High, and Unfamiliar-Low. All analyses were done using SPSS™ Version 24.

Perceived Credibility and the Background Variables

For all the analyses reported in this sub-section, N = 303.

Overall, Perceived Credibility score took a mean of 2.7 (SD = 1.3). No differences were found in Perceived Credibility between male and female students, with t(301) = 0.8, at p = 0.42. Also, no significant correlations were found between Perceived Credibility and age (Pearson’s r = ‑0.02, p = 0.78), grade-level (Spearman’s ρ = 0.01, p = 0.86).

Perceived Credibility (N = 303) was not associated with Facebook experience (r = 0.03, p = 0.65), number of Facebook friends (r = 0.06, p = 0.32), or Facebook activity intensity (r = 0.05, p = 0.36). It was found significantly positively associated with Facebook daily use, however to a low degree, with r = 0.12, at p < 0.05.

Perceived Credibility and the Independent Variables (Separately)

Overall, Perceived Credibility was found to be significantly higher in the Familiar condition (M = 3.7, SD = 0.8, N = 152) than in the Unfamiliar condition (M = 1.7, SD = 0.9, N = 151), with t(296.6) = 20.9, at p < 0.001; as Levene’s test was found significant (F = 7.7, at p < 0.01), we did not assume equal variances. Therefore, effect size is calculated using Glass’s D (Glass, McGaw, & Smith, 1981), with the standardizer being the SD of the unfamiliar group, and gives D = 2.27.

As expected, Perceived Credibility was found to be significantly higher in the High-popularity condition (M = 3.1, SD = 1.11, N = 150) than in the Low-popularity condition (M = 2.3, SD = 1.3 N = 153), with t(296.7) = 5.9, at p < 0.001; as Levene’s test was found significant (F = 7.4, at p < 0.01), we did not assume equal variances. Effect size is D = 0.64 (with low-popularity group’s SD as standardizer).

Perceived Credibility in the Four Conditions

Interestingly, all participants in the Familiar High-Popularity group gave the post the highest credibility score of 4, so Perceived Credibility in this group has a mean of 4 and SD = 0 (N = 75), which is, obviously, the highest mean among the four research groups. Then, Perceived Credibility takes a mean of M = 3.3 (SD = 0.9, N = 77) at the Familiar Low-Popularity group; M = 2.2 (SD = 1.0, N = 75) at the Unfamiliar High-Popularity group; and, finally, M = 1.3 (SD = 0.4, N = 76) at the Unfamiliar Low-Popularity. Using One-Way ANOVA, we can see that these differences are statistically significant, with F(3) = 233.2, at p < 0.001. Post-hoc Scheffe’s test demonstrates that all pair-wise comparisons between the groups result with significant differences, as summarized in Table 1.

Table 1:  Absolute mean difference in Perceived Credibility between groups, using post-hoc Scheffe’s test

Breaking Down by Grade-Level

We first repeat testing for associations that were previously reported, this time by grade-level. In all grade-levels, there are no significant differences in Perceived Credibility between male and female students. Running independent t-test, we get t(97) = 0.74, at p = 0.46, for 6th-grade; t(100) = 0.35, at p = 0.73, for 9th-grade; and t(100) = 0.33, at p = 0.75, for 12th-grade.

However, some interesting differences arise when correlating, in each of the grade-levels separately, Perceived Credibility with the four variables that measure Facebook activity. For the 6th-graders, the dependent variable is not associated with neither of these variables; for the 9th-graders, it is positively correlated with Facebook experience to a low extent (r = 0.24, at p < 0.05); and for the 12th-graders, it is positively correlated with Facebook activity intensity to a low extent (r = 0.34, at p < 0.001). Results are summarized in Table 2.

Table 2:  Correlation of Perceived Credibility and the four variables that measure Facebook activity, by grade-level (significant results are shaded in grey)

Across all grade-levels, Perceived Credibility was found to be significantly higher in the Familiar condition than in the Unfamiliar condition. In 6th- and 9th-grade levels, Perceived Credibility was found to be significantly higher in the High-popularity condition than in the Low-popularity condition, there was no difference between these conditions for the 12th-grade students. Results are summarized in Table 3.

Table 3:  Comparing Perceived Credibility between Familiar/Unfamiliar and Low-/High-Popularity conditions, in each grade-level

Finally, we compare between Perceived Credibility among the four groups defined by the two independent variables, in each grade-level separately; each such group in each grade level has (N = 25±1). At all grade-levels, mean Perceived Credibility is minimal over the Unfamiliar Low-Popularity group; then, it is higher over the Unfamiliar High-Popularity, and further higher over the Familiar Low-Popularity group; finally, it is maximal over the Familiar High-Popularity group. This can be clearly seen in Figure 1. At all grade-levels, differences between the four research groups are significant, with One-Way ANOVA test gives F(3) = 76.5, F(3) = 72.4, and F(3) = 151.4, for 6th-, 9th-, and 12th-grades, respectively, all at p < 0.0001.

Post-hoc Scheffe’s tests show that at the 6th-grade level, all but one pair-wise comparison between groups resulted with significant difference; the only non-significant pair is Unfamiliar High-Popularity and Familiar Low-Popularity (p = 0.60). At the 9th-grade level, all pair-wise comparisons resulted with significant differences. Finally, at the 12th-grade level, all but two pair-wise comparisons resulted with significant difference; the non-significant pairs are Unfamiliar High-Popularity and Unfamiliar Low-Popularity, and Familiar High-Popularity (p = 0.06) with Familiar Low-Popularity (p = 0.32), which is in line with the abovementioned non-significant association with level of popularity and perceived credibility for that age group.

Figure 1

Figure 1. Comparing Perceived Credibility between conditions, in each grade-level separately

Discussion

In this study, we explored associations between perceived credibility of scientific information posted on Facebook and the content’s familiarity and popularity, among children and teens. In today’s era of information explosion, millions of children and teens are constantly exposed to information via Social Networking Sites (SNS). This information is often not checked for correctness or accuracy, and can be shared by virtually anyone connected to the site. Hence, credibility assessment, to be carried out by the information consumers, is a key means to distinguish between true and fake. However, as the process of assessing credibility is highly demanding and time-consuming, people often use cognitive heuristics to take a shortcut; mostly, they do so when the content is not familiar to them (Petty & Cacioppo, 1986). One of these heuristics has to do with measures of popularity (Sundar, 2008), which is our focus here. More specifically, we empirically tested for the credibility of scientific contents, as perceived by elementary-, middle-, and high-school students in four conditions, as derived by two manipulated variables: content familiarity (high/low) and content popularity (high/low).

Overall, it was found that Perceived Credibility is not associated with the background variables (age, gender, Facebook use. Perceived Credibility was found to be positively associated with content familiarity and popularity. While the former is well understood, the latter demonstrates the effects of familiarity and popularity on the credibility assessment process. From a wider point of view, these findings, taken together, may illuminate that for young information consumers, Credibility lies more in the information to be evaluated than it does in the person evaluating it. We add on previous studies, which found associations between source popularity and content credibility assessment on SNS (Edwards, Spence, Gentile, Edwards, & Edwards, 2013; Granjon & Benedic, 2017), by highlighting the influence of the content popularity.

Nevertheless, when breaking down our population by grade-level, we see some evidence (albeit, quite limited) for Facebook experience that is associated with the credibility assessment process. While Perceived Credibility is not associated with any of the Facebook-related activity measurement in the 6th-grade population, it is positively associated with Facebook Experience in the 9th-grade population, and is positively associated with Facebook Activity Intensity in the 12th-grade population. Which means that along adolescence, some cognitive, social or emotional changes may impact the credibility assessment process. This is clearly evident by our finding that the younger our participants are, the more prone they are to be influenced by popularity. While popularity did not play an important role in the 12th-graders’ credibility evaluation (they only distinguished between familiar and not-familiar content), for the 6th-graders -high-popularity somehow compensated for non-familiarity (they did not distinguish between unfamiliar highly-popular and familiar poorly-popular contents); the 9th-graders, as an intermediate age-level, demonstrate an equal influence of both familiarity and popularity. These findings may also be associated with general perceptions of trust in online resources, as it was found that younger children tend to trust what they find online more than older children (e.g., Ofcom, 2015); therefore, we should control for this general phenomenon in future explorations.

Previous studies suggested unique characteristics of credibility assessment among young children, including social influence (Schwarz & Morris, 2011; Subramaniam et al., 2015). Our findings contribute a more nuanced understanding of that process among children and teens, which may have explained by their use of SNS. Hence, we shall further explore cognitive and meta-cognitive aspects of credibility assessment, which may be related to familiarity with the platform (Fogg, 2003).

References

  1. Bucchi, M. (2017). Credibility, expertise and the challenges of science communication 2.0. Public Understanding of Science, 26(8), 890–893.
  2. Buchem, I. (2011). Serendipitous learning: Recognizing and fostering the potential of microblogging. Form@re – Open Journal per La Formazione in Rete, 11(74), 7–16.
  3. Edwards, C., Spence, P. R., Gentile, C. J., Edwards, A., & Edwards, A. (2013). How much Klout do you have – A test of system generated cues on source credibility. Computers in Human Behavior, 29(5), A12–A16.
  4. Fogg, B. J. (2003). Prominence-interpretation theory: Explaining how people assess credibility online. Proceedings of the Conference on Human Factors in Computing Systems (April 5-10, Fort Lauderdale, FL), 722–723.
  5. Gasser, U., Cortesi, S., Malik, M., & Lee, A. (2012). Youth and digital media: From credibility to information quality (research publication No. 2012-1). Cambridge, MA: Berkman Klein Center for Internet & Society.
  6. Glass, G. V., McGaw, B., & Smith, M. L. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage Publications.
  7. Granjon, V., & Benedic, R. (2017). Instagram’s social media influencers: A study of online popularity from source credibility to brand attitude (unpublished Masters’ thesis). Uppsala University, Uppsala, Sweden.
  8. Haugsbakken, H., & Langseth, I. (2014). YouTubing: Challenging traditional literacies and encouraging self-organisation and connecting in a connectivist approach to learning in the K‑12 system. Digital Culture & Education, 4(3), 133–151.
  9. Hayat, T., & Hershkovitz, A. (in press). The role social cues play in mediating the effect of eWOM over purchasing intentions. Journal of Costumer Behaviour.
  10. Hayat, T., Hershkovitz, A., & Azran, T. S. (in press). The independent reinforcement effect: Diverse social ties and the credibility assessment process. Public Understanding of Science.
  11. Heesacker, M., Petty, R. E., & Cacioppo, J. T. (1983). Field dependence and attitude change: Source credibility can alter persuasion by affecting message-relevant thinking. Journal of Personality, 51(4), 653–666.
  12. Hermida, A., Fletcher, F., Korell, D., & Logan, D. (2012). Share, like, recommend: Decoding the social media news consumer. Journalism Studies, 13(5–6), 815–824.
  13. Huang, L. S. (2015). Trust in product review blogs: The influence of self-disclosure and popularity. Behaviour and Information Technology, 34(1), 33–44.
  14. Kang, B., Höllerer, T., & O’Donovan, J. (2015). Believe it or not? Analyzing information credibility in microblogs. Proceedings of the IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (August 25-28, Paris, France), 611–616.
  15. Kop, R. (2012). The unexpected connection: Serendipity and human mediation in networked learning. Journal of Educational Technology & Society, 15(2), 2-11.
  16. Kwak, H., Lee, C., Park, H., & Moon, S. (2010). What is Twitter, a social network or a news media? Proceedings of the 19th International Conference on World Wide Web (April 26-30, Raleigh, NC), 1–10.
  17. Lenhart, A. (2015). Teens, social media and technology overview 2015. Washington, DC: Pew Research Center.
  18. Lin, X. (2016). Social media and credibility indicator: The effects of bandwagon and identity cues within online health and risk contexts (unpublished Ph.D. dissertation). University of Kentucky, Lexington, KY.
  19. Metzger, M. J. (2007). Making sense of credibility on the Web: Models for evaluating online information and recommendations for future research. Journal of the American Society for Information Science and Technology, 58(13), 2078–2091.
  20. O’Reilly, K., & Marx, S. (2011). How young, technical consumers assess online WOM credibility. Qualitative Market Research: An International Journal, 14(4), 330–359.
  21. Ofcom (2015). Children and parents: Media use and attitudes report. London, UK: Ofcom.
  22. Petty, R. E., & Cacioppo, J. T. (1986). The elaboration likelihood model of persuasion. Advances in Experimental Social Psychology, 19, 123–183.
  23. Schwarz, J., & Morris, M. R. (2011). Augmenting Web pages and search results to help people find trustworthy information online. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (May 7-12, Vancouver, BC), 1245–1254.
  24. Sharon, A. J., Ryder, J., Osborne, J., Laslo, E., & Swirski, H. (2017, October). 7 steps to evaluating science online. Sage Public Understanding of Science Blog [Blog post]. Retrieved from https://sagepus.blogspot.com/2017/10/7-steps-to-evaluating-science-online.html
  25. Shearer, E., & Gottfried, B. Y. J. (2017). News use across social media platforms 2017. Washington, DC: Pew Research Center.
  26. Subramaniam, M., Taylor, N. G., St. Jean, B., Follman, R., Kodama, C., & Casciotti, D. (2015). As simple as that?: Tween credibility assessment in a complex online world. Journal of Documentation, 71(3), 550–571.
  27. Sundar, S. S. (2008). The MAIN model: A heuristic approach to understanding technology effects on credibility. In M. J. Metzger & A. J. Flanagin (Eds.), Digital media, youth, and credibility (pp. 73–100). Cambridge, MA: MIT Press.
  28. Tandoc, E. C., & Johnson, E. (2016). Most students get breaking news first from Twitter. Newspaper Research Journal, 37(2), 153–166.
  29. The Associated Press-NORC Center for Public Affairs Research. (2016). Instagram and Snapchat are most popular social networks for teens; Black teens are most active on social media, messaging apps. Chicago, IL.
  30. Waddell, T. F. (2015). What does the crowd think? How online comments and popularity metrics affect news credibility and issue importance. New Media & Society, 34(1), 33–44. >

     

Tags

e-learning, distance learning, distance education, online learning, higher education, DE, blended learning, ICT, MOOCs, information and communication technology, collaborative learning, internet, learning management system, interaction, LMS,

Current issue on Sciendo

– electronic content hosting and distribution platform

EURODL is indexed by ERIC

– the Education Resources Information Center, the world's largest digital library of education literature

EURODL is indexed by DOAJ

– the Directory of Open Access Journals

EURODL is indexed by Cabells

– the Cabell's Directories

EURODL is indexed by EBSCO

– the EBSCO Publishing – EBSCOhost Online Research Databases

For new referees

If you would like to referee articles for EURODL, please write to the Chief Editor Ulrich Bernath, including a brief CV and your area of interest.