Methodology for identifying and tracking social media misinformation in tweets about the impact of the COVID-19 pandemic on reproductive health

Abstract

The purpose of the study was to develop the methodology identifying and tracking social media misinformation in tweets about the impact of the coronavirus and COVID-vaccine on reproductive health, one of the reasons for which is the lack of awareness about aspects of the coronavirus infection. We use a combination of machine and expert methods, and use the latest scientific articles as the standard for detecting disinformation. The proposed methodology includes the study of scientific articles as a source of reliable truthful information about the topic (information standard) and Twitter messages (assessment of information compliance with the standard). The result of the study is a methodology for detecting disinformation in the messages of social network users. Based on this methodology, the following aspects of the problem have been developed: 1) the formation of a scientific standard; 2) the principle of comparing the directions of scientific research and discussions on Twitter; 3) the principle of contextual comparison of user and scientific ideas about problems. An original methodology for identifying disinformation in social networks is proposed. In contrast to existing works, principles based on the processing of information from the content of scientific articles and messages from social networks are formulated.

1. Introduction

2. Materials And Methods

3. Results

3.1. Scientific terminology (PubMed)

Terms related to the field of coronavirus and fertility were divided by experts into thematic categories [15]. Eleven thematic categories were identified: Genetics and molecular biology, COVID and other diseases, Coronavirus, Infertility (without gender), Female reproductive health (FRH), Male reproductive health (MRH), COVID vaccination, Research objects, Methods, Relationships, Marker terms (sentiment words). The minor categories were: 1) relationships (couple relationship, sexual behavior, sexual, etc.); 2) methods (antioxidant, drug, ultrasound, anti-inflammatory, etc.); 3) marker terms (anxiety, emotion(al), death, psychological, etc.). 

A total of 388 terms were identified in 11 groups. The frequency of the most common term in each category is taken as 100%. Top-5 terms for the major categories of scientific terminology related to “Coronavirus and Fertility” are listed below:

• “Coronavirus” (COVID-19, 100%; SARS CoV-2, 55%; pandemic, 45%; SARS-CoV-2 infection, 38%; COVID-19 pandemic, 34%);
• “COVID and other diseases” (stress(or), 100%; immune, 73%; diabetes, 32%; endocrine, 27%;

Inflammation, 27%);

• “Female reproductive health” (pregnancy, 100%; pregnancy rate, 35%; assisted reproduction technology (ART), 33%; oocyte, 33%; embryo, 25%);
• “Male reproductive health” (male fertility, 100%; Semen, 56%; testicular, 50%; testosterone (T), 44%; semen parameters, 44%);
• “Infertility (without gender)” (fertility, 100%; reproductive, 47%; infertility, 48%; fertility treatment, 26%; Fertility preservation (FP), 24%);
• “COVID vaccination” (COVID-19 vaccination, 100%; vaccine, 73%; vaccination, 65%; miR-371a-3p, 19%; side effects, 16%);
•   “Genetics and molecular biology” (express(ion), 100%; angiotensin converting enzyme (ACE2), 67%; hub genes, 53%; luteinizing hormone (LH), 47%; protein, 47%);
• “Research objects” (woman, 100%; child(ren), 57%; animal (rabbit, rat, mice), 45%; pregnant, 41%; male, 39%).

3.2. Social network terminology (Twitter)

Terms related to the field of coronavirus and fertility were divided by experts into thematic categories. Nine thematic categories were identified: COVID and other diseases, Coronavirus, Infertility (without gender), Female reproductive health (FRH), Male reproductive health (MRH), COVID vaccination, Discussion objects, Marker terms (sentiment words), Conspiracy theory. The minor category was the Conspiracy theory (poison, depopulation, Microchips, etc.).

A total of 220 terms were identified in 9 groups. The frequency of the most common term in each category is taken as 100%. Top-5 terms for the major categories tweets related to Coronavirus and Fertility are listed below:

-     “Coronavirus” (COVID(-19), 100%; get covid, 10%; Long Covid, 7%; virus, 7%; COVID(-19) infection, 5%);

-     “COVID and other diseases” (lung, 100%; cancer, 67%; heart attack, 67%; myocarditis, 67%; heart, 67%);

-     “Female reproductive health” (pregnancy, 100%; baby, 43%; birth, 29%; pregnant women, 21%; in vitro fertilization (IVF) , 14%);

-     “Male reproductive health” (male infertility, 100%; erectile dysfunction, 62%; sperm, 37%; sperm count, 37%; sex, 37%);

-     “Infertility (without gender)” (fertility, 100%; infertility, 52%; sterility, 12%; fertility issues, 11%; fertility rate, 11%); 

-     “COVID vaccination” (COVID(-19) vaccine, 100%; vaccine, 98%; vaccinate, 38%; Covid(-19) Vaccination, 26%; vax, 18%);  

-     “Markers” (risk, 100%; death, 62%; damage, 56%; recover, 50%; safe, 37%);

-     “Discussion objects” (people, 100%; men, 90%; women, 75%; children, 75%; population, 70%).

3.3. Comparison of scientific and social network terminologies

For matching categories in scientific and social network terminologies, a comparison was made by frequency terms. Terminological usage (frequency) indicates how often the terms of a given category are used in a discussion (Figure 4).

3.4. Detection of true and false judgments

3.4.1. Revealing scientific judgments (PubMed)

In the previous sections, we found that the interests of the scientific community and the community of Internet users coincide in many ways. Our next task was to find out how the opinions of Twitter users differ from scientific judgments. The following topics were chosen for the study: Male reproductive health and coronavirus, Male reproductive health and the vaccine, Female reproductive health and coronavirus, Female reproductive health and the vaccine. The contextual analysis described in the Methods section was used.

We applied contextual analysis of scientific texts. The contextual analysis included determining the evidence of the effect of coronavirus (COVID) on a reproductive system (organ, parameter) of the object of study, indicating the mechanism of action (or without it).   Examples of results are shown in the table 1.

Table 1.   Some results of contextual analysis of scientific texts related to “Reproductive system and Coronavirus”

For topics “Male reproductive health and the vaccine”, “Female reproductive health and the vaccine” (table 2) we show only few results without mechanism of action because they are similar mainly.

Table 2.   Some results of contextual analysis of scientific texts related to “Male/Female reproductive system and Vaccine”

We divided statements about the confirmation and denial of the impact of the coronavirus/vaccine on the reproductive health of men and women. Confirmatory markers include the following: “is significantly reduced”, “negative impact”, “can affect”, “decreased”, “reduced”, “downregulation”, “has been reported”, “observed”, “participates in”, “strong association”, “dysregulation”, “damage”, “direct effects”, etc. Negative markers include the following: “very little evidence”, “no viral RNA was detected”, “unclear”, “many unresolved questions”, “no evidence”, “limited evidence”, “remains unknown”, “does not seem to affect”, “no significant changes”, “did not affect”, “does not impair”, etc.

We then calculated the ratio of confirmatory (“yes”) and negative (“no”) markers in scientific documents by topics “Male reproductive health and coronavirus” (77,59% for “yes” and 22,41% for “no”), “Male reproductive health and the vaccine” (100% for “no”), “Female reproductive health and coronavirus” (66,67% for “yes” and 33,33% for “no”), “Female reproductive health and the vaccine” (8,33% for “yes” and 91,67% for “no”). 

3.4.2. Revealing user judgments (Twitter)

The following topics were chosen for the study: Male reproductive health and coronavirus, Male reproductive health and the vaccine, Female reproductive health and coronavirus, Female reproductive health and the vaccine. We applied contextual analysis of tweets. The contextual analysis included determining the evidence of the effect of coronavirus (COVID) on a reproductive system (organ, parameter) of the object of discussion.  Examples of results are shown in the table 3.

Table 3.   Some results of contextual analysis of tweets related to “Male and female reproductive system and Coronavirus (Vaccine)”

We divided statements about the confirmation and denial of the impact of the coronavirus/vaccine on the reproductive health of men and women. Confirmatory markers include the following: “may be messing”, “reduces”, “possible”, “impacts”, “causes”, “effect”, “would lead”, “potentially negative effect”, ‘is messing”, “alters”, “it seriously affected”, “affected”, “destroy”, “diminished”, “impaired”, etc. Negative markers include: “no information”, “does not impact”, “no effects”, “do not cause”, “no evidence”, “there was no link”, “the effect is small and temporary”, “has no impact”, “misinformation”, “affects is a lie”, etc.

We then calculated the ratio of confirmatory (“yes”) and negative (“no”) markers in tweets by topics “Male reproductive health and coronavirus” (100% for “yes”), “Male reproductive health and the vaccine” (77,27% for “yes” and 22,73% for “no”), “Female reproductive health and coronavirus”, (100% for “yes”), “Female reproductive health and the vaccine” (22,22% for “yes” and 77,78% for “no”).

4. Discussion

In scientific articles, much attention is paid to the genetic and molecular biological aspects of the impact of coronavirus and vaccines on fertility. Naturally, research and diagnostic methods are mentioned. Many categories of people were subjected to the study (women, animals, men, pregnant women, patients with coronavirus, recovered from coronavirus, etc.). Often fertility is mentioned without specifying gender.

Among the thematic categories, the category coronavirus is the most mentioned, both in terms of the number of terms extracted and the number of terms used (frequency). The category Female reproductive health is in second place in the number of terms and in third place in frequency. The category Research objects is in third place in the number of terms and in second place in frequency.

In terminological diversity in tweets, COVID and other diseases is the leader, Discussion objects are in second place, and Female reproductive health is in third place. Coronavirus is in first place, COVID vaccination is in second, and Infertility (without gender) is in third place in frequency of use of terms.

A comparative analysis of terminological diversity of the thematic categories showed that terms related to Coronavirus, Female reproductive health, and Research objects are leading in scientific papers. There are more different terms in tweets when discussing diseases/complications of COVID (COVID and other diseases), Objects, Coronavirus and different markers are used to indicate the user's attitude to the subject.

A comparison of the frequency of the most used terms by thematic categories showed that in scientific articles there are more frequent terms related to Coronavirus, Research objects, Female reproductive health. In tweets, we noted a greater number of terms used in the categories Coronavirus, COVID vaccination, Infertility (without gender).

It can be concluded that the focus of researchers and social network users are the same thematic categories related to coronavirus and fertility, such as COVID and other diseases, COVID vaccination, Male reproductive health, Female reproductive health, Infertility (without gender), Coronavirus. Only the ratio of categories in terms of variety and frequency of terms changes.

A contextual analysis of scientific articles to determine the impact of coronavirus and covid on the reproductive system of men and women made it possible to draw the following conclusions, which we a priori consider correct and truthful.

Scientific articles prove that:

1) the vaccine does not affect male reproductive health (100% negative statements);

2) the vaccine has practically no effect on women's reproductive health (91.67% of negative statements);

3) coronavirus greatly affects male reproductive health (77.59% of positive statements);

4) coronavirus greatly affects women's reproductive health (66.67% of positive statements).

A contextual analysis of tweets to determine the impact of coronavirus and covid on the reproductive system of men and women made it possible to draw the following conclusions:

1) the vaccine strongly influenced male reproductive health (77,27% positive statements);

2) the vaccine little effects on women's reproductive health (22,22% of positive statements);

3) coronavirus absolutely affects male reproductive health (100.00% of positive statements);

4) coronavirus absolutely affects women's reproductive health (100.00% of positive statements).

Scientific articles and tweets are actively discussing the impact of coronavirus on the reproductive health of men and women. However, on social media, users are convinced of the full and absolute impact of coronavirus on the reproductive health of men and women, while in scientific articles the figure 3 were 77.59 and 66.67%, respectively. Consequently, the conclusions about the impact of coronavirus on the reproductive health of men and women are exaggerated by 1.29 times for men, 1.50 times for women.

The greatest discrepancies are observed in the impact of the vaccine on the reproductive health of men and women. In scientific articles, there are 8.33% of positive statements about the impact of the vaccine on women's reproductive health, and in tweets - 22.22%, that is, 2.67 times higher. Quite disastrous is the judgment of Twitter users about the impact of the vaccine on male reproductive health. 100% of users are convinced of the presence of such an influence, while scientific articles deny such an influence.

The methodology we proposed is shown in Figure 4.

The methodology proposed includes, as an initial stage of the analysis, checking the coincidence of interests of the scientific community and users of social networks. In the case of coincidence of interests (and in our case they practically coincide), the second stage of the methodology is implemented. In the second step, by comparing judgments from scientific sources and social networks, we find out how they differ. With a strong difference, it can be concluded that false information is being disseminated on social networks.

Declarations

Ethical Approval and Consent to participate

Not applicable.

Consent for publication

Not applicable.

Availability of supporting data

https://github.com/AidaKh155/Misinformation

Competing interests

The authors declare that they have no competing interests.

Funding

The reported study was funded by RFBR and NSFC, project number 21-57-53018.

Authors' contributions

AK and OZ both contributed to the conceptional ideas of this work. AK performed all experiments and is the primary author of the manuscript. AK created the methodology, OZ created the term detection software, OZ participated in drafting and revising the manuscript, and both authors read and approved the final manuscript.

Acknowledgements

The authors are grateful to Zatzman I. for his advices in preparing the manuscript.

 Authors' information 

Department of Development of Scientific and Innovation Activities, Russian New University, 105005, Moscow, Russia

Aida Khakimova

Department of Information Systems in Economics and Management, Russian New University, 105005, Moscow, Russia

Oleg Zolotarev 

References

1. WHO. Infodemic management 101. https://openwho.org/courses/infodemic-management-101
2. Ali I. (2020). The COVID-19 Pandemic: Making Sense of Rumor and Fear. Medical anthropology, 39(5), 376–379. https://doi.org/10.1080/01459740.2020.1745481
3. Katsaros D., Stavropoulos G., Papakostas D. Proceedings of the international conference on web intelligence. IEEE; 2019. Which machine learning paradigm for fake news detection? pp. 383–387.
4. Shu K., Sliva A., Wang S., Tang J., Liu H. Fake news detection on social media: A data mining perspective. ACM SIGKDD Explorations Newsletter. 2017;19(1):22–36.
5. Molina, M. D., Sundar, S. S., Le, T., & Lee, D. (2021). “Fake News” Is Not Simply False Information: A Concept Explication and Taxonomy of Online Content. American Behavioral Scientist, 65(2), 180–212. https://doi.org/10.1177/0002764219878224
6. Conroy, N. J., Rubin, V. L., Chen, Y. (2015). Automatic deception detection: Methods for finding fake news. Proceedings of the Association for Information Science and Technology, 52, 1-4. doi:10.1002/pra2.2015.145052010082
7. Wang, W. Y. (2017). “Liar, liar pants on fire”: A new benchmark dataset for fake news detection. Retrieved from https://arxiv.org/abs/1705.00648
8. Kaliyar R.K. Proceedings of the international conference on computing communication and automation. IEEE; 2018. Fake news detection using a deep neural network; pp. 1–7.
9. Wang, W. Y. (2017). “Liar, liar pants on fire”: A new benchmark dataset for fake news detection. Retrieved from https://arxiv.org/abs/1705.00648
10. Liang Wu, Fred Morstatter, Kathleen M. Carley, and Huan Liu. 2019. Misinformation in Social Media: Definition, Manipulation, and Detection. SIGKDD Explor. Newsl. 21, 2 (December 2019), 80–90. https://doi.org/10.1145/3373464.3373475
11. Choudrie, J., Banerjee, S., Kotecha, K., Walambe, R., Karende, H., & Ameta, J. (2021). Machine learning techniques and older adults processing of online information and misinformation: A covid 19 study. Computers in human behavior, 119, 106716. https://doi.org/10.1016/j.chb.2021.106716
12. Vosoughi, S., Roy, D., & Aral, S. (2018). The spread of true and false news online. Science (New York, N.Y.), 359(6380), 1146–1151. https://doi.org/10.1126/science.aap9559
13. Barua Z. (2022). COVID-19 Misinformation on Social Media and Public’s Health Behavior: Understanding the Moderating Role of Situational Motivation and Credibility Evaluations. Human Arenas, 1–24. Advance online publication. https://doi.org/10.1007/s42087-022-00291-w
14. Zolotarev, O., Solomentsev, Y., Khakimova, A., Charnine M. Identification of Semantic Patterns in Full-text Documents Using Neural Network Methods. GraphiCon 2019. Computer Graphics and Vision. Proceedings of the 29th International Conference on Computer Graphics and Vision. Bryansk, Russia, September 23-26, 2019. http://ceur-ws.org/Vol-2485/paper64.pdf
15. Khakimova, A.Kh., Zolotarev, O.V., Berberova, M.A. Coronavirus Infection Study: Bibliometric Analysis of Publications on Covid-19 using PubMed and Dimensions Databases. Scientific Visualization, 2021, 12(5), стр. 112–129. DOI: 10.26583/sv.12.5.10.
16. Li, X., Chen, Z., Geng, J., Mei, Q., Li, H., Mao, C., & Han, M. (2022). COVID-19 and Male Reproduction: A Thorny Problem. American journal of men's health, 16(1), 15579883221074816. https://doi.org/10.1177/15579883221074816
17. Adamyan, L., Elagin, V., Vechorko, V., Stepanian, A., Dashko, A., Doroshenko, D., Aznaurova, Y., Sorokin, M., Garazha, A., & Buzdin, A. (2022). A Review of Recent Studies on the Effects of SARS-CoV-2 Infection and SARS-CoV-2 Vaccines on Male Reproductive Health. Medical science monitor : international medical journal of experimental and clinical research, 28, e935879. https://doi.org/10.12659/MSM.935879
18. Ghosh, S., Parikh, S., Nissa, M. U., Acharjee, A., Singh, A., Patwa, D., Makwana, P., Athalye, A., Barpanda, A., Laloraya, M., Srivastava, S., & Parikh, F. (2022). Semen Proteomics of COVID-19 Convalescent Men Reveals Disruption of Key Biological Pathways Relevant to Male Reproductive Function. ACS omega, 7(10), 8601–8612. https://doi.org/10.1021/acsomega.1c06551
19. Collins, A. B., Zhao, L., Zhu, Z., Givens, N. T., Bai, Q., Wakefield, M. R., & Fang, Y. (2022). Impact of COVID-19 on Male Fertility. Urology, S0090-4295(22)00009-7. Advance online publication. https://doi.org/10.1016/j.urology.2021.12.025
20. Goebel, H., Koeditz, B., Huerta, M., Kameri, E., Nestler, T., Kamphausen, T., Friemann, J., Hamdorf, M., Ohrmann, T., Koehler, P., Cornely, O. A., Montesinos-Rongen, M., Nicol, D., Schorle, H., Boor, P., Quaas, A., Pallasch, C., Heidenreich, A., & von Brandenstein, M. (2022). COVID-19 Infection Induce miR-371a-3p Upregulation Resulting in Influence on Male Fertility. Biomedicines, 10(4), 858. https://doi.org/10.3390/biomedicines10040858
21. Jiang, Q., Linn, T., Drlica, K., & Shi, L. (2022). Diabetes as a potential compounding factor in COVID-19-mediated male subfertility. Cell & bioscience, 12(1), 35. https://doi.org/10.1186/s13578-022-00766-x
22. Sengupta, P., Dutta, S., Roychoudhury, S., D'Souza, U., Govindasamy, K., & Kolesarova, A. (2022). COVID-19, Oxidative Stress and Male Reproduction: Possible Role of Antioxidants. Antioxidants (Basel, Switzerland), 11(3), 548. https://doi.org/10.3390/antiox11030548
23. Verrienti, P., Cito, G., Di Maida, F., Tellini, R., Cocci, A., Minervini, A., & Natali, A. (2022). The impact of COVID-19 on the male genital tract: a qualitative literature review of sexual transmission and fertility implications. Clinical and experimental reproductive medicine, 49(1), 9–15. https://doi.org/10.5653/cerm.2021.04511
24. Hu, B., Liu, K., Ruan, Y., Wei, X., Wu, Y., Feng, H., Deng, Z., Liu, J., & Wang, T. (2022). Evaluation of mid- and long-term impact of COVID-19 on male fertility through evaluating semen parameters. Translational andrology and urology, 11(2), 159–167. https://doi.org/10.21037/tau-21-922
25. Zhu, G., Du, S., Wang, Y., Huang, X., Hu, G., Lu, X., Li, D., Zhu, Y., Qu, D., Cai, Q., Liu, L., & Du, M. (2022). Delayed Antiviral Immune Responses in Severe Acute Respiratory Syndrome Coronavirus Infected Pregnant Mice. Frontiers in microbiology, 12, 806902. https://doi.org/10.3389/fmicb.2021.806902
26. Ziert, Y., Abou-Dakn, M., Backes, C., Banz-Jansen, C., Bock, N., Bohlmann, M., Engelbrecht, C., Gruber, T. M., Iannaccone, A., Jegen, M., Keil, C., Kyvernitakis, I., Lang, K., Lihs, A., Manz, J., Morfeld, C., Richter, M., Seliger, G., Sourouni, M., von Kaisenberg, C. S., … COVID-19-Related Obstetric and Neonatal Outcome Study (CRONOS) Network (2022). Maternal and neonatal outcomes of pregnancies with COVID-19 after medically assisted reproduction: results from the prospective COVID-19-Related Obstetrical and Neonatal Outcome Study. American journal of obstetrics and gynecology, S0002-9378(22)00300-3. Advance online publication. https://doi.org/10.1016/j.ajog.2022.04.021
27. Youngster, M., Avraham, S., Yaakov, O., Landau Rabbi, M., Gat, I., Yerushalmi, G., Sverdlove, R., Baum, M., Maman, E., Hourvitz, A., & Kedem, A. (2022). IVF under COVID-19: treatment outcomes of fresh ART cycles. Human reproduction (Oxford, England), 37(5), 947–953. https://doi.org/10.1093/humrep/deac043
28. Jerzak, M., & Szafarowska, M. (2022). Preliminary Results for Personalized Therapy in Pregnant Women with Polycystic Ovary Syndrome During the COVID-19 Pandemic. Archivum immunologiae et therapiae experimentalis, 70(1), 13. https://doi.org/10.1007/s00005-022-00650-z
29. Carp-Veliscu, A., Mehedintu, C., Frincu, F., Bratila, E., Rasu, S., Iordache, I., Bordea, A., & Braga, M. (2022). The Effects of SARS-CoV-2 Infection on Female Fertility: A Review of the Literature. International journal of environmental research and public health, 19(2), 984. https://doi.org/10.3390/ijerph19020984
30. Sun, J., Liu, Q., Zhang, X., Dun, S., & Liu, L. (2022). Mitochondrial hijacking: a potential mechanism for SARS-CoV-2 to impair female fertility. Medical hypotheses, 110778. Advance online publication. https://doi.org/10.1016/j.mehy.2022.110778
31. Safrai, M., Herzberg, S., Imbar, T., Reubinoff, B., Dior, U., & Ben-Meir, A. (2022). The BNT162b2 mRNA Covid-19 vaccine does not impair sperm parameters. Reproductive biomedicine online, 44(4), 685–688. https://doi.org/10.1016/j.rbmo.2022.01.008
32. Reschini, M., Pagliardini, L., Boeri, L., Piazzini, F., Bandini, V., Fornelli, G., Dolci, C., Cermisoni, G. C., Viganò, P., Somigliana, E., Coccia, M. E., & Papaleo, E. (2022). COVID-19 Vaccination Does Not Affect Reproductive Health Parameters in Men. Frontiers in public health, 10, 839967. https://doi.org/10.3389/fpubh.2022.839967
33. Braun, A. S., Feil, K., Reiser, E., Weiss, G., von Steuben, T., Pinggera, G. M., Köhn, F. M., & Toth, B. (2022). Corona and Reproduction, or Why the Corona Vaccination Does Not Result in Infertility. Geburtshilfe und Frauenheilkunde, 82(5), 490–500. https://doi.org/10.1055/a-1750-9284
34. Avraham, S., Kedem, A., Zur, H., Youngster, M., Yaakov, O., Yerushalmi, G. M., Gat, I., Gidoni, Y., Hochberg, A., Baum, M., Hourvitz, A., & Maman, E. (2022). Coronavirus disease 2019 vaccination and infertility treatment outcomes. Fertility and sterility, 117(6), 1291–1299. https://doi.org/10.1016/j.fertnstert.2022.02.025
35. Han, A. R., Lee, D., Kim, S. K., Choo, C. W., Park, J. C., Lee, J. R., Choi, W. J., Jun, J. H., Rhee, J. H., Kim, S. H., Korean Society for Reproductive Medicine (KSRM), Korean Society for Reproductive Immunology (KSRI), & Korean Society for Assisted Reproduction (KOSAR) (2022). Effects and safety of COVID-19 vaccination on assisted reproductive technology and pregnancy: A comprehensive review and joint statements of the KSRM, the KSRI, and the KOSAR. Clinical and experimental reproductive medicine, 49(1), 2–8. https://doi.org/10.5653/cerm.2022.05225