Piloting a novel approach to crowdsource screening for a systematic review investigating the biological impact of ionizing radiation coming from the space environment.
- Paper number
IAC-23,E1,7,4,x80311
- Author
Mr. Giorgio Lorini, ESA - European Space Agency, Germany
- Coauthor
Dr. Mona Nasser, University of Plymouth, United Kingdom
- Coauthor
Dr. Anna Fogtman, ESA, European Astronaut Centre (EAC), Germany
- Coauthor
Dr. Alexandra Bannach-Brown, Germany
- Coauthor
Ms. Olivia Drayson, University of California, United States
- Coauthor
Mr. Murray Mackay, ESA - European Space Agency, United Kingdom
- Year
2023
- Abstract
This work focuses on two systematic reviews (SRs) investigating the impact of ionizing radiation (IR) on human biology. The first review focuses on the effects of IR on the central nervous system (CNS), and the second one identifies sex differences in susceptibility to radiation (SD). The reviews involve $>$50k research studies (human, animal, and in-vitro) that are screened for relevance to the reviews. Systematic reviews of this size require a large amount of resources to deliver data in a timely and efficient way. Crowdsourcing has been used in past systematic reviews to increase the efficiency of the process, but not for systematic reviews with this level of complexity. Moreover, there is an increased interest in initiatives in which members of the public not only benefit from the outputs of scientific research, but also get involved in the scientific process itself. Crowdsourcing provides a unique opportunity to engage individuals in the scientific process of evaluating evidence. A Crowdsourcing campaign, such as our Citizen Science Project (CSP), is an extremely valuable approach to screening scientific studies. Additionally, it can complement the automated models in support of gold-standard evidence synthesis research. As part of the SRs, a machine learning algorithm was developed using screened data of a random sample of 4228 for the CNS and 3715 studies for the SD. A second sample of 977 for CNS and 948 for SD studies was used to validate the results of the machine learning process. Both the training and the test sets were screened by the scientific team, the results of which are considered the ground truth. Individuals were provided with feedback forms to evaluate their experience engaging in the CSP. The number of studies included by the experts is 86 for CNS and 84 for SD, and the rest are excluded. 304 people signed up for the pilot stage of the CSP. A review of the misclassified studies from ground truth, the CSP, and the ML and a comparison between them is described in this paper. We will introduce changes to the process before initiating the full-text screening. It is our observation that many individuals engage with science, most participants enjoyed being part of the project, and we plan to continue this initiative in the near future.
- Abstract document
- Manuscript document
IAC-23,E1,7,4,x80311.pdf (🔒 authorized access only).
To get the manuscript, please contact IAF Secretariat.