Prema reformi vrednovanja znanstvenog rada u društvenim i humanističkim znanostima
napredak i uvidi iz projekta GraspOS
DOI:
https://doi.org/10.15291/libellarium.4548Ključne riječi:
društvene i humanističke znanosti, istraživačka kultura, odgovorno vrednovanje znanstvenog rada, otvorena znanostSažetak
Cilj. Ova studija usmjerena je na razvoj kriterija za vrednovanje znanstvenog rada u društvenim i humanističkim znanostima (SSH) kroz projekt GraspOS. Cilj je izmijeniti tradicionalne metode vrednovanja koje se uvelike oslanjaju na bibliometrijske pokazatelje i izraditi preporuke prilagođene SSH zajednici, s posebnim osvrtom na ograničenja trenutnih praksi koje pretežno naglašavaju kvantitativne metrike.
Pristup/metodologija/dizajn. Ova istraživanje predstavlja rezultate dobivene kroz tri radionice održane s ispitanicima iz različitih SSH područja. Radionice su omogućile vođene rasprave o vrednovanju temeljenom na vrijednostima, raznolikosti rezultata i uključivanju kvalitativnih pokazatelja u vrednovanje. Prikupljeni podaci analizirani su deduktivnom metodom, s unaprijed definiranim kodovima za analizu.
Rezultati. Analiza je utvrdila usuglašeni stav ispitanika o potrebi za izradom uključivijeg okvira za vrednovanje znanstvenog rada koji prepoznaje različite rezultate, pored tradicionalnih članaka u časopisima. Ispitanici su naglasili važnost kvalitativnih mjera, poput recenzije, i predložili korištenje narativnih životopisa kako bi se bolje utvrdili pojedinačni doprinosi.
Ograničenja istraživanja. Istraživanje je provedeno unutar specifičnog konteksta pokreta za odgovorno vrednovanje znanstvenog rada (RRA) i projekta GraspOS, koji možda ne predstavljaju u potpunosti cijeli krajolik SSH. Generalizacija rezultata i zaključaka na sve SSH discipline nije moguća.
Praktična primjena. Rezultati provedene studije oblikovani su u vidu preporuka za pripremu općih kriterija za vrednovanje znanstvenog rada kako su predložili znanstvenici s područja društvenih i humanističkih znanosti koji su sudjelovali u provedbi istraživanja.
Originalnost/Vrijednost. Ovaj rad pruža vrijedne uvide u razvoj pravednijih i učinkovitijih praksi vrednovanja u akademskoj zajednici za širu zajednicu društvenih i humanističkih znanosti.
Preuzimanja
Reference
Abramo, Giovanni. 2024. “The forced battle between peer-review and scientometric research assessment: Why the CoARA initiative is unsound.” Research evaluation, May. https://doi.org/10.1093/reseval/rvae021.
Agate, Nicky, Christopher P. Long, Bonnie Russell, Rebecca Kennison, Penelope Weber, Simone Sacchi, Jason Rhody, and Bonnie Thornton Dill. 2022. “Walking the talk: Toward a values- aligned academy.” Knowledge commons. https://hcommons.org/deposits/objects/hc:44632/datastreams/CONTENT/content.
Aksnes, D. W., Langfeldt, L., & Wouters, P. 2019. Citations, citation indicators, and research quality: an overview of basic concepts and theories. Sage Open, 9(1). https://doi.org/10.1177/2158244019829575
Archambault, Éric, and Étienne Vignola Gagné. 2004. “The use of bibliometrics in the social sciences and humanities: final report.” Science-Metrix. https://www.science-metrix.com/pdf/SM_2004_008_SSHRC_Bibliometrics_Social_Science.pdf.
Archambault, Éric, Étienne Vignola-Gagné, Grégoire Côté, Vincent Larivière, and Yves Gingrasb. 2006. “Benchmarking scientific output in the social sciences and humanities: the limits of existing databases.” Scientometrics 68 (3): 329–42. https://doi.org/10.1007/s11192-006-0115-z.
Australian Research Council. 2019. “Non-traditional research outputs (NTROs)”. State of Australian University Research. https://dataportal.arc.gov.au/era/nationalreport/2018/pages/section1/non-traditional-research-outputs-ntros/
Balboa, Luciana, Elizabeth Gadd, Janne Pölönen, Karen Stroobants, and Erzsébet Tóth-Czifra. 2024. “The role of scientometrics in the pursuit of responsible research assessment.” LSE Impact Blog. September 4, 2024. https://blogs.lse.ac.uk/impactofsocialsciences/2024/09/04/the-role-of-scientometrics-in-the-pursuit-of-responsible-research-assessment/.
Bingham, Andrea J. 2023. “From data management to actionable findings: a five-phase process of qualitative data analysis.” International journal of qualitative methods 22 (22).
Checco, Alessandro, Lorenzo Bracciale, Pierpaolo Loreti, Stephen Pinfield, and Giuseppe Bianchi. 2021. “AI-assisted peer review.” Humanities and social sciences communications 8 (1): 1–11. https://doi.org/10.1057/s41599-020-00703-8.
CoARA. 2022. “Agreement on reforming research assessment.” CoARA. July 20, 2022. https://coara.eu/app/uploads/2022/09/2022_07_19_rra_agreement_final.pdf.
Curry, Stephen, Sarah de Rijcke, Anna Hatch, Dorsamy (Gansen) Pillay, Inge van der Weijden, and James Wilsdon. 2020. “The changing role of funders in responsible research assessment: progress, obstacles and the way ahead.” Rori.figshare.com. November 18, 2020. https://doi.org/10.6084/m9.figshare.13227914.v2.
DORA. 2012. “San Francisco Declaration on Research Assessment.” https://sfdora.org/read/.
Engels, Tim C. E., Truyken L. B. Ossenblok, and Eric H. J. Spruyt. 2012. “Changing publication patterns in the social sciences and humanities, 2000–2009.” Scientometrics 93 (2): 373–90. https://doi.org/10.1007/s11192-012-0680-2.
European Commission, European Research Executive Agency, A Oancea, and S Wilson. 2024. “Report on research assessment - Publications Office of the EU.” Publications Office of the EU. 2024. https://op.europa.eu/en/publication-detail/-/publication/219aa5ea-fae2-11ee-a251-01aa75ed71a1/language-en.
Galleron, Ioana, Michael Ochsner, Jack Spaapen, and Geoffrey Williams. 2017. “Valorizing SSH research: towards a new approach to evaluate SSH research’ Value for Society.” Fteval Journal for research and technology policy evaluation 44 (September). https://doi.org/10.22163/fteval.2017.274.
Hicks, Diana. 2005. “The four literatures of social sciences.” Journal of management and social sciences 1 (1). https://ibtjbs.ilmauniversity.edu.pk/journal/jbs/1.1/1.%20The%20Four%20Literatures%20of%20Social%20Sciences.pdf.
Himanen, Laura, and Iiris Liinamaa. 2024. “GraspOS Deliverable D5.2: Pilot Findings and Progress Report.” Zenodo, August. https://doi.org/10.5281/zenodo.13629147.
Himanen, Laura, Erica Conte, Marianne Gauffriau, Tanja Strøm, Baron Wolf, and Elizabeth Gadd. 2024. “The SCOPE framework – implementing ideals of responsible research assessment.” F1000Research 12 (May): 1241–41. https://doi.org/10.12688/f1000research.140810.2.
Himanen, Laura. 2023. “GraspOS Deliverable 5.1 ‘Report on pilot setup, current practices and initial requirements.’” Zenodo, August. https://doi.org/10.5281/zenodo.11091718.
Information Commissioners Office. 2021. “Introduction to anonymisation draft anonymisation, pseudonymisation and privacy enhancing technologies guidance.” https://ico.org.uk/media/about-the-ico/consultations/2619862/anonymisation-intro-and-first-chapter.pdf.
Ioannidis, John P. A., and Zacharias Maniadis. 2023. “In defense of quantitative metrics in researcher assessments.” PLOS Biology 21 (12): e3002408–8. https://doi.org/10.1371/journal.pbio.3002408.
Joachim Schöpfel, and Otmane Azeroual. 2024. “Responsible research assessment and research information management systems.” Encyclopedia 4 (2): 915–22. https://doi.org/10.3390/encyclopedia4020059.
Krüegel, Sybil. 2019. The informed consent as legal and ethical basis of research data production. FORS Guide No. 05, Version 1.0. Lausanne: Swiss Centre of Expertise in the Social Sciences FORS. doi:10.24449/FG-2019-00005
Leeuwen, van. 2013. “Bibliometric research evaluations, Web of Science and the social sciences and humanities: a problematic relationship?” Bibliometrie Scientometrics 2 (September): 1–18. https://doi.org/10.5283/bpf.173.
Mystakopoulos, Fotis. 2024. “The role of open science in fostering a more inclusive research assessment process in social sciences and humanities.” Zenodo. https://doi.org/10.5281/zenodo.10562683.
National Research Council. 2012. Improving measures of science, technology, and innovation. National Academies Press EBooks. https://doi.org/10.17226/13358.
Ochsner, Michael, Sven Hug, and Ioana Galleron. 2017. “The future of research assessment in the humanities: bottom-up assessment procedures.” Palgrave Communications 3 (1). https://doi.org/10.1057/palcomms.2017.20.
Pinto da Costa, Mariana. 2021. “Conducting cross-cultural, multi-lingual and multi-country focus groups: guidance for researchers.” International Journal of Qualitative methods 20 (January): 160940692110499. https://doi.org/10.1177/16094069211049929.
Rafols, Ismael. 2024. “The UNESCO Open Science Outlook: OS progresses, but unequally.” Leidenmadtrics.nl. February 2024. https://www.leidenmadtrics.nl/articles/the-unesco-open-science-outlook-there-is-progress-in-os-but-it-is-unequal.
Reale, Emanuela, Dragana Avramov, Kubra Canhial, Claire Donovan, Ramon Flecha, Poul Holm, Charles Larkin, et al. 2017. “A review of literature on evaluating the scientific, social and political impact of social sciences and humanities research.” Research Evaluation 27 (4): 298–308. https://doi.org/10.1093/reseval/rvx025.
Rijcke, Sarah de, Clemencia Cosentino, Robin Crewe , Carlo D’Ippoliti, Shaheen Motala-Timol , Noorsaadah Binti A Rahman , Laura Rovelli , David Vaux, and Yao Yupeng . 2023. “The future of research evaluation: a synthesis of current debates and developments,” May. https://doi.org/10.24948/2023.06.
Rodriguez, Katrina L., Jana L. Schwartz, Maria K. E. Lahman, and Monica R. Geist. 2011. “Culturally responsive focus groups: reframing the research experience to focus on participants.” International journal of qualitative methods 10 (4): 400–417. https://doi.org/10.1177/160940691101000407.
Ross-Hellauer, Tony. 2017. “What is open peer review? A systematic review.” F1000Research 6 (588): 588. https://doi.org/10.12688/f1000research.11369.2.
Schmoch, Ulrich, Torben Schubert, Dorothea Jansen, Richard Heidler, and Regina von Görtz. 2010. “How to use indicators to measure scientific performance: a balanced approach.” Research evaluation 19 (1): 2–18. https://doi.org/10.3152/095820210x492477.
Schönbrodt, Felix D, Anne Gärtner, Maximilian Frank, Mario Gollwitzer, Malika Ihle, Dorothee Mischkowski, Le Vy Phan, et al. 2022. “Responsible research assessment I: implementing DORA for hiring and promotion in psychology.” psyArXiv (OSF Preprints), November. https://doi.org/10.31234/osf.io/rgh5b.
Sharma, Mohit, Anurag Sarin, Priyanka Gupta, Shobhit Sachdeva, and AnkurV Desai. 2014. “Journal impact factor: its use, significance and limitations.” World journal of nuclear medicine 13 (2): 146. https://doi.org/10.4103/1450-1147.139151.
Sivertsen, Gunnar. 2016. “Patterns of internationalization and criteria for research assessment in the social sciences and humanities.” Scientometrics 107 (2): 357–68. https://doi.org/10.1007/s11192-016-1845-1.
Snijder, Ronald, 2019. "The deliverance of open access books: examining usage and dissemination". https://doi.org/10.26530/oapen_1004809
Sorbonne University. 2024. “Sorbonne University unsubscribes from the Web of Science.” Sorbonne Université. 2024. https://www.sorbonne-universite.fr/en/news/sorbonne-university-unsubscribes-web-science.
Stam, Alexandra, and Pablo Diaz. 2023. “Qualitative data anonymisation: theoretical and practical considerations for anonymising interview transcripts.” FORS Guide No. 20 Version 1.0 (March). https://forscenter.ch/wp-content/uploads/2023/03/qualitative-data-anonymisation_final.pdf.
Tatum, Clifford, Zeynep Anli, Ludo Waltman, Anna-Kaisa Hyrkkänen, Janne Pölönen, and Josephine Nordling. 2023. “GraspOS Deliverable 2.2: Open Science Assessment Framework (OSAF).” Zenodo, December. https://doi.org/10.5281/zenodo.11091512.
UNESCO. 2023. “Open Science Outlook 1: status and trends around the world.” UNESDOC Digital Library. https://doi.org/10.54677/giic6829.
Vergoulis, Thanasis, Serafeim Chatzopoulos, Giulia Malaguarnera, Zenia Xenou, Leonidas Pispiringas, Harry Dimitropoulos, Silvio Peroni, et al. 2024. “GraspOS Deliverable D3.2: Tools and Services.” Zenodo, August. https://doi.org/10.5281/zenodo.13619176.
Vergoulis, Thanasis, Serafeim Chatzopoulos, Kleanthis Vichos, Ilias Kanellos, Andrea Mannocci, Natalia Manola, and Paolo Manghi. 2022. “BIP! SCHOLAR: a service to facilitate fair researcher assessment.” In JCDL ’22: Proceedings of the 22nd ACM/IEEE Joint Conference on Digital Libraries. https://doi.org/10.1145/3529372.3533296.
Wang, Ruishu. 2023. “Research on problems of scientific research performance evaluation mechanism of humanities and social sciences.” Frontiers in Educational Research 6 (18). https://doi.org/10.25236/fer.2023.061823.
White, Howard D., Sebastian K. Boell, Hairong Yu, Mari Davis, Concepción S. Wilson, and Fletcher T.H. Cole. 2009. “Libcitations: a measure for comparative assessment of book publications in the humanities and social sciences.” Journal of the American Society for Information Science and Technology 60 (6): 1083–96. https://doi.org/10.1002/asi.21045.
Preuzimanja
Objavljeno
Broj časopisa
Rubrika
Licenca
Autorska prava (c) 2025 Fotis Mystakopoulos, Carol Delmazo
Ovo djelo je licencirano pod licencom Creative Commons Attribution 4.0 Međunarodna licenca.
