Relevance assessments, bibliometrics, and altmetrics: a quantitative study on PubMed and arXiv.

Autor/inn/en	Breuer, Timo; Schaer, Philipp; Tunger, Dirk
Titel	Relevance assessments, bibliometrics, and altmetrics: a quantitative study on PubMed and arXiv.
Quelle	In: Scientometrics, (2022) 5, S.2455-2478 PDF als Volltext Verfügbarkeit
Dokumenttyp	gedruckt; online; Zeitschriftenaufsatz
ISSN	0138-9130
DOI	10.1007/s11192-022-04319-4
Schlagwörter	Relevance assessments; Polyrepresentation; Altmetrics; Relevance theory; Information retrieval; Test collections
Abstract	Abstract Relevance is a key element for analyzing bibliometrics and information retrieval (IR). In both domains, relevance decisions are discussed theoretically and sometimes evaluated in empirical studies. IR research is often based on test collections for which explicit relevance judgments are made, while bibliometrics is based on implicit relevance signals like citations or other non-traditional quantifiers like altmetrics. While both types of relevance decisions share common concepts, it has not been empirically investigated how they relate to each other on a larger scale. In this work, we compile a new dataset that aligns IR relevance judgments with traditional bibliometric relevance signals (and altmetrics) for life sciences and physics publications. The dataset covers PubMed and arXiv articles, for which relevance judgments are taken from TREC Precision Medicine and iSearch, respectively. It is augmented with bibliometric data from the Web of Science and Altmetrics. Based on the reviewed literature, we outline a mental framework supporting the answers to our research questions. Our empirical analysis shows that bibliometric (implicit) and IR (explicit) relevance signals are correlated. Likewise, there is a high correlation between biblio- and altmetrics, especially for documents with explicit positive relevance judgments. Furthermore, our cross-domain analysis demonstrates the presence of these relations in both research fields.
Erfasst von	OLC
Update	2023/2/05