This special issue brings together eight papers from experts of communities which often have been perceived as different once: bibliometrics, scientometrics and informetrics on the one side and ...information retrieval on the other. The idea of this special issue started at the workshop “Combining Bibliometrics and Information Retrieval” held at the 14th International Conference of Scientometrics and Informetrics, Vienna, July 14–19, 2013. Our motivation as guest editors started from the observation that main discourses in both fields are different, that communities are only partly overlapping and from the belief that a knowledge transfer would be profitable for both sides.
Science studies are persistently challenged by the elusive structures of their subject matter, be it scientific knowledge or the various collectivities of researchers engaged with its production. ...Bibliometrics has responded by developing a strong and growing structural bibliometrics, which is concerned with delineating fields and identifying thematic structures. In the course of these developments, a concern emerged and is steadily growing. Do the sets of publications, authors or institutions we identify and visualise with our methods indeed represent thematic structures? To what extent are results of topic identification exercises determined by properties of knowledge structures, and to what extent are they determined by the approaches we use? Do we produce more than artefacts? These questions triggered the collective process of comparative topic identification reported in this special issue. The introduction traces the history of bibliometric approaches to topic identification, identifies the major challenges involved in these exercises, and introduces the contributions to the special issue.
Open research data are heralded as having the potential to increase effectiveness, productivity and reproducibility in science, but little is known about the actual practices involved in data search. ...The socio-technical problem of locating data for reuse is often reduced to the technological dimension of designing data search systems. We combine a bibliometric study of the current academic discourse around data search with interviews with data seekers. In this article, we explore how adopting a contextual, socio-technical perspective can help to understand user practices and behaviour and ultimately help to improve the design of data discovery systems.
This paper describes how semantic indexing can help to generate a contextual overview of topics and visually compare clusters of articles. The method was originally developed for an innovative ...information exploration tool, called
Ariadne
, which operates on bibliographic databases with tens of millions of records (Koopman et al. in Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems. doi:
10.1145/2702613.2732781
,
2015b
). In this paper, the method behind
Ariadne
is further developed and applied to the research question of the special issue “Same data, different results”—the better understanding of topic (re-)construction by different bibliometric approaches. For the case of the
Astro
dataset of 111,616 articles in astronomy and astrophysics, a new instantiation of the interactive exploring tool,
LittleAriadne
, has been created. This paper contributes to the overall challenge to delineate and define topics in two different ways. First, we produce two clustering solutions based on vector representations of articles in a lexical space. These vectors are built on semantic indexing of entities associated with those articles. Second, we discuss how
LittleAriadne
can be used to browse through the network of topical terms, authors, journals, citations and various cluster solutions of the
Astro
dataset. More specifically, we treat the assignment of an article to the different clustering solutions as an additional element of its bibliographic record. Keeping the principle of semantic indexing on the level of such an extended list of entities of the bibliographic record,
LittleAriadne
in turn provides a visualization of the context of a specific clustering solution. It also conveys the similarity of article clusters produced by different algorithms, hence representing a complementary approach to other possible means of comparison.
Recently, the need to contribute to the evaluation of the scientific, social, and political impact of Social Sciences and Humanities (SSH) research has become a demand of policy makers and society. ...The international scientific community has made significant advances that have transformed the impact of evaluation landscape. This article reviews the existing scientific knowledge on evaluation tools and techniques that are applied to assess the scientific impact of SSH research; the changing structure of social and political impacts of SSH research is investigated based on an overarching research question: to what extent do scholars attempt to apply methods, instruments, and approaches that take into account the distinctive features of SSH? The review also includes examples of European Union (EU) projects that demonstrate these impacts. This article culminates in a discussion of the development of the assessment of different impacts and identifies limitations, and areas and topics to explore in the future.
A cross‐disciplinary examination of the user behaviors involved in seeking and evaluating data is surprisingly absent from the research data discussion. This review explores the data retrieval ...literature to identify commonalities in how users search for and evaluate observational research data in selected disciplines. Two analytical frameworks, rooted in information retrieval and science and technology studies, are used to identify key similarities in practices as a first step toward developing a model describing data retrieval.
Digital archives are the preferred means for open access to research data. They play essential roles in knowledge infrastructures—robust networks of people, artifacts, and institutions—but little is ...known about how they mediate information exchange between stakeholders. We open the “black box” of data archives by studying DANS, the Data Archiving and Networked Services institute of The Netherlands, which manages 50+ years of data from the social sciences, humanities, and other domains. Our interviews, weblogs, ethnography, and document analyses reveal that a few large contributors provide a steady flow of content, but most are academic researchers who submit data sets infrequently and often restrict access to their files. Consumers are a diverse group that overlaps minimally with contributors. Archivists devote about half their time to aiding contributors with curation processes and half to assisting consumers. Given the diversity and infrequency of usage, human assistance in curation and search remains essential. DANS' knowledge infrastructure encompasses public and private stakeholders who contribute, consume, harvest, and serve their data—many of whom did not exist at the time the DANS collections originated—reinforcing the need for continuous investment in digital data archives as their communities, technologies, and services evolve.
This special issue brings together eight papers from experts of communities which often have been perceived as different once: bibliometrics, scientometrics and informetrics on the one side and ...information retrieval on the other. The idea of this special issue started at the workshop “Combining Bibliometrics and Information Retrieval” held at the 14th International Conference of Scientometrics and Informetrics, Vienna, July 14–19, 2013. Our motivation as guest editors started from the observation that main discourses in both fields are different, that communities are only partly overlapping and from the belief that a knowledge transfer would be profitable for both sides. Hereby, we were inspired by the bibliometric analysis of the broader field of Library and Information Science done by White and McCain (1998). The discussions during the ISSI workshop in Vienna and the papers in this special issue highlighted the following features of distinction: * The audiences served by IR and scientome ...
This is the last paper in the Synthesis section of this special issue on ‘Same Data, Different Results’. We first provide a framework of how to describe and distinguish approaches to topic extraction ...from bibliographic data of scientific publications. We then compare solutions delivered by the different topic extraction approaches in this special issue, and explore where they agree and differ. This is achieved without reference to a ground truth, since we have to assume the existence of multiple, equally important, valid perspectives and want to avoid bias through the adoption of an ad-hoc yardstick. Instead, we apply different ways to quantitatively and visually compare solutions to explore their commonalities and differences and develop hypotheses about the origin of these differences. We conclude with a discussion of future work needed to develop methods for comparison and validation of topic extraction results, and express our concern about the lack of access to non-proprietary benchmark data sets to support method development in the field of scientometrics.
PDF
