In their Discussion Paper, Franzoni and Stephan (F&S, 2023) discuss the shortcomings of existing peer review models in shaping the funding of risky science. Their discussion offers a conceptual ...framework for incorporating risk into peer review models of research proposals by leveraging the Subjective Expected Utility (SEU) approach to decouple reviewers' assessments of a project's potential value from its risk. In my Response, I build on F&S's discussion and attempt to shed light on three additional yet core considerations of risk in science: 1) how risk and reward in science are related to assessments of a project's novelty and feasibility; 2) how the sunk cost literature can help articulate why reviewers tend to perceive new research areas as riskier than continued investigation of existing lines of research; and 3) how drawing on different types of expert reviewers (i.e., based on domain and technical expertise) can result in alternative evaluation assessments to better inform resource allocation decisions. The spirit of my Response is to sharpen our understanding of risk in science and to offer insights on how future theoretical and empirical work—leveraging experiments— can test and validate the SEU approach for the purposes of funding more risky science that advances the knowledge frontier.
The evaluation and selection of novel projects lies at the heart of scientific and technological innovation, and yet there are persistent concerns about bias, such as conservatism. This paper ...investigates the role that the format of evaluation, specifically information sharing among expert evaluators, plays in generating conservative decisions. We executed two field experiments in two separate grant-funding opportunities at a leading research university, mobilizing 369 evaluators from seven universities to evaluate 97 projects, resulting in 761 proposal-evaluation pairs and more than $250,000 in awards. We exogenously varied the relative valence (positive and negative) of others’ scores and measured how exposures to higher and lower scores affect the focal evaluator’s propensity to change their initial score. We found causal evidence of a negativity bias, where evaluators
lower
their scores by more points after seeing scores more
critical
than their own rather than
raise
them after seeing more
favorable
scores. Qualitative coding of the evaluators’ justifications for score changes reveals that exposures to lower scores were associated with greater attention to uncovering weaknesses, whereas exposures to neutral or higher scores were associated with increased emphasis on nonevaluation criteria, such as confidence in one’s judgment. The greater power of negative information suggests that information sharing among expert evaluators can lead to more conservative allocation decisions that favor protecting against failure rather than maximizing success.
This paper was accepted by Alfonso Gambardella, business strategy.
Research Summary
We investigate how knowledge similarity between two individuals is systematically related to the likelihood that a serendipitous encounter results in knowledge production. We conduct ...a field experiment at a medical research symposium, where we exogenously varied opportunities for face‐to‐face encounters among 15,817 scientist‐pairs. Our data include direct observations of interaction patterns collected using sociometric badges, and detailed, longitudinal data of the scientists' postsymposium publication records over 6 years. We find that interacting scientists acquire more knowledge and coauthor 1.2 more papers when they share some overlapping interests, but cite each other's work between three and seven times less when they are from the same field. Our findings reveal both collaborative and competitive effects of knowledge similarity on knowledge production outcomes.
Managerial Summary
Managers often try to stimulate innovation by encouraging serendipitous interactions between employees, for example by using office space redesigns, conferences and similar events. Are such interventions effective? This article proposes that an effective encounter depends on the degree of common knowledge shared by the individuals. We find that scientists who attend the same conference are more likely to learn from each other and collaborate effectively when they have some common interests, but may view each other competitively when they work in the same field. Hence, when designing opportunities for face‐to‐face interactions, managers should consider knowledge similarity as a criteria for fostering more productive exchanges.
https://www.youtube.com/watch?v=c7gCUtYb0dQ
Competence development in digital technologies, analytics, and artificial intelligence is increasingly important to all types of organizations and their workforce. Universities and corporations are ...investing heavily in developing training programs, at all tenure levels, to meet the new skills needs. However, there is a risk that the new set of lucrative opportunities for employees in these tech-heavy fields will be biased against diverse demographic groups like women. Although much research has examined the experiences of women in science, technology, engineering, and mathematics (STEM) fields and occupations, less understood is the extent to which gender stereotypes influence recruiters’ perceptions and evaluations of individuals who are deciding whether to apply to STEM training programs. These behaviors are typically unobserved because they occur prior to the application interface. We address this question by investigating recruiters’ initial outreach decisions to more than 166,000 prospective students who have expressed interest in applying to a midcareer level online tech training program in business analytics. Using data on the recruiters’ communications, our results indicate that recruiters are less likely to initiate contact with female than male prospects and search for additional signals of quality from female prospects before contacting them. We also find evidence that recruiters are more likely to base initial outreach activities on prospect gender when they have higher workloads and limited attention. We conclude with a discussion of the implications of this research for our understanding of how screening and selection decisions prior to the application interface may undermine organizational efforts to achieve gender equality and diversity as well as the potential for demand-side interventions to mitigate these gender disparities.
Funding:
We gratefully acknowledge Harvard Business School and MIT Sloan School of Management for generously funding this work.
Supplemental Material:
The online appendix is available at
https://doi.org/10.1287/orsc.2022.16499
.
This paper addresses the need for theoretical advancements in understanding team processes and the impact of technology on teams. Specifically, it examines the use of digital collaboration ...technologies by organizational teams and their effect on team communication and collaboration. Using the concept of affordances as a theoretical lens, the paper explores the potential relationships between technology affordances and essential team processes. It also provides an agenda for future research on social technologies and teams as well as novel methodological approaches for better understanding the ways in which digital technologies are affecting team processes and performance in the workplace.
Competence development in digital technologies, analytics, and artificial intelligence is increasingly important to all types of organizations and their workforce. Universities and corporations are ...investing heavily in developing training programs, at all tenure levels, to meet the new skills needs. However, there is a risk that the new set of lucrative opportunities for employees in these tech-heavy fields will be biased against diverse demographic groups like women. Although much research has examined the experiences of women in science, technology, engineering, and mathematics (STEM) fields and occupations, less understood is the extent to which gender stereotypes influence recruiters’ perceptions and evaluations of individuals who are deciding whether to apply to STEM training programs. These behaviors are typically unobserved because they occur prior to the application interface. We address this question by investigating recruiters’ initial outreach decisions to more than 166,000 prospective students who have expressed interest in applying to a midcareer level online tech training program in business analytics. Using data on the recruiters’ communications, our results indicate that recruiters are less likely to initiate contact with female than male prospects and search for additional signals of quality from female prospects before contacting them. We also find evidence that recruiters are more likely to base initial outreach activities on prospect gender when they have higher workloads and limited attention. We conclude with a discussion of the implications of this research for our understanding of how screening and selection decisions prior to the application interface may undermine organizational efforts to achieve gender equality and diversity as well as the potential for demand-side interventions to mitigate these gender disparities.
Funding: We gratefully acknowledge Harvard Business School and MIT Sloan School of Management for generously funding this work.
Supplemental Material: The online appendix is available at https://doi.org/10.1287/orsc.22.16499 .
The evaluation and selection of novel projects lies at the heart of scientific and technological innovation, and yet there are persistent concerns about bias, such as conservatism. This paper ...investigates the role that the format of evaluation, specifically information sharing among expert evaluators, plays in generating conservative decisions. We executed two field experiments in two separate grant-funding opportunities at a leading research university, mobilizing 369 evaluators from seven universities to evaluate 97 projects, resulting in 761 proposal-evaluation pairs and more than $250,000 in awards. We exogenously varied the relative valence (positive and negative) of others' scores and measured how exposures to higher and lower scores affect the focal evaluator's propensity to change their initial score. We found causal evidence of a negativity bias, where evaluators lower their scores by more points after seeing scores more critical than their own rather than raise them after seeing more favorable scores. Qualitative coding of the evaluators' justifications for score changes reveals that exposures to lower scores were associated with greater attention to uncovering weaknesses, whereas exposures to neutral or higher scores were associated with increased emphasis on nonevaluation criteria, such as confidence in one's judgment. The greater power of negative information suggests that information sharing among expert evaluators can lead to more conservative allocation decisions that favor protecting against failure rather than maximizing success.
Who Would You Like to Work With? Gómez-Zará, Diego; Paras, Matthew; Twyman, Marlon ...
Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems,
05/2019
Conference Proceeding
Odprti dostop
People and organizations are increasingly using online platforms to assemble teams. In response, HCI researchers have theorized frameworks and created systems to support team assembly. However, ...little is known about how users search for and choose teammates on these platforms. We conducted a field study where 530 participants used a team formation system to assemble project teams. We describe how users' traits and social networks influence their teammate searches, teammate choices, and team composition. Our results show that (a) what users initially search for differs from what they finally choose: initially they search for experts and sociable users, but they are ultimately more likely to choose their prior social connections as their teammates; (b) users' decisions lead to non-diverse and segregated teams, where most of the expertise and social capital are concentrated in a few teams. We discuss the implications of these results for designing team formation systems than promote users' agency.
PDF
