独自更快还是结伴更远：研究者应对社会危机的速度与影响力

Faster alone vs. further together: The speed and impact of researchers’ responses to societal crises

Article Unpublished manuscript
 
Faster alone vs. further together: The speed and impact of researchers’ responses to societal crises

Nicholas A. Coles^1,∗, Joao Francisco Goes Braga Takayanagi¹, Stephen M. Fiore², Lingfei Wu³

¹Department of Psychology, University of Florida, Gainesville, USA. ²Institute of Simulation and Training, University of Central Florida, Orlando, USA. ³School of Computing and Information, University of Pittsburgh, Pittsburgh, USA. ^∗Corresponding author: ncoles@ufl.edu

Throughout the 21st century, society encountered numerous crises that demanded quick and widespread action. In the present work, we documented researchers’ responses to 25 of these crises, analyzing 1,537,284 published responses from 3,262,152 researchers spanning 225 countries and territories. An analysis of publication speed and citation patterns yielded support for a proverbial claim: “to go fast, go alone; to go far, go together”. Solo authors were quicker to publish responses to societal crises – but their responses received less attention in scholarly, news, and policy venues. Growth towards large-scale collaboration was associated with increased impact; However, this team size advantage was characterized by diminishing returns and significantly varied across sampled crises. Furthermore, pluralistic machine learning modeling highlights the importance of considering additional context, such as government effectiveness and the expertise of team leaders. Taken together, results facilitate evidence-based conversations regarding how researchers optimize speed and impact in the midst of societal crises.

Throughout history, human societies have encountered and navigated crises that demanded quick, urgent, and widespread action. For example, in the first quarter of the 21st century, society witnessed some of the deadliest terrorist attacks in modern history, a global pandemic, and the rapid release of AI technologies. For decades, researchers have worked to document how people respond to these societal crises, helping separate proverbial myths from reality. For example, contrary to common belief, research suggests that people tend to respond to crises not with panic – but with order, solidarity, and cooperation^1,2.

Curiously, the actions of researchers themselves are often overlooked in crisis response research – leaving an incomplete narrative. For example, to understand society’s response to the September 11 terrorism attacks, it is important to consider that a leading academic society endorsed psychologists’ participation in detainee interrogation³. To understand society’s response to the COVID-19 pandemic⁴, it is important to consider the record-setting speed in which researchers developed mRNA vaccines⁵. Last, to understand how society responds to the shock of AI technologies, we argue it will once again be important to consider the role of researchers – such as the ultra-massive teams that developed GPT-4⁶, DeepSeek V3⁷, and other technologies at the center of an ongoing “AI race”⁸. In the present work, we document researchers’ responses to 25 of these societal crises, analyzing a total of 1,537,284 published responses from 3,262,152 researchers spanning 225 countries and territories (Figure 1).

Proverbial claims about collaboration suggests “to go fast, go alone; to go far, go together”. This belief is also often echoed in the scientific community. Proponents of team science often suggest that collaboration is a way to “go further,” as it (a) facilitates role specialization, (b) increases total effort dedicated to a problem, and (c) creates conditions believed to facilitate recombinant growth, wisdom of the crowd, and/or collective intelligence^9–15. These benefits, however, are often suggested to come at a cost that may be particularly noteworthy during crisis: speed. Indeed, proponents of team science also often emphasize the time needed to (a) assemble teams, (b) negotiate responsibilities, (c) navigate different institutional policies, (d) manage disagreements, and so on^16–19. Working alone is presumed to be quicker, with simpler coordination, fewer bureaucratic constraints, and greater decision-making agility.

In the present work, we test proverbial claims at scale, examining 1,537,284 separate published responses from a total of 3,262,152 researchers. As methodological tests of speed, we borrowed from the sociological notion of focusing events²⁰. More specifically, we sampled societal crises that (a) entered mainstream awareness relatively suddenly, and (b) received relatively novel names (e.g., “Hurricane Katrina”; Figure 2, Panel A). This allowed us to estimate response speed by comparing (a) the approximate date that most people became aware of each crisis, to (b) the publication dates of papers explicitly mentioning relevant keywords (retrieved via OpenAlex²¹). To index the impact of researchers’ published responses, we considered the number of citations their work received in scholarly, news, and policy venues (tracked by OpenAlex²¹ and Altmetric²²). Last, to establish comparative benchmarks, we compared team size to 12 other predictors often considered at the individual (e.g., leader prestige), group (e.g., number of involved institutions), and societal level (e.g., political stability). Here, we leveraged a pluralistic set of machine learning models to derive robust estimates of the extent to which each variable predicts the speed and impact of researchers’ responses to crises²³.

Expanding upon insights from seminal field studies^1,2, the present work provides a relatively comprehensive assessment of researchers’ published responses to an extreme phenomenon: societal crises. Theoretically, this tests potentially extreme boundary conditions of a robust finding in research on team science: work from larger teams often achieves more impact (e.g., in scholarly articles, patents, and policy venues), often at the cost of other potentially desirable outcomes (e.g., innovativeness)^24–29. By sampling a relatively large number of events, we further estimate the reliability of such relationships across a wide variety of crises (e.g., the H1N1 pandemic, Brexit, and the announcement of an impending “metaverse”). Taken together, our work not only lends itself to a more complete narrative of how society responds to crisis, but also to evidence-based conversations regarding how to optimize the speed and impact of researchers’ responses to these crises.

Results

Similar to what was documented decades ago with the general population, our results illustrate that researchers also often respond to crisis with order, solidarity, and cooperation^1,2. In total, we uncovered 1,537,284 published responses to 25 societal crises – representing the collective response of 3,262,152 researchers from 225 countries and territories (Figure 1).

In many instances, we found that researchers do “go alone”. Averaged across crises, 49.1% of published responses were attributed to a single author (SD = 22.0% across crises). The other 50.9% of the time, researchers “go together,” with teams ranging from 2 to 100+ co-authors. Next, we describe the implications this decision has for the speed and impact of their published responses.

When responding to crises, solo authorship is faster but less impactful

Proverbial claims about collaboration suggests “to go fast, go alone; to go far, go together”. Consistent with such notions, researchers were significantly quicker to publish responses as solo authors vs. teams (Δ = 100 days; Mann-Whitney W = 1.88 × 10¹¹, p < .001, rank-biserial r = .18). Intriguingly, however, the link between the speed and size of teams was best characterized as weakly quadratic – as opposed to linear (ΔBIC = 4195) or logarithmic (ΔBIC = 1135; Figure 2, Panel B). More specifically, speed most sharply decreased as researchers moved from (a) solo to small team efforts (Δ = 96 days; Mann-Whitney W = 1.14 × 10¹¹, p < .001, rank-biserial r = .18), vs. (b) small to medium team efforts (Δ = 12 days; Mann-Whitney W = 1.09 × 10¹¹, p < .001, rank-biserial r = .02). Speed then slightly recovered as researchers moved towards their largest team efforts (see Table S1 for additional descriptives).

Despite reductions in speed, team science was more impactful, in terms of tracked mentions in scholarly, news, and policy venues (Figure 2, Panel B). For example, consider a crisis-relevant paper published by a solo author who “went alone”. On average, these solo-authored papers received 6.08 mentions in scholarly articles (SD = 53.58), 0.34 mentions in the news (SD = 6.45), and 0.03 mentions in policy documents (SD = 0.47). Comparatively, papers published by small teams of 2 - 4 authors received (1) 149% more mentions in scholarly articles (M = 15.11, SD = 77.00; rank-biserial r = .29), (2) 57% more mentions in the news (M = 0.53, SD = 9.75; rank-biserial r = .02), and (3) 76% more mentions in policy documents (M = 0.06, SD = 0.77; rank-biserial r = .01; all Mann-Whitney p < .001). As an even more extreme example, papers published by medium-sized teams of 5 - 9 authors received (1) 311% more mentions in scholarly articles (M = 24.95, SD = 128.07; rank-biserial r = .49), (2) 275% more mentions in the news (M = 1.27, SD = 16.52; rank-biserial r = .07), and (3) 181% more mentions in policy documents (M = 0.10, SD = 0.94; rank-biserial r = .02; all Mann-Whitney p < .001).

Movement towards large-scale collaboration yields diminishing returns in impact

Past field studies have documented a sociological tendency for people to mass organize in response to crisis¹. Our results illustrate a similar dynamic in science, not only in terms of the number of teams but also their size. Of 1,537,284 total published responses to crises, we observed 122,353 publications (8.0%) with ten to over a hundred authors.

Observing a large number and range of big teams helps us better understand their reliability and limits. Here, we once again replicate patterns from classic field studies on crisis response: collaboration growth yields diminishing returns³⁰. More specifically, links between the size and measured impact of papers were best characterized as logarithmic, as opposed to linear (ΔBIC ≥ 6,639; Wilcoxon signed-rank p < .001) or quadratic (ΔBIC ≥ 5,203; Wilcoxon signed-rank p < .001). In other words, growth in team size was associated with diminishing returns in measured impact. Notably, however, the rate of diminishing returns significantly varied across crises; Including crisis-specific random slopes consistently improved model fit (all ΔBIC ≥ 8,063; likelihood-ratio tests, all χ²(1) ≥ 8,091, p < .001). These results provide initial evidence of the importance of context: teams of similar size respond more impactfully to some crises than others.

Size in context: Integrating predictors of quick and impactful crisis responses in science

Not surprisingly, many claims about the role of collaboration – both proverbial and scientific – fail to consider size in context. For example, perhaps it is difficult for people to “go fast” in the face of political dysfunction or “go far” with relatively inexperienced leaders. To illustrate the utility of considering team size alongside 12 other predictors measured at the individual, group, and societal level, we deployed a pluralistic set of machine learning architectures (e.g., XGBoost, neural nets, random forests). When trained to consider the full set of variables – vs. team size alone – models were significantly more accurate at predicting response speed (ΔR² = .38) and mentions in scholarly (ΔR² = .39), news (ΔR² = .09), and policy venues (ΔR² = .07) in a hold-out test dataset (all Wilcoxon signed-rank p < .001).

Considering team size along a dozen other variables also allows us to compare how useful they are for predicting speed and impact. For example, despite attracting significant attention from lay people and researchers alike, permutation variable importance scoring indicates that team size is not the strongest sampled predictor of speed and impact. Team size was the (a) 11th strongest sampled predictor of response speed (model reliance M = 0.16, SD = 0.10), (b) 3rd strongest predictor of scholarly citations (M = 0.15, SD = 0.05), (c) 2nd strongest predictor of news mentions (M = 0.75, SD = 0.20), and (d) 2nd strongest predictor of policy citations (M = 0.61, SD = 0.28; Figure 3). For response speed, the strongest sampled predictors were expert-derived ratings of the corresponding author’s government – most notably its effectiveness, voice and accountability, and control of corruption (see Figure 3). For mentions in scholarly, news, and policy venues, the strongest sampled predictor was the h-index of the corresponding author.

Discussion

Arguably every human society has had to navigate crises, whether they be natural, technological, social, or of some other nature. Across generations, these cumulative experiences have helped inspire folk beliefs about how humans respond to crises — some more accurate than others. For instance, past research helped debunk beliefs that people respond to crises with panic^1,2. Instead, it seems that people respond with a surprising amount of order, solidarity, and cooperation. Our results strengthen such assertions in the sense that we show ≈1.6 million instances where researchers organized and published responses to societal crises, approximately half of which involved a team of co-authors.

More centrally, we evaluated a popular but largely untested folk belief about collaboration: “to go fast, go alone; to go far, go together”. Here, we found more support for conventional wisdom. When considering publication dates and mentions across academic, news, and policy venues, results suggest that researchers do indeed publish faster alone and more impactfully together. However, results also point to oft-neglected nuance: larger teams yield diminishing returns in impact. Furthermore, context matters: team sizes that were impact-maximizing in some crises (e.g., COVID-19) were less successful in others (e.g., responses to September 11 terrorism attacks).

To further highlight the role of context, we considered the predictive validity of 12 additional candidate variables. Results help illustrate that a combination of individual-level, group-level, and society-level context is needed to understand effective crises responses²⁴. For example, the prestige of the corresponding author and the number of co-authors were consistently strong predictors of measured impact, as was the number of papers a team ultimately cites. For speed, conversely, the strongest predictors were expert-derived ratings of facets of government effectiveness. In other words, while it is generally the case that researchers publish faster alone and more impactfully together, maximizing the speed and impact of researchers’ responses to crises will require considering context from multiple angles.

In terms of limitations, we navigated conceptual challenges regarding the operational definition of a crisis, a crisis response, and the impact of those responses. We similarly wrestled with mechanistic uncertainty — e.g., uncertainty regarding whether large teams produce vs. flock to better ideas. Below, we elaborate on these challenges and their implications for research on crises.

First, we encountered considerable differences in how societal crises are defined. Scholars across different research traditions have emphasized different characteristics, such as suddenness, urgency, social disruption, and public perception^31–33. We did not seek to resolve these long-standing debates. Instead, we sought a working definition that permitted relatively clean tests of response speed. Ultimately, we borrowed most heavily from the notion of sudden, attention-grabbing focusing events²⁰. Due to their suddenness, these events allowed us to operationalize response speed in a literal fashion: the number of days that passed since people typically became aware of the development. However, in doing so, we excluded a variety of societal crises, such as those that (a) developed relatively gradually (e.g., climate change), (b) attracted less attention (e.g., sand supply scarcity), and/or (c) have not received historically unique names (e.g., recent waves of political polarization). Crises such as these make it difficult to estimate response speed, but nonetheless represent critical threats to modern day societies. There is an urgent need to understand such developments more thoroughly (e.g., via future analyses of self-reported timelines, research repository commitment history, and/or grant activity reports).

We similarly faced difficulty defining what constitutes a crisis response in science, as well as how to measure its impact. Here, our operationalizations were largely shaped by data availability. We sought large quantities of data because the phenomena we seek to understand are rare; Societal crises represent an extreme disruption of norms, as do the mass collaborations they often inspire. In the context of science, we found the largest swaths of data in bibliometric repositories of published scholarly papers, the researchers who authored them, and the extent to which the work was subsequently discussed across multiple communities. However, a drawback of this approach is that it misses other ways that researchers respond to crises (e.g., via community service) and seek to improve the world (e.g., increasing societal well-being vs. increasing citation counts). There are obvious challenges to documenting those types of responses and quantifying those broader forms of impact. Nonetheless, doing so is essential in order to develop a comprehensive understanding of how researchers contribute to responses to societal crises.

A third set of limitations involves mechanistic uncertainty. Put simply, we do not know why we observe correlations between the size, speed, and impact of crisis responses in science. It is possible there is a simple causal story, wherein simply increasing the size of one’s team leads to higher-quality insights. However, an equally plausible story is that the most promising ideas better attract sustained attention from larger groups of researchers. Similarly, it’s possible that large teams have stronger social capital instead of stronger ideas — elevating the impact of otherwise non-superior research. Evaluating the mechanisms underlying such links is a oft-discussed goal in research on teams^{27–29,34,35}, a goal we encourage future researchers to address via methods that better facilitate causal inference (e.g., field experiments, naturalistic experiments, etc.).

Until these limitations are better addressed, we caution against a de-prioritization of solo responses or a fetishization of large-scale responses. Effective crisis response likely requires a balance of solo and large team effort from researchers, a mixture of traditionally and non-traditionally tracked activities, and an eye towards outcomes rarely measured or discussed in hiring and promotion.

To conclude, human societies have always had to navigate periods of calm and crises — a continuously repeating cycle. In the present work, we take stock of the role of researchers in the aftermath of societal crises, weighing in on a proverbial claim about collaboration: “to go fast, go alone; to go far, go together”. Across 1,537,284 published responses from 3,262,152 researchers spanning 225 countries and territories, we find strong support for this proverbial claim. Researchers do indeed publish faster alone and more impactfully together. However, growth towards large-scale collaboration yields diminishing returns that vary across crises. Furthermore, a pluralistic set of machine learning models converge on the importance of considering additional context, such as government effectiveness or the expertise of team leaders. In the future, we emphatically encourage stakeholders to consider these insights – as well as the potential interconnectedness of different crisis responses, different types of activity and impact, and the more conceptually challenging crises we did not sample in the present work. After all, if society encounters crises we cannot ultimately overcome, there will arguably be no institution of science left to document and understand such developments.

Materials and methods

Ethics

To examine researchers’ published responses to crisis events, we interoperated secondary data on (1) research articles, authors, and institutions (via OpenAlex²¹), (2) tracked mentions in scholarly, news, and policy documents (via Altmetric Explorer²²), and (3) country-level governance indicators (via WorldBank’s Worldwide Governance Indicators³⁶). Because the analysis focused on secondary data, the University of Florida Institutional Review Board declared this work exempt (Protocol #ET00042723). Anthropic Claude (3.7 Sonnet - Fable 5) was used to generate focusing event suggestions, as well as to review all analysis code for accuracy, accessibility, and reproducibility issues.

Sampling crisis events

The sampling plan was designed to identify crises that provide relatively clean methodological tests of response speed. To do so, we borrowed from the sociological notion of sudden, attention-grabbing, focusing events²⁰. More specifically, we identified societal crises that (a) entered mainstream awareness relatively suddenly, and (b) received relatively novel names. This allowed us to estimate response speed by simply comparing (a) the approximate date that each crisis entered mainstream awareness, to (b) the publication dates of papers explicitly mentioning relevant keywords (retrieved via OpenAlex²¹). For example, there are virtually no mentions of the phrase “ChatGPT” in the research literature before 2022. However, in the three-year aftermath of its release, researchers have mentioned it in 111,646 research publications (Figure 2, Panel A).

To identify candidate focusing events, we considered (a) events frequently discussed in crisis response research (e.g., the Deepwater Horizon oil spill), (b) recommendations from foundational large language models (e.g. Claude 3.7 Sonnet), and (c) our own lived experiences. From this list, we conducted initial bibliometric searches to ensure that the crisis had relatively unique identifying keywords and large growth around the approximate date of the focusing event. From this list, we chose 25, seeking a variety of different societal crises (Figure 1), including ones often characterized as natural (e.g., Hurricane Katrina), sociopolitical (e.g., #MeToo), technological (e.g., ChatGPT), and economic realms (e.g., the 2008 Global Financial Crisis). This was done to promote more generalizable conclusions about links between the size, speed, and impact of researchers’ responses to crises.

Identifying published responses

Researchers respond to societal crises in many ways, including advising policymakers, providing media commentary, and engaging the public directly. Here, we focus on one of their most thoroughly tracked contributions: publications. For each sampled crisis, we first identified keywords or phrases that researchers used to explicitly signal relevance. Second, we specified the approximate date that each sampled crisis entered mainstream awareness. Third, we used OpenAlex to pull papers containing the relevant keywords in their title, abstract, or (if available) main text.

As an example of this workflow, consider the COVID-19 pandemic. This crisis entered mainstream awareness near late December 2019, when initial reports of the virus emerged from Wuhan, China. We sampled papers that made an explicit connection to this crisis by using keywords like “COVID-19”, “SARS-CoV-2”, and “novel coronavirus”. See Table S3 for full list of keyword strings.

In total, we identified 1,610,388 papers containing keywords linked to our 25 sampled crises within the three years before and after each event. Consistent with the notion of focusing events, only 2.9% of this keyword use occurred in the 3 years preceding the approximate date that each crisis entered mainstream awareness (Figure 2, Panel A). Our analysis focuses on the 97.1% of papers published in the 3 years following these events. After removing non-articles and records without tracked authorship meta-data, this produced a final sample of 1,537,284 papers.

Operationalizing response speed and impact

For each paper, we operationalized response speed as the number of days between (a) their publication date and (b) the approximate date each crisis entered mainstream awareness. For impact, we considered multiple indicators: the number of times each paper was mentioned in scholarly works (N = 27,898,094 mentions examined), the news (N = 1,462,597 mentions), and policy documents (N = 115,423 mentions). Scholarly citation counts were pulled from OpenAlex article meta-data²¹. News and policy document citation counts were pulled from Altmetric Explorer²². Because a Digital Object Identifier is required to match a paper to an Altmetric Explorer record, analyses of response speed and scholarly citations draw on all 1,537,284 papers, whereas analyses of news and policy mentions draw on the 1,370,810 papers carrying one (see Table S1 and Table S2 for the sample sizes behind each estimate). To reduce sensitivity to outliers, speed and impact indices were converted to percentiles when fitting regression models. Because crises vary in the overall amount of attention they receive, percentiles were calculated separately for each event (i.e., standardized). (See Figure S1 for similar results without this standardization.) For ease of interpretation, percentiles are always reported in a positive direction, wherein higher values indicate faster and more impactful publication.

Predictors of quick and impactful crisis responses in science

The primary analyses focused on evaluating proverbial claims about links between the size, speed, and impact of researchers’ published responses to crises. Team size was extracted from OpenAlex meta-data. For most analyses, team size was treated as a continuous variable. However, for ease-of-interpretation, we also report common (but arbitrary^28,37) categorizations: papers published by a (a) solo author, (b) small teams (2 - 4 authors), (c) medium teams (5 - 9 authors), and (d) big teams (10+ authors).

The secondary analyses focused on considering team size alongside three other sets of predictors. The first captures other characteristics of teams and their work, such as the unique number of institutions and countries involved, the primary discipline, and the volume of literature cited. The second set of predictors focused on the person who leads the work – more specifically the corresponding author’s past academic success (h-index) and type of home institution (education, healthcare, government, company, non-profit, archival). These predictors were all extracted from OpenAlex meta-data²¹.

The third set of predictors considered the corresponding author’s broader societal context. More specifically, we connected their affiliation data to the World Bank’s Worldwide Governance Indicators, matched by year of publication (up to last recorded year: 2022) and country³⁶. These indicators contain expert ratings of six dimensions, with higher values indicating normatively better governance: (1) voice and accountability (e.g., freedom of expression), (2) political stability (e.g., absence of violent destabilization), (3) government effectiveness (e.g., quality of public services), (4) regulatory quality (e.g., ability to promote private sector development), (5) rule of law (e.g., adherence to social rules), and (6) control of corruption (e.g., non-privatization of public power). Because the World Bank series is biennial before 2002, intervening years were filled by within-country linear interpolation.

Analytic strategy

Primary analysis

The primary analysis focused on evaluating the extent to which links between the size and effectiveness of crisis responses are linear, logarithmic, or quadratic. Conceptually, this represents constant, diminishing, and curvilinear returns, respectively. For example, rather than leading to constant linear gains, it is possible that increases in team size lead to diminishing returns. This would suggest that researchers navigate a plateau, wherein they must make increasingly larger investments to increase the speed and impact of their crisis responses. Alternatively, increases in team size may exhibit a curvilinear “Goldilocks zone”: increases in team size may lead to improved performance up to a point, after which further growth becomes detrimental. Such relationships are captured via linear (β1​Xi), logarithmic (β1​l​n​(Xi)), and quadratic regression (β1​Xi+β2​Xi2), respectively.

Linear, logarithmic, and quadratic relationships between team size and each outcome were modeled separately using the lme4 mixed-effect modeling package³⁸ (v. 2.0.1) in R (v. 4.5.3). To accommodate the fact that papers published on similar crises are non-independent, we included random intercepts for each sampled crisis: β0+u0,j, wherein u0,j​∼​N​(0,𝜎02). To ensure that relatively uncommon large team efforts (e.g., 20 co-authors) received the same priority as relatively common small team efforts (e.g., 2 co-authors) during model fitting, we applied inverse frequency weights. (See Figure S1 for similar patterns from raw data plots.) When comparing fit, we favored a model (e.g., logarithmic over linear) when two criteria were both met: (a) it reduced the Bayesian Information Criterion by more than 5 (ΔBIC > 5), and (b) a one-sided paired Wilcoxon signed-rank test indicated that its distribution of weighted absolute residuals was significantly lower (using p < .05 threshold). Mann-Whitney comparisons of team-size categories were two-sided. No adjustments were made for multiple comparisons. Given the unusually large size of our sample, we focus more on interpreting patterns than statistical significance.

To evaluate the extent to which the best-fitting relationship varied across sampled crises, random slopes were added. For example, when a logarithmic model best fit the data, random slopes would be included for the logarithmic term: (β1+u1,j)​l​n​(Xi​j), wherein u1,j​∼​N​(0,𝜎12). The present work mostly focuses on describing the results of these models. Nonetheless, for convenience, we also report comparisons of team size categories; e.g., papers published by small (2 - 4 authors) vs. medium teams (5 - 9 authors). In these scenarios, we used Mann-Whitney tests to compare group distributions (using p < .05 threshold).

Secondary analysis

A complete understanding of how researchers respond to crises may ultimately require considering team size alongside other variables that may confound or moderate its effects. To do so, we fit a pluralistic set of machine learning algorithms containing a fuller set of potentially relevant predictors. (For more information on predictor selection, see Predictors of quick and impactful crisis responses in science.) To evaluate the utility of adding these additional predictors, we compared models with vs. without their inclusion. Because different modeling approaches can produce meaningful variation in researchers’ conclusions³⁹, we used AutoGluon 1.5.0⁴⁰ to train a variety of different models for each outcome: gradient-boosted decision trees (LightGBM, CatBoost, and XGBoost), randomized tree ensembles (random forests and extremely randomized trees), and neural networks. When fitting models, we sought authoritative inferences by using best-quality AutoGluon presets, with dynamic stacking disabled and no additional stacking levels. Each model was given up to 60 hours to train.

Prior to model training, we divided papers by crisis and then randomly assigned 80% into a training/validation set. The remaining 20% was held out as test data. Crucially, dividing papers by crisis before splitting data helped ensure that their statistical patterns were considered during both training and testing.

To estimate the importance of each predictor, we used a model-agnostic permutation method²³. This methodology estimates the importance of each predictor by randomly shuffling its values and then re-testing the accuracy of each algorithm. When the algorithm relies on the predictor, randomly shuffling its values diminishes performance. We applied this permutation-based method to each predictor of each top-fitting base model for each of the 4 outcomes. Importance was estimated with five random shuffles per predictor on a random subsample of 20,000 hold-out papers. To facilitate comparability across outcomes and models, variable importance values were rescaled to a [0, 1] range within each base model before being plotted in Figure 3.

References

References

• 1

  1.

• 2

  Fritz, C. E. & Williams, H. B. The human being in disasters: A research perspective. The ANNALS of the American Academy of Political and Social Science 309, 42–51 (1957).

• 2.

  Quarantelli, E. L. & Dynes, R. R. Response to social crisis and disaster. Annual Review of Sociology 3, 23–49 (1977).

• 3.

  Report of the american psychological association presidential task force on psychological ethics and national security. (2005) doi:10.1037/e661622007-001.

• 4.

  Hill, R. et al. The pivot penalty in research. Nature 642, 999–1006 (2025).

• 5.

  Baden, L. R. et al. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. New England Journal of Medicine 384, 403–416 (2021).

• 6.

  OpenAI et al. GPT-4 technical report. (2023) doi:10.48550/ARXIV.2303.08774.

• 7.

  DeepSeek-AI et al. DeepSeek-V3 technical report. (2024) doi:10.48550/ARXIV.2412.19437.

• 8.

  Grant, N. & Weise, K. In A.I. Race, Microsoft and Google choose speed over caution. The New York Times https://www.nytimes.com/2023/04/07/technology/ai-chatbots-google-microsoft.html (2023).

• 9.

  Rasti, S., Vaesen, K. & Lakens, D. A systemic approach to better coordination in science. Nature Human Behaviour 9, 1762–1764 (2025).

• 10.

  Coles, N. A., Hamlin, J. K., Sullivan, L. L., Parker, T. H. & Altschul, D. Build up big-team science. Nature 601, 505–507 (2022).

• 11.

  Weitzman, M. L. Recombinant growth. The Quarterly Journal of Economics 113, 331–360 (1998).

• 12.

  Uzzi, B., Mukherjee, S., Stringer, M. & Jones, B. Atypical combinations and scientific impact. Science 342, 468–472 (2013).

• 13.

  Simoiu, C., Sumanth, C., Mysore, A. & Goel, S. Studying the “wisdom of crowds” at scale. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing 7, 171–179 (2019).

• 14.

  Mulgan, G. Big Mind. (Princeton University Press, 2018). doi:10.1515/9781400888511.

• 15.

  Fiore, S. M. Interdisciplinarity as teamwork: How the science of teams can inform team science. Small Group Research 39, 251–277 (2008).

• 16.

  Baumgartner, H. A. et al. How to build up big team science: A practical guide for large-scale collaborations. Royal Society Open Science 10, (2023).

• 17.

  Coles, N. A., DeBruine, L. M., Azevedo, F., Baumgartner, H. A. & Frank, M. C. ‘Big team’ science challenges us to reconsider authorship. Nature Human Behaviour 7, 665–667 (2023).

• 18.

  Coles, N. A. et al. Team scientists should normalize disagreement. Science 384, 1076–1077 (2024).

• 19.

  Hall, K. L. et al. Assessing the value of team science. American Journal of Preventive Medicine 42, 157–163 (2012).

• 20.

  Birkland, T. A. Focusing events, mobilization, and agenda setting. Journal of Public Policy 18, 53–74 (1998).

• 21.

  Priem, J., Piwowar, H. & Orr, R. OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts. (2022) doi:10.48550/ARXIV.2205.01833.

• 22.

  Altmetric Engineering et al. The state of altmetrics: A tenth anniversary celebration. (2020) doi:10.6084/M9.FIGSHARE.13010000.

• 23.

  Fisher, A., Rudin, C. & Dominici, F. All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously. arXiv (2018) doi:10.48550/ARXIV.1801.01489.

• 24.

  Hall, K. L. et al. The science of team science: A review of the empirical evidence and research gaps on collaboration in science. American Psychologist 73, 532–548 (2018).

• 25.

  Hsiehchen, D., Espinoza, M. & Hsieh, A. Multinational teams and diseconomies of scale in collaborative research. Science Advances 1, (2015).

• 26.

  Larivière, V., Gingras, Y., Sugimoto, C. R. & Tsou, A. Team size matters: Collaboration and scientific impact since 1900. Journal of the Association for Information Science and Technology 66, 1323–1332 (2014).

• 27.

  Jones, B. F. The rise of research teams: Benefits and costs in economics. Journal of Economic Perspectives 35, 191–216 (2021).

• 28.

  Coles, N. A., Goes Braga Takayanagi, J. F., Grant, G. L. & Basnight-Brown, D. M. The rise, impact, and imbalances of big-team psychology. Psychological Science 37, 287–297 (2026).

• 29.

  Wu, L., Wang, D. & Evans, J. A. Large teams develop and small teams disrupt science and technology. Nature 566, 378–382 (2019).

• 30.

  Heide, E. A. der. Convergence behavior in disasters. Annals of Emergency Medicine 41, 463–466 (2003).

• 31.

  Hermann, C. F. Some consequences of crisis which limit the viability of organizations. Administrative Science Quarterly 8, 61 (1963).

• 32.

  Rosenthal, U. E., Charles, M. T. & Hart, P. T. Coping with Crises: The Management of Disasters, Riots and Terrorism. (Charles C Thomas, Publisher, 1989).

• 33.

  Boin, A., Hart, P., Stern, E., Stern, E. & Sundelius, B. The Politics of Crisis Management. (Cambridge University Press, 2017).

• 34.

  Falk-Krzesinski, H. J. et al. Mapping a research agenda for the science of team science. Research Evaluation 20, 143–156 (2011).

• 35.

  Hall, K. L., Vogel, A. L. & Croyle, R. T. Strategies for Team Science Success. (Springer International Publishing, 2019). doi:10.1007/978-3-030-20992-6.

• 36.

  Kaufmann, D. & Kraay, A. The Worldwide Governance Indicators: Methodology and 2024 Update. (Washington, DC: World Bank, 2024). doi:10.1596/1813-9450-10952.

• 37.

  Hsiehchen, D., Espinoza, M. & Hsieh, A. Multinational teams and diseconomies of scale in collaborative research. Sci. Adv. 1, e1500211 (2015).

• 38.

  Bates, D., Mächler, M., Bolker, B. & Walker, S. Fitting linear mixed-effects models using lme4. Journal of Statistical Software 67, 1–48 (2015).

• 39.

  Aczel, B. et al. Investigating the analytical robustness of the social and behavioural sciences. Nature 652, 135–142 (2026).

• 40.

  Erickson, N. et al. AutoGluon-tabular: Robust and accurate AutoML for structured data. (2020) doi:10.48550/ARXIV.2003.06505.

Data availability

Processed data and code are available at https://github.com/Social-Science-Collaboratory/btmobile_sharing (release assets include the processed data). Data were sourced from OpenAlex (CC0), Altmetric Explorer (licensed; cannot be redistributed), and the World Bank Worldwide Governance Indicators (CC-BY 4.0). Altmetric data cannot be openly shared because of its proprietary nature. For users without licenses, we provide simulated copies of the data. For users with licenses, we provide information on how to replicate the data download.

Code availability

Author contributions

Conceptualization: N.A.C., L.W., J.F.G.B.T., S.M.F. Data curation: N.A.C., J.F.G.B.T. Formal analysis: N.A.C., J.F.G.B.T. Funding acquisition: N.A.C. Investigation: N.A.C., J.F.G.B.T. Methodology: N.A.C., L.W., S.M.F., J.F.G.B.T. Project administration: N.A.C., J.F.G.B.T. Software: N.A.C., J.F.G.B.T. Supervision: L.W., S.M.F., N.A.C. Validation: J.F.G.B.T., N.A.C. Visualization: N.A.C., J.F.G.B.T. Writing - original draft: N.A.C. Writing - review & editing: N.A.C., J.F.G.B.T., L.W., S.M.F.

Competing interests

The authors declare no competing interests.

Figures

注意转载授权 · 阅读原文