Code Brain | How to Identify Disruptive Achievements in Basic Research

Breakthroughs in basic research are the cornerstone of human social development. But if we rewind to the moment when "great" achievements were born, those who created disruptive results were often labeled "heretical," and the acceptance and recognition of their work was an agonizingly slow process. The progress of human civilization today tells us that learning to identify disruptive achievements in basic research is crucial. How do we recognize such breakthroughs? Today's discussion offers one approach for collective exploration.

The following article is from Fanpu (fanpu2019)

By Xia Li and Shuang Li (Institute for the History of Science and Scientific Culture, Shanghai Jiao Tong University)

Disruptive achievements fall into two categories: those that occur by chance, and those that depend more on policy support. Identifying these two types requires completely different methods.

With the 2018 State Council release of Opinions on Comprehensively Strengthening Basic Scientific Research, China sounded the call to bolster basic research. Two years later, in 2020, five ministries jointly issued the Work Plan for Strengthening "From Zero to One" Basic Research — a refinement and operationalization of the State Council policy by science and technology administrators. By then, a broad consensus had formed nationwide on strengthening basic research. In the two years since, the proportion of basic research investment in R&D (research and experimental development) has steadily increased, indicating that policy effects are gradually becoming visible.

Now we face an urgent practical question: how to identify and evaluate basic research achievements, especially those with disruptive potential. If this problem is not solved well, it can easily lead to policy failure and misallocation of resources.

The identification and evaluation of basic research achievements is a global challenge. The difficulty lies in the fact that basic research sits at the front end of knowledge production, and its outputs mostly exist in the form of ideas, theories, or hypotheses — leaving no ready-made identification standards available.

This inevitably brings two potential risks: first, the inability to distinguish whether a theory/achievement is scientific or non-scientific (reasonable or unreasonable); second, policy incentives failing to hit their targets — in other words, what theories/achievements and who in basic research should receive incentives or funding?

Because basic research achievements mostly exist as conceptual forms, distant from everyday life, they cannot be verified or falsified within specific spatiotemporal contexts. This is even truer for disruptive achievements. As American philosopher of science Dudley Shapere (1928–2016) noted: because they are inherently ambiguous, they can only be described in ambiguous terms; their clarity is only gradually achieved. Every era has a set of seemingly reasonable, dominant beliefs. When disruptive achievements (new ideas) emerge, people face conflict and contradiction between old and new beliefs in their minds, leading to feelings of aversion or fear — which further exacerbates the difficulty of identifying and accepting disruptive achievements. To address this difficulty, I attempt two approaches: first, using materials from the history of science to reveal some common characteristics of disruptive achievements, providing a cognitive foundation for identification; second, using some results from philosophy of science to construct a set of evaluation standards, thereby improving the accuracy of identifying disruptive achievements.

Classification of Basic Research and the Space Where Disruptive Achievements Emerge

Across the entire knowledge production chain, research types can be divided into basic research, applied research, and experimental development research — this is the well-known basis for R&D investment classification.

Even returning to basic research at the front end of knowledge production, its internal structure is highly complex. American policy expert D. E. Stokes once divided basic research based on research purpose: pure basic research (the Bohr quadrant) and use-inspired basic research (the Pasteur quadrant). The former is basic research driven by researchers' curiosity, aimed at revealing the nature of the world; the latter is basic research triggered by solving specific applied problems, which can be seen as knowledge byproducts brought about by addressing practical issues.

Based on the difficulty of basic research, I proposed in 2021 a three-stage model of basic research: the 0–0.5 stage, the 0.5–1.0 stage, and the 1–N stage. The third stage is essentially where basic research transforms into applied research, so true basic research lies in the first two stages. This division is important for identifying disruptive achievements. We can call achievements in the 0–0.5 stage of basic research Disruptive Achievement Type I, and those in the 0.5–1.0 stage Disruptive Achievement Type II. This division provides an operable path for selecting different identification methods going forward. See the figure below (where the "Tycho quadrant" in the lower left was named by Professor Jiarui Wu of the Chinese Academy of Sciences):

From the perspective of occurrence frequency, Disruptive Achievement Type I in the Bohr quadrant is extremely rare in human history. Such achievements are completely contrary to mainstream views/paradigms of the time, difficult for people of that era to accept in the short term, often treated as heresy — yet they tend to trigger far-reaching scientific revolutions.

Disruptive Achievement Type II in the Pasteur quadrant is what we commonly see in the history of science. They usually bring cognitive分歧 and confusion, but as the theory evolves and becomes increasingly refined, it provides tremendous impetus for rapid development in a particular field.

Relatively speaking, the identification of Disruptive Achievement Type II has more empirical methods to draw upon than that of Type I. The output of Disruptive Achievement Type II requires policy regulation, while the emergence of Type I is entirely a chance event requiring no excessive policy intervention.

Identification Methods for Disruptive Achievement Type I and Type II

Due to the vast differences in occurrence frequency and difficulty between the two types of disruptive achievements, their identification methods are completely different. We discuss each below.

Disruptive Achievement Type I The main characteristic of this type is "heretical." It is completely contrary to people's beliefs at the time, far beyond that era's cognition, leading to long periods where it can be neither verified nor falsified. The acceptance and recognition of such achievements is a very slow process, and their proponents often endure much personal hardship — there are many such cases in the history of science.

No one knows which cloud will bring rain; this captures the difficulty of all ex ante evaluation. Therefore, the identification of such achievements can only adopt a historical evaluation approach, and the focus of evaluation is not the achievement itself but the proposer. Specifically: the proposer's personality characteristics and specific epistemological doubts about the theory.

Consider the case of Copernicus publishing De revolutionibus orbium coelestium in 1543 — a classic Disruptive Achievement Type I. Copernicus's heliocentric theory sounded the trumpet of the first scientific revolution, but at the time, his theory was completely contrary to the dominant Ptolemaic geocentric model.

In fact, Copernicus had already developed the core ideas of this theory by 1509, but to avoid persecution by the Inquisition for heresy, he only decided to publish the work on his deathbed. Fifty-seven years later, Bruno paid with his life (1600); eighty-nine years later, in 1632, Galileo narrowly escaped being burned at the stake by the Inquisition for propagating this theory. Clearly, Disruptive Achievement Type I looks great in retrospect but was extraordinarily perilous at the time. Their acceptance was universally slow and tortuous.

Consider the fate of such achievements in the 20th century. German meteorologist Alfred Lothar Wegener (1880–1930) proposed the continental drift hypothesis in 1912. Wegener's "earth-shattering" view immediately shocked the scientific community of the time, drawing far more attacks than support — yet he steadfastly believed in his own view.

Regrettably, Wegener did not live to see his theory confirmed. He died in the icy wasteland of Greenland searching for evidence. Thirty-three years after his death, in 1963, people discovered the widespread existence of seafloor magnetic stripes, thereby proving his theory correct.

Through梳理 of the history of science, we can also discover an interesting phenomenon: scientists who made major contributions often possess unique personality traits. Summarized, revolutionary scientists mostly exhibit four personality traits: introversion, intuition, thinking, and judgment (Li and Li, forthcoming). This means that scientists who produce disruptive achievements are well aware of the enormous impact of their theories; proposing such theories is never a hasty decision but the result of deep deliberation, and they are willing to bear the risks. This personality trait ensures the academic moral reliability of the achievement. At this moment, I am reminded of a Nietzsche book title — Ecce Homo — which quite expresses our current sentiment.

Doubt must be specific. So how should the scientific community of the time evaluate these "heretical" theories? At the time, Einstein's explanation of the photoelectric effect was a major advance on Planck's theory, even overturning the entire classical physics system, yet two years after Einstein's paper was published, Planck still harbored resistance; he warned that the young patent clerk had gone too far. However, six years later, when nominating Einstein for a coveted seat at the Prussian Academy of Sciences, his recommendation letter was full of praise, though he still added: "Sometimes he may have speculated too far, as with his light quantum hypothesis, for which we should not blame him too harshly." (Einstein: His Life and Universe, Walter Isaacson, translated by Butian Zhang)

Thus, doubt should be directed at specific points of the theory, not a wholesale rejection. In other words, what you disagree with should be specified, not comprehensively doubted. Combined with smooth publication processes, open discussion spaces, and specific rather than comprehensive negation, this allows penetration to the core of the theory — something the history of science has long proven to be an important path for advancing scientific development.

To simply summarize, the identification method for this type of achievement consists of three elements: historical evaluation + personality characteristics + specific doubt. Although we have no way to immediately identify which achievement is genuine, at least this identification model leaves precious time, space, and tolerance for the unfolding of theoretical content. Moreover, such research is originally entirely driven by scientists' curiosity; not persecuting or suppressing it is the greatest support for disruption itself.

Disruptive Achievement Type II Since this type of disruptive achievement is mostly built on some prior research foundation, its identification method differs from that of Type I. Relatively speaking, prior research has accumulated some useful identification methods. Broadly speaking, there are three methods for identifying Disruptive Achievement Type II:

First, comparing the empirical content richness of theories. This is a method proposed by philosopher Popper for distinguishing superior from inferior theories. Its essence lies in the belief that progressive theories can solve more empirical problems than degenerating ones, and over time, form richer theoretical accumulations.

Second, comparing the predictive capacity of theories. In Popper's view, progressive theories have stronger predictive power, which is also the main marker of theoretical progress. For example, many predictions made by Einstein's general theory of relativity were subsequently proven by experiments, so Einstein's theory is progressive. The implementation of both methods requires drawing on the brainstorming method commonly used in identifying disruptive technologies — that is, through rapid thinking and discussion to identify the empirical content richness of theories and their predictive capacity. This is also the responsibility of the scientific community in identifying disruptive achievements.

The problem is that many disruptive theories do not have such obvious advantages initially. At this point, a third method can be adopted: the comprehensive scoring method. It divides theoretical merits into five dimensions: accurate, consistent, broad scope, simple, and fruitful. These five indicators can be assigned different weights as needed, and then scores are calculated; the theory with the highest total score is the best theory. In short, this identification model is ΣT = A + C + B + S + F. This model is summarized based on the views of philosopher Thomas Kuhn.

How to Promote the Output of Disruptive Achievements in Basic Research?

To increase the output of disruptive achievements in basic research, what should we do?

Given the lag and difficulty in identifying such achievements, we still need to look for clues in cases from the history of science. To this end, I selected 40 scientists who produced disruptive achievements in basic research between 1800 and 2006 (such as Wöhler, Maxwell, Einstein, Heisenberg, Watson, etc.) to see what secrets still hidden behind their success. The summary is as follows:

First, statistical analysis found that the average age of these 40 scientists when they produced disruptive scientific achievements was 36.45 years. This conclusion is basically consistent with the results of our previous multiple studies — that scientists reach their peak age of academic creativity around 38 years old (Li et al., 2000). This suggests that disruptive achievements are mostly produced by young people. This conclusion provides a useful reference for future policy-making regarding basic research talent.

Second, based on the nationality information of these 40 outstanding scientists, we can roughly infer their cultural types: predominantly Protestant/Puritan culture, accounting for 82.5% (33/40) of the total. This conclusion once again confirms our previous research findings: cultural transformation affects talent development and scientific and technological development in three ways:

First, new cultural paradigms can shape new worldviews. For talent development, new cultures can provide a new perspective for viewing the world, allowing them to see phenomena that old paradigms could not; second, based on new cultural paradigms, a new social order can be established. New social orders often bring liberation and expanded freedom, providing potential space for action; third, new cultural paradigms can expand the trust radius of interpersonal trust and system trust, especially trust between groups.

The expansion of a society's trust radius can effectively reduce transaction costs across society, thereby facilitating the growth of talent, knowledge, and idea markets. (Li and Zhaoyi Gu, 2022)

See the figure below for nationality information of the 40 scientists:

Third, increase investment in basic research. All knowledge production requires corresponding scientific and technological resources; for basic research, it is the same, especially for Disruptive Achievement Type II in the 0.5–1 stage, which requires more funding support (Disruptive Achievement Type I is more sensitive to the research environment and ecosystem). In short, the basic conditions for supporting basic research can be expressed by the formula: Basic research output = talent + investment + policy + culture + public opinion, where output includes various types of disruptive achievements. There is no such thing as a free lunch; knowledge production follows the same principle — without investment, how could achievements appear from thin air? According to our statistics, developed Western countries have maintained the proportion of basic research investment in R&D at around 15% for many years.

Taking the United States as an example, over the past 21 years (2000–2020), the proportion of basic research investment in R&D averaged 17.2%, which proves a fundamental reason why those science and technology powerhouses perform so well in basic research. In any era, material poverty also causes intellectual and creative poverty among groups.

Fourth, a free public opinion environment plays an important role in improving research quality and identifying disruptive achievements. As shown in the figure above regarding the nationality distribution of outstanding scientists, countries with excellent basic research performance tend to have relatively free and relaxed public opinion environments, which are conducive to knowledge production and the emergence and exchange of innovative ideas. The logic is simple: with a relaxed public opinion environment, one can avoid the phenomenon of unqualified people passing as qualified, and through the collective wisdom of the scientific community, eliminate inferior low-level repetition and fraud; at the same time, with the group wisdom of the scientific community, the significance and value of disruptive achievements can be more effectively identified. Perhaps more importantly, the development of the scientific community can raise the overall cognitive level of the group, thereby helping to improve basic research capabilities and identification levels.

Perhaps worth further elaboration: the richer and more complete the social foundational support conditions for knowledge production, the more knowledge output there will be, and the more likely it is to achieve major disruptive achievements in basic research. Research in the history of science has long fully proven that this development path is the only way for science and technology; major disruptive basic research achievements are rarely produced on barren land. This forms a paradox: the better the conditions, the more knowledge can be produced, and the more knowledge is needed, thereby forming positive feedback in the interaction between knowledge and development — and vice versa.

And more importantly, where the scientific community is more fully developed, the speed and accuracy of identifying disruptive achievements also increase. Recall that Copernicus's achievement took a tortuous 100 years from proposal to acceptance; the entire society's cognitive baseline was then relatively low, delaying the acceptance of Copernicus's theory. Yet from Einstein's proposal of general relativity to the verification of its light-bending prediction took only four years — all because Western Europe at that time possessed a relatively developed scientific community and related conditions, enabling Arthur Stanley Eddington's (1882–1944) famous total solar eclipse observation, which convinced people that light bends, partially verifying general relativity and sending Einstein to the pantheon.

For China, there are no special exceptions; it equally needs to improve the social foundational support conditions for basic research, following and learning from the successful experiences and paths in the history of science.

As the saying goes, to do a good job, one must first sharpen one's tools. Only thus can disruptive scientific achievements in basic research be produced in the shortest possible time.

About the authors: Xia Li is Professor and Doctoral Supervisor at the Institute for the History of Science and Scientific Culture, Shanghai Jiao Tong University.

Shuang Li is a doctoral candidate at the Institute for the History of Science and Scientific Culture, Shanghai Jiao Tong University.