This article explains the evaluation methods for discontinuous innovation in science and technology policy based on the latest trends in metascience research.
Key Points
1. Evaluation Method Using Disruption Index
- An index developed by a University of Chicago team and published in Nature in 2019
- Calculated by classifying subsequent papers into three categories (citing only the target, citing both, or citing only references)
- Completely disruptive papers score D=1, while developmental papers score D=-1
- Watson & Crick's DNA structure paper (D=0.96) and Mandelbrot's fractal paper (D=0.95) received high scores
2. Evaluation of Japanese Research Disruption
- Top cited paper is Hirotugu Akaike's "Akaike Information Criterion (AIC)" with 48,000 citations
- Most disruptive paper is Akira Fujishima's "Photocatalysis" with an extremely high D=0.998
- Toyota Motor's "Toyota Production System and Kanban Method" also has a high D value
- Professor Koichiro Tamura's MEGA series has many citations but low disruption (developmental type)
3. Evaluation Method Using Novelty
- Complements the weakness of disruption index that "can only evaluate old research"
- First appearance of field pairs: Evaluates new combinations of fields never seen before
- Field pair distance: Evaluates the rarity of combinations
- First appearance and distance of word pairs: Measures novelty from word combinations in paper abstracts
4. Practical Application as Policy Tools
- UK plans to hold "Metascience Novelty Indicators Challenge"
- Prize money of £300,000 (approximately 60 million yen) co-sponsored by Elsevier and RAND Corporation
- Expected to be utilized for setting priority fields and risk analysis
- Important to combine with qualitative evaluation while recognizing limitations of quantitative indicators
The article concludes that while these quantitative evaluation methods can capture technological discontinuity and novelty early and be utilized for effective policy making, appropriate implementation considering their limitations is necessary.