Transcript
Many scientists and futurists believe that socalled “self-driving labs,” will dramatically accelerate innovation, especially in the fields of chemistry and materials science. At a minimum, “self-driving labs,” make use of artificial intelligence (or AI) and automated robotic systems to expedite research and discovery.
At Business Briefings and Trends, we’ve been watching the evolution of this concept for nearly two decades. Real-world results indicate that self-driving labs hold tremendous promise for accelerating the discovery of new molecules, materials and manufacturing processes, with applications ranging from electronic devices to pharmaceuticals. While the technologies are still fairly new, some have been shown to reduce the time needed to identify new materials from months or years to days.
As a result, they appear to be on the verge of igniting a revolution in the way scientific discovery happens. But, to make the most of this opportunity, scientists need to learn from each other. To make this happen, DARPA-funded researchers at North Carolina State University are now proposing a suite of definitions and performance metrics that will allow researchers, non-experts, and future users to better understand what these self-driving labs are doing and how each self-driving lab is performing in comparison to others.
Self-driving labs are garnering a great deal of attention right now. But there are a lot of outstanding questions regarding this technology as the researchers explained recently in Nature Communications. For example, different research teams are defining ‘autonomous’ differently. By the same token, different research teams are reporting various elements of their work in different ways. This makes it difficult to compare these technologies to each other, and comparison is important if we want to be able to learn from each other and push the field forward.
Researchers need to be able to address fundamental methodological questions, such as, “What does Self-Driving Lab One do really well?” and “How could we use that to improve the performance of Self-Driving Lab Two?” The team at NC State is proposing a set of shared definitions and performance metrics, which they hope will be adopted by everyone working in this space. The end goal is to allow scientists to learn from each other and more rapidly advance these powerful research acceleration technologies. For example, scientists seem to be experiencing challenges in self-driving labs related to the performance, precision, and robustness of some autonomous systems.
This raises questions about how useful these technologies can be when widely deployed. If scientists adopt standardized metrics and reporting of results, they can identify such challenges and better understand how to address them. At the core of the new proposal is a clear definition of self-driving labs as well as the following seven proposed performance metrics, which researchers would include in any published work utilizing self-driving labs.
-
The degree of autonomy, that is “how much guidance does a system need from users?”
-
The operational lifetime, that is “how long can the system operate without intervention from users?”
-
The throughput rate, that is “how long does it take the system to run a single experiment?”
-
Experimental precision, that is “how reproducible are the system’s results?”
-
Material usage, that is, “what’s the total amount of materials used by a system for each experiment?”
-
The accessible parameter space, that is “to what extent can the system account for all of the variables in each experiment?”
-
Optimization efficiency, which involves quantitatively analyzing the performance of the self-driving lab and its experiment-selection algorithm by benchmarking it against a baseline, such as, random sampling.
According to the NC State team, having a standardized approach to reporting on self-driving labs will help to ensure that this field is producing trustworthy, reproducible results that make the most of AI programs that capitalize on the large, high-quality data sets produced by self-driving labs
|
