Breaking out of the Black Box

To their credit, instrument companies are making impressive technological progress across many areas; we are all benefitting. However, this progress also raises a concern over the gap that is emerging between the instruments being developed and the customers who press the buttons. It’s the dreaded “black box” syndrome: some users simply don’t understand the underlying science or methodology; they just assume that the results they generate are correct – without questioning them, or even knowing how to question them.

This is not the fault of the instrument manufacturers. Their goal, like all businesses, is to make money, and the products that sell best are the ones that are easiest to use and maintain: in other words, black boxes. In my opinion, we should not rely solely on companies for instrument development.

I can understand that people like clear and simple results, but the misinterpretation of data is a constant risk of unknown quantity; however, given the current heavy reliance on data, we need to quantify the issue. In certain circumstances, it could be a serious problem; for example, in hospital clinics, where results are often interpreted in a “black or white” (that is, positive or negative) fashion. Clinicians must make life-critical decisions on the basis of “the numbers” but don’t have the background, the focus or the time to question those data. They rely on being given a single, correct result; reality, however, is often more complicated.

The solution to this black box problem is to emphasize education. People – not instruments or computers – need to fully interpret data. So we need people who understand what the samples being interrogated are, and what the analytical question being asked is. Generally, those performing analysis are insufficiently trained in quality control of data and in assessing methodology. This is something that we must correct from an educational perspective, not only at university, but also in analytical laboratories and in industry.

There is an additional, linked issue. While we need better training for analytical chemists, there are worrying signs that the importance of the field within the scientific world is on the wane. This is a threat to the profession but, more importantly, it carries risks for science and society. We have to correct the reputation of analytical science, and we can start by developing a clearer sense of what we believe our role and input should be. Then we need an educational program for the upcoming generation, at a European scale. Properly set up, this will help to attract more attention to the challenges that we face. At present, the handling of this issue is ominously discreet.

Nowadays, fellow scientists tend to look on us analytical chemists as collaborators who generate data and, perhaps on some level, interpret it. However,  if we are not fully integrated into projects there is a practical consequence: we can’t guarantee that we will provide the right data (or answer) since mistakes may have been made prior to our involvement. Furthermore, analytical chemists need to learn about the fields that they are working in. In proteomics, you need biochemical knowledge, for example, otherwise, you’re not speaking the right language; for environmental samples, you need to understand organic compound degradation. It’s not just data. Thus the analytical scientist needs to be a broadly educated chemist first, and must match that to a fundamental understanding of several different fields. Many analytical chemists are even missing basic knowledge of organic chemistry, which is something that I find quite disturbing.

I’ve always taught analytical chemists that there is an absolute need to relate data back to the sample and to the analytical question being asked. Of course, this brings the risk of digging too deep and getting lost but more often than not it helps discover something new and interesting. I remember doing sample preparation with ultrasonication and discovering that my molecules were being degraded by the process; I used that knowledge to create a new technology to degrade mycotoxins in grain and corn samples. That was a real innovation, made possible by real observation. Analytical chemists tend not to think in this way or to explore issues to this depth, and without the proper educational background, they simply can’t. Constantly augmenting one’s knowledge is an essential task for the modern analytical scientist.