Enlightened Experimental Design

I hate to see people wasting their time – and money – which is why I am such a passionate advocate of statistical Design of Experiments (DoE).

By definition, DoE is the most efficient way to experiment to understand a process or system and it is extremely relevant for analytical science. Just enter “HPLC design of experiments” into your favorite search engine; you will find plenty of examples where scientists have optimized the resolution of HPLC methods using DoE. The authors will likely have listed the benefits of the approach: the reduced time, cost and risk; the increased understanding of the complex behavior of the system; the structure and rigor that it brings to the development process. They may also mention that they identified that some of the factors have important interactions. For example, they discovered that the best temperature was dependent on the flow rate. They will acknowledge that they would not have found the optimum using a conventional one-factor-at-a-time (OFAT) approach.

Those of you who have developed chromatography methods will probably have seen such examples and, if so, will be aware of the benefits. And yet many of you will not use DoE. And I would be willing to wager that a large number of the DoE case study authors still use OFAT as their default approach. Published examples, by their nature, are special cases. Nevertheless, HPLC development lends itself very well to DoE because experimentation is relatively cheap. A large experiment can be run with few human, equipment and material resources, for instance. In a setting where each run or trial is more expensive, my experience tells me that the scientist will be less likely to use DoE – despite the fact that the imperative for efficiency is even greater!

Working in the chemical industry, my job was to find out how to optimize and control complex and messy systems. I remember one particularly challenging situation where each trial preparation was very costly and we did not have the time for many of them because we were under pressure to get quick results. The system behavior was expected to be complex because of the large number of physical and chemical interactions. To add to this, both experimental and measurement repeatability were very poor. Given these problems, the chances of finding useful information with a designed experiment were not good.

One manager, who was normally a big fan of DoE, said we should just see if we are able to find something that works by OFAT. This was, of course, the wrong response to the situation. But heading down this route would have further lowered the chances of finding useful information, making it a good way to waste the time of many scientists. DoE is, therefore, a project management tool that enables rational decisions, making it about the most effective way to spend resource to maximize information and minimize risks. But people – even managers (!) – are not rational. When the pressure is on, we all tend to resort to our comfort zone, which in this case was OFAT.

So, how do we make DoE the comfort zone? Advances in methods and improved software for designing and analyzing experiments are enabling more people (in more situations) to consider DoE. Another part of the answer has to be education. The message that you should only change one thing at a time was ingrained in me from an early age and was never challenged throughout my years in school followed by higher education (my master’s degree and PhD in chemistry). I don’t remember hearing about anyone using DoE in my department during my undergraduate or postgraduate studies. Looking back, these methods would have been extremely useful. However, academic research is generally narrow in scope and focuses on simple, model systems. Real-life problems in industry are more of a challenge. So, like most scientists that I have spoken to, I only really found out about DoE when I started in industry.

Universities should be preparing scientists with the key skills that they need in industry. Things are changing. A small number of universities recognize that teaching DoE as part of undergraduate science courses gives students – and therefore the institution – a competitive advantage. Development of analytical methods would be an ideal context for undergraduate science students to learn about DoE.

Change cannot happen quickly enough. It pains me to think of the billions of hours (and dollars, pounds, Euros...) of wasted experiments.