John P. O'Connell, Ph. D.
Chair, U.Va. Department of Chemical Engineering
University of Virginia
"On the Nature and Conduct of Technical Research"
September 6, 2001

John O'Connell: So, what is research? Well, what do you do when you want to know a definition? You go to the Oxford Unabridged Dictionary:

Research: A research or investigation directed to the discovery of some fact by careful consideration and studying of the subject by close and scientific inquiry.

That's kind of dry. Would you be inspired to do research like that? I don't think so. But, the Oxford Unabridged also has quotes of the usage of the words.

The matter that lies deep in nature requires much research to unfolding. -- William Older, 1694

Our most profound researches are frequently nothing better than guessing at the causes of the phenomenon.

Guessing? That's 1799. Are we any better off these days? Well, we like to think that we are not "guessing" in the sense they might have been implying. Scientific truth is not fixed in universal; one of things you have to realize is that there is an evolution, just like the issue of what was guessing then and what is guessing now. Even what we call the scientific method, in fact, does evolve. Something really does seem to attract people all over the world, whether poor or rich, to seek, observe and develop new ideas for describing the physical, chemical, biological, an d social worlds more accurately and completely.

One of the things I have been fascinated with as I travel around the world is the diversity of people who say, "I must do this," it is sort of like art--the starving artist starves because the artist has to do art--that is just who they are. The same thing happens in terms of scientists and engineers, interestingly enough. So, why do people do it? The upside is they like it and they consider it exciting and fun and you get success, there are personal and communal triumphs, it stimulates challenging assumptions, relative freedom and responsibility. The results can positively affect the quality of life. Of course, if there is an upside, there is always a downside. The downside is the failure of the experiment. The failure of hypothesis--you make a guess and guess what? You are wrong. There are disagreements, also. Arguing can be fun, depending on the personality, but there can be a downside, also. There are also disagreements in interpretations and priorities. Maybe it is just a lottery. The state lottery for scientists only--the grand prize is the new paradigm! Second prize is major discovery, third prize is a minor discovery, fourth prize is a minor discovery, and fifth prize is a viable hypothesis. But, everybody can get a prize! Not everybody can get a Nobel prize, right? But, people who do ordinary science, as it is called, get rewarded as well about the things that are on the upside. It is not always this positive. Science is a human process and it is never perfect. Murphy's Law, the original law that if something can go wrong, it will, it does.

This is one of my favorite quotes:

"The subtlety of Nature, secret recesses of truth, obscurity of things, difficulty of experiment, implication of cause and the infirmity of man's discerning power, [will make it so that] men are no longer excited, either out of desire or hope, to penetrate farther." -- Sir Francis Bacon

So, it turns out that there are more downs than ups to research. So, if you are going to make a career of it, the ups better be pretty good for you.

Who should do research? A set of books were written about this and they actually give a set of questions which you can ask, and depending upon your answer, you can figure if this is the way you really are or not. So, this is the personal traits, also, of those with likely success in research, whether you are talking Nobel prizes or ordinary science--the lottery prizes.

One of the quotes is, "scientists are people of very dissimilar temperaments doing different things in very different ways." That is the richness of research, fortunately. Among scientists are collectors, classifiers and compulsive "tidier-up." Many are just detectives, explorers, artists, and artisans. There are poet scientists, philosopher scientists, and even a few mystics. Don't forget the modelers. In fact, that is a lot of what we have as our business. And people might say, "is this legitimate, is this good science or research?" I would argue yes, because successful models do advance knowledge and practice by showing the essence of a behavior. We can't figure out how to do everything in nature completely because we are just humans and we have limited resources. And so, the models will show the essence of what is going on and then we can use this to predict might happen. So, they provide an understanding of the relative importance among many contributions of a complex situation and a reliable basis to implement the use of them. That is why engineers tend to use models actually more than scientists do. We use them more openly and unabashedly, I think, than scientists do.

How does one do research? Wilson and Booth's books are actually sort of like manuals of investigation whereas the book by Oliver is more philosophical and attitudinal. Let me show you the tables of contents, for example. This is from 1952, and the recipe to follow.

Also, Oliver’s book: The Incomplete Guide to The Art of Discovery is about discoveries, strategies, tactics, the personal traits and attitudes of discoverers, certain caveats, a few views and comments on science and the inside story of one discovery.

The young workers need to realize and understand that it is okay not to understand everything—you are on a journey. In fact, research is a game unlike class. Remember the game in class is that the teacher pretends to know everything and the student’s game is to find out what the teacher knows. It turns out in research that no one knows the answer. So, you can’t expect to understand everything as you go along because that is what you are trying to find out. It is actually better, sometimes, to be wrong. A certain philosophy of science is that you do the best you can, expecting it is not going to be completed perfect, put it out there, and let people chew on it. Progress will alleviate because they will get inspired to say, "I know that is wrong. What is right?"

The social foundation of research is that it is not the popular stereotype of a lonely, isolated search for truth. It used to be that way, but it is not that way anymore. We have to do it with other people and use the literature (which is huge), and we are doing collaborations which are becoming much more of the mode of operation. It inevitably takes place within a broad, social and historical context, which gives substance, direction and ultimate meaning to the work of individuals. So, individuals have to be part of a team, part of a community, in order to nowadays get the substance of direction and meaning.

An individual’s knowledge properly enters the domain of science only after it is presented to others so that they can independently judge the validity—that is the scientific method. You do the best you can; you put it out there, and people figure out whether you are right or not by validating it separate from you. Part of what you had to do was to help that process. So, proper presentations turn out to be in conversations, computer mail, meeting presentations, manuscripts that are reviewed before publication and published papers. The process of review and revision is critical because it minimizes the influence of individual subjectivity by requiring the research and results to be accepted by other scientists. We easily fool ourselves into thinking that we have it all right—again, a problem we have as humans. The scientific method, unlike the part of all others, we actually put it all out there and accept criticism and review. Most workers that I know will say that a paper that was finally published is much better after review than it was before.

It is also a powerful inducement for researchers to be critical of their own conclusions because they know they must try to convince their colleagues. So, what you should do, according to Michael Brown, is take a very possible way to shoot down your own idea before you begin to accept it. You should be self-critical, as much as possible. That is kind of frustrating, and it can be kind of a downer because, sometimes, after you go through this, you say, "uh-oh, I was wrong after all." But, on the other hand, you know something, and then you can make progress out of it. So, science has progressed through a uniquely productive marriage of human creativity and hard-nosed skepticism, of openness to new contributions and persistent questioning of these contributions and the existing consensus. The productive marriage is the difference back and forth about doing something and then looking at it harshly, and then going back and doing it again. The process of evaluation has evolved as knowledge and techniques—the way I review my papers now is thought different from when I started, particularly because we have computational tools available to us. Someone comes out with a model that fits everything and it has a database. It takes a lot of work to do this, but that is the kind of thing we can do more easily these days.

What problems are encountered? If we are trying to find the truth through our empirical validation, we do this with data and in some sense, simulation. If it were easy and fun, everyone would do it, right? Here is "pseudo-science incorporated!" That is the nice thing about working in this place—we don’t have to finish any of our experiments. Wouldn’t it be neat if we just did something and said, "Yeah, there it is"?

Then, we could ask if we have to do this or that measurement. Well, let’s go back to history:

"To learn secrets of nature, we must first observe." –Francis Bacon

"Developing theories without data is like making bricks without clay."

-Sherlock Holmes

"Speak (listen) to the Earth and it shall teach thee." – Job12: 8

Experiment and data treatment are tough, but crucial to establishing scientific truth. So, we must utilize experiments effectively. We have to figure out how we do the minimum experiment and still get the information that leads us on. Data are fallible. We have to maximize the truth by examining and validating all the data with organized, searching skepticism. It is the same thing as we discussed before—you analyze it skeptically in terms of what you have. It may be less or it may be more than meets your eye. Let’s look at some cases.

Lord Rayley (?) discovered argon. How he did this was he noticed that the density of the gas, which was thought to be nitrogen (after which it absorbed the oxygen from there), was different than if you took a chemical reaction and generated nitrogen. He realized that there was more than just nitrogen and oxygen in there. That had to be pretty subtle to figure that out, but that was how he did it.

Galileo, in 1613, actually suggested that the Planet Neptune existed, even though it could not be seen. He actually made a drawing and ignored it. So, it took another two hundred and thirty-four years before people could say that Neptune existed. What a shame—he wasn’t paying enough attention. Or, maybe he did not have that driving force to alter the data again.

Mick Elson, who did the oil-drop experiment to determine the electron charge and so forth, decided, in advance, that electrons came only in integer values (1, 2, 3, 4, and so forth). It turned out there was a whole bunch of data that had only one third the values and he threw them out. It turned out that actually quarks would be ones that would do that, and he just missed that.

If you want to look at ways between measured value, you can look at Wilson. There is a long chapter about this. There are limitations, techniques, and equipment that will cause uncertain values and error bounds. We always have to recognize this. No experiment is perfect. As many of you have found, or will find, when you take an oral exam with the faculty, our faculty is particularly interested in when a student makes a plot and there are data points. Someone will say that there is uncertainty in those numbers and that they are not very happy when students do not have an answer to that question.

There is incomplete communication and measurement condition in analysis. One of the most frustrating things to encounter when you are doing an experiment or when you are trying to reproduce what someone else has done is that they do not tell you everything. It is very hard to figure out how to communicate, but that is our job. We want to make sure that we help the process of independent verification.

Rejection and retention of data points—what do we do about those out-lyers? Is there reality in them? How does one figure this out? These are the kinds of questions that Wilson considers. What is considered a good fit of a model, and would you stake your job on that?

(Showing graph) When data doesn’t tell us enough on its own, I change it. I have done three things, here, which have helped me decide that this is the way it has to be. One thing is that I happen to know that y=0 when x=0. That wasn’t on the plot, but it is a validated and anchored point. The second one is that we look at the fact that some theory will say that this is in a linear region. That may or may not be true and you may or may not know that. But, let’s say that it is polynomial in form, and so we are only in a linear region. The last thing is we ask if that is consistent with the data, and this is where we put in the uncertainties. We would say, on the basis of the presentation of this plot, that this is the best that we could do because it has to do this and this. It is not inconsistent with the data, but still, this is the perspective that I want you to get.

"I have never made a contribution that I didn’t get by fiddling with the equations."

-Linus Pawling quoting physicist

"I have never made a contribution that I didn’t get by just having a new idea. Then, I would fiddle with the equations to help support it."

-Linus Pawling

Fiddling was probably the dominant mode of quantitative exploration until computers came along and allowed us to make big discoveries. So, let’s talk about computation and research. Computers allow us to do simulation imaging and synthesis in ways that we could never do before. Now in our research repertoire, it is pushing us into quantitative descriptions of nature which allow us to examine multitudes of data and models, creating images at distance and time scales that one cannot even do experimentally.

Are there issues in simulation? Of course there are. How good are the computer results? Are they qualitative, reliable, and accurate? What are the error bounds? We have to validate simulations in the same way we validate experiments—we have to be aware of sensitivity, assumptions, sampling, coding errors and significant figures. Validation of something that you are calculating about nature should be compared with real experiments. This is not always easy because the results are not always put in a form that the simulation gives. But the job of an experimentalist is to put it in a form that the simulators can check out. And, the job of a simulator is to put it in a form that the experimentalist can check out. That is where the meeting comes together. We have, at least, our fools and scoundrels in simulations. Simulation is very seductive, but, like most things, seductive is not necessarily wholesome.

This is from Hans Christian Anderson who is a chemist at Stanford--not the original:

"Machines should work, people should think."

I often tell students that if they think that their job is to reproduce what a computer does they can be replaced by a computer, and they will be. On the other hand, guys play games, like, "I am a molecule or two away from the finished formula, chemicals in ways that my computer never dreamed of." That is right—a computer only does what you tell it. Nature is generally richer. What you figure out is that, if nature does this, can a computer do it?

What kinds of problems are accounted in the human issues? First off, there is this issue of selecting the best hypothesis. Galileo had two opportunities: he could have said that they fell at the same rate or at different rates. So, he picked one. There are many competing hypotheses and we have to figure out which ones to proceed with. What you want is a hypothesis that shows internal consistency, has accurate correlation and prediction of experimental data, and unifies, if at all possible, the apparently disparate results. People can’t see the connection and your job is to show them how they connect—a major contribution. Our your instincts and experience going to give you progress on this? Part of your education is to, in fact, hone your instincts, skills and experience so that you have a better batting average when you get to making advances. You should have a desire for truth, beauty and quality. That is the way that life should be lived, in my own opinion, so here is the opportunity to do it. One of Murphy’s Laws is that assumption is the mother of all screw-ups. You need to know what you have assumed so that you can minimize the chance of screwing up. Does that help or hinder things in terms of assumption? Assumptions about things that do not matter are fine—it doesn’t matter. On the other hand, with things that do matter, you could get in trouble if you perceive the wrong thing.

There is also the issue of ethics. Human issues have caused questioning and threatening in the U.S. scientific enterprise. I first got into this because there were a couple of publications, one that came out called, On Being A Scientist: The Responsible Conduct in Research. The reason this came out was because there were a number of incidents in which people either published things wrongly, or they would publish in the newspaper, and then when it went up for journal review, it turned out they were not wrong. At this point we were asking how much money we should be putting in to the research enterprise. Of course, people said that they did not want to put it into that kind of thing. Congress got involved and investigated it. The U.S. scientific enterprise was sick. We don’t hear that so much now. I don’t know if that is because we are richer now, or if people do not care now or what. I expect it to come back. Most of the problems that arise are human issues. How do the values of science get understood, particularly in practice, in the face of the inevitable conflicts of value. There are things that we have that are precious that are put upon us. The main thing is that it brings the irrationalities. Research is supposed to avoid irrationalities! Therefore, it is sort of ironic that this is the problem, but that is again because we are humans and we just have to accept that.

Normally, in chemical engineer research programs, this is not a problem. But, I will also comment that you and I have both professional and civic responsibilities to deal with such issues, even if people close to us are not involved.

What are the conflicts?

• One is the personal interest—financial involvement, confidential knowledge and so forth. Everyone signs a form that says that they do not have any conflict in research.
• Publications and openness: There are false claims of discovery, commercial proprietary secrets, multiple publication of the same work, many short papers

If a faculty member is coming up for a promotion in tenure, one of the things that is easy to do is to count publications. This is dreadful. You don’t count things and decide if it is good enough or not. So, there is a tendency of a lot of professors under pressure to write a lot of short papers. It is not the right thing to do, but that is what happens.

We have to avoid that. There is often a tendency to think, "well, I did that," and ignore what someone else might have had as an influence. You want to give credit because you might be on the other side sometime.

There are errors in negligence in standards of quality and sloppiness. We have to rush through if we have a two-year grant. Students are under the pressure of writing out the results so that they can get into the papers or if there is a meeting coming and they have to make a presentation. What does that breed? It often breeds sloppiness. Then the reading can get bad—you can have misconduct and deception, fabricate data, falsify the results, plagiarism, cover-ups, malicious allegations, and due process violations. These are all listed in the publication.

So, what do we do about these kinds of problems? I happen to live by this Murphy’s Law:

Do not ascribe to maliciousness what can be ascribed to incompetence, ignorance, and insensibility. While, sometimes, bad things are intended, mostly they are not.

You tend to be more patient and forgiving about what might be happening, but that does not mean that you don’t confront the problems that are coming from this, it just simply says, "what are the motivations for those?" So, what are we going to do? First of all we have to be aware of what we are getting into in this business. It is going to happen. We have to have our own sense of values established and our priorities. We have to know what is most important to us so that when someone asks us what we think about something, we actually have an answer. Only the prepared mind and spirit can stand the pressures that can arise.

A quote from Alan Weinburg, who is an eminent physicist:

What that means is that he accepts the idea that he has to follow with the rules, that he wants to get into the spirit of how research is done. If you don’t do that, you can be all those things and it isn’t important to someone like Weinberg.

We cannot tolerate, much less support, sub-standard conduct, especially unethical behavior.