Learning about Research as an undergraduate

On his blog DrugMonkey asked the question "Are University Professors Doing Their Job if Undergrads Do Not Know How Research Works?" Several people posted their experiences about what they learnt about research and the work of academic scientists  and I decided to add my bit -

I became an undergraduate in a leading UK Chemistry Department 55 years ago and perhaps the best way for describing what I learnt (and did not learn) about research is best illustrated by discussing a final year project which led to my going on to do a Ph.D. in this area of Chemistry.

The project was an investigation into a recently published paper on a chemical compound called benzimidazole. The author had carried out a theoretical calculation to predict its chemical properties and had quoted 6 experimental papers which "supported" his findings while ignoring perhaps 200 papers in the literature which failed to support his calculations. Even worse four of the quoted papers were not really relevant and another was followed by a correction which had been "overlooked". My project was to investigate the 6th reference.

What I learnt at this stage: OK the paper was so wrong it could only be explained by fraud - but it soon became clear (particularly when I started on my Ph.D.) that many theoretical papers quoted experimental results without understanding the experimental limitations. In addition many experimental papers measured the properties of the chemicals they had synthesized and quoted theoretical papers without understanding the theoretical assumptions or the bounds of the measurement techniques. The result was that I became very cautious of automatically accepting papers which didn't feel right. I also set myself very high standards of proof - which meant that quite a bit of interesting research has never been published because I was not satisfied that I have proved it to my own satisfaction.

What I didn't learn: I failed to understand motivations behind the scientific rat race for publication, promotion, prestige and funding. To become a successful "scientist" it is essential to put a strong positive spin on your research. It also helps to develop a strong social network- ideally including some of the "peers" that could be reviewing your papers or grant applications. (As someone who had been seriously bullied at school, my personal ambitions were low and my ability to develop effective social support networks was poor.)

 The probable reference was to a first page reference to a 200 page long paper in German. The 19th century chemist had pioneered the use of a nitrating mixture of very strong acid and typically had boiled chemicals in this mixture for an hour and then reported on the result - which (if I remember correctly) was about 20% of one product and 80% a tarry mess. I found that if you took a very much weaker acid, froze it in dry ice, and added the benzimidazole I had 100% of one product almost instaneously. This showed the theory was wrong - and also that the Victorian chemist has been repeatedly applying a technique without understanding its limitations.

What I learnt at this stage: Much Science is basically data collection and analysis of the results - followed by publication. The German had developed a technique and "turned the handle" and out had come publishable results without having to wonder whether there were more appropriate techniques. Of course such data collection can be extremely important - and modern scientific equipment linked to computers can make it possible to collect vast quantities of data and have it analysed automatically to show the relevant patterns. The result can be very exciting - for instance in redrafting the evolutionary tree using DNA. The trouble was (and still is) that many post-graduate researchers (and some professors) are really acting as scientific technicians - with little scope to learn to use their imagination. I decided that such a narrow view of science was not for me.

What I didn't learn: The politics behind interdisciplinary research could be very difficult because of the need to deal with people who academic credibility was based on a very deep knowledge of a very narrow topic.

Finally there was an additional unplanned spin-off from the project in that I was able to look at the underlying theory and suggest reasons why the mathematical model that was used in the paper was very susceptible to very minor variations in the original assumptions.

What I learnt: Never be afraid to question the assumptions underlying any particular area of science if you feel that they are unsound or incomplete.

What I later learnt: If you find what you believe are good reasons for questioning the assumptions you will be told that "Exceptional claims need exceptional evidence". The problem is that exceptional evidence needs exceptional funding to gather and to get such support you have to negotiate a way through the the establishment peer review framework.

What I failed to realise was that for the average research scientist the safest career option was to select a narrow field where it was possible to gather large quantities of new data by turning the handle of a suitable technique and get plenty to publish without having to use too much imagination. If you try to explore original and imaginative  ideas you either become very successful, but in most cases you will fall flat on your face and be trampled by those who put the academic rat race above scientific ideals.

Some of you may think that this is a rather cynical view of scientific research - but looking back over the last 50 years I am sure I would have been far better off in financial terms if I had not decided to dedicate my life to scientific research.