Developing a teaching resource on peer review Understand article

Author(s): Ellen Raphael

Ellen Raphael from the charity Sense About Science explains why peer review is so important in science, and describes how an existing guide is being adapted to meet the needs of science teachers.

Image courtesy of Sense about
Science

Changes in the English national science curriculum will allow more time than ever before for exploring the inner workings of the scientific method (see Burden, 2007). How scientific ideas are presented, evaluated and disseminated is now a core requirement for key stage 4 students (ages 14-16). At Sense About Science, a charity to promote good science and evidence in public debates, we welcome this emphasis. Students need to discover early in their education what science is, if they are to handle data and evidence maturely and with discrimination, and gain their first insights into mastering a subject rather than being mastered by it.

This is important not just for understanding the scientific method but also for helping students to negotiate the world outside school. We need students to understand that the scientific knowledge we now regard as established fact – such as the Earth revolving around the Sun – is actually the result of many years of academic argument and gathering of evidence. In this way, students can be encouraged to consider new research critically, to consider its evidence base, and not simply to believe new theories because they appear to make sense. At the moment, there are few resources available for teachers to work out how scientific information is evaluated and added to the ongoing body of knowledge.

In 2005, Sense About Science published I don’t know what to believe…,a short guide to peer revieww1. It has been hugely popular with over 60,000 copies disseminated worldwide. The guide aims to help people understand the journey that scientific ideas undertake before they tentatively join the body of scientific knowledge and other researchers can begin to repeat them. The guide also highlights the difference between published and unpublished research, helping people to determine whether the research claims they read in newspapers or on the Internet come from scientific journals or from scientists who have chosen not to subject their work to the critical scrutiny of their peers.

Image courtesy of Sense about
Science

The guide was developed through workshops with a range of groups, including secondary-school teachers and their students. It was very helpful to hear the teachers’ and students’ perspectives on the guide from the beginning, to find out what information they found useful and what they regarded as superfluous. It also confirmed our belief that very few students know what peer review is, how it relates to science, and how it can help them to evaluate new scientific ideas and research claims.

The first time that most students learn about the system is when they study a science subject at university, which means that those who do not continue with science will probably never know the role that peer review plays in the advancement of science. Following the guide’s release, we received many requests for its use in the classroom. These requests came from primary- and secondary-school teachers, and others involved in science education.

We have always been keen to develop a resource specifically for schools on peer review and the acceptance of new scientific ideas. It was decided, therefore, that rather than providing schools only with I don’t know what to believe…, we would begin to build a full education resource, with a web resource centre, where teachers and students alike could access further information about peer review, scientific knowledge and evidence. The resource is now under development with a planned completion date in summer 2007.

There have been many arguments lately that the new science curriculum dumbs down science and can be more appropriately termed the ‘sociology of science’. There are also concerns that the curriculum does not require consideration of how science is regulated both internally and externally.

We hope that our resource will go some way towards illuminating the regulation of science by scientists through peer review. However, it is worth highlighting what the new curriculum does well. For the first time students are being encouraged to consider what scientific evidence is, where scientific ideas come from, and the impact that they might have on society. The curriculum allows us to look deeper into what makes science science.

Although we must fight to retain the practical aspects of science education and the teaching of a broad scientific knowledge base, discussion about the applications of science and the ethical and societal implications should not be seen as detrimental or as the ‘easy’ option. Providing future scientists and consumers with a true understanding of the scientific method, its theory and practice, and how ideas and evidence are generated, should produce individuals who are well able to hold their own and stand up for science, particularly in controversial areas where it is not easy to see the path ahead.

How to get involved

Sense About Science is recruiting science teachers to trial the resource. We want to ensure that it hits the mark in the same way as I don’t know what to believe… has. If you would like to be involved, we will email you the resource with questions to raise with your students. Although the resource is based on the English national curriculum, teachers across Europe are invited to get involved in the trial – peer review is, after all, an international process!

The resource will be freely available online when finished. If you are interested in being part of the review team, or in finding out more, contact Ellen Raphael: eraphael@senseaboutscience.org

What is peer review?

  • Science has a system for assessing the quality of research before it is published. This system is called peer review.
  • Peer review means that other scientific experts in the field (the authors’ peers) check research papers for validity, significance and originality – and for clarity.
  • Editors of scientific journals draw on a large pool of suitable experts to scrutinise papers before deciding whether to publish them.
  • Many of the research claims you read in newspapers and magazines, find on the Internet, or hear on television and the radio are not published in a peer-reviewed journal.
  • Some of this research may turn out to be good but much of it is flawed or incomplete. Many reported findings, such as claims about ‘wonder cures’ and ‘new dangers’, never amount to anything.

Peer review ensures that a research paper has been checked by other qualified scientists for mistakes and omissions, as well as to clarify what the findings show. It also means that the results are available to the wider scientific community, so that others in the field can try to replicate the findings, or use them, in conjunction with other work or results, to reach further conclusions.

Scientific research that has not been subjected to this form of review is of no help to anyone. Scientists cannot repeat or use it and society cannot base decisions about public safety – or individual health, for example – on work that has a high chance of being flawed. The need to clarify the status of scientific evidence is growing. In the UK, the government increasingly uses public consultations, inquiries and commissions to gather evidence for regulatory purposes and everyone involved needs to be clear about the measures of scientific plausibility.

So, no matter how exciting or compelling new scientific or medical research is, you must always ask, “Is it peer-reviewed? If not, why not?”

If it is peer-reviewed, you can look for more information on what other scientists say about it, the size and approach of the study, and whether it is part of a body of evidence pointing towards the same conclusions.

Edited extract from I don’t know what to believe…w1

Case study: mobile phones

One of the first claims that mobile-phone emissions are unsafe was made by Roger Coghill, a self-employed researcher who had previously argued that mobile phones cause headaches and memory loss. In 1998, Coghill said that the waves produced by mobile phones could damage the activity of lymphocytes in the body’s immune system. Coghill published these claims himself and released them to the media, rather than submitting them for peer review. Many other studies failed to show damage to the body’s immune system as a result of mobile-phone usage. Despite the lack of corroboration, Coghill’s claims were widely reported, and fuelled discussion about mobile-phone safety. Between 1998 and 2003, he was cited in 119 printed news publications in the UK, most of which made no reference to the lack of peer review of his research or to the fact that other, peer-reviewed research did not corroborate his hypothesis.

More recent large-scale peer-reviewed studies have also found no evidence that mobile phones cause harm, and Coghill’s hypothesis still remains unsubstantiated. Claims like Coghill’s, which have not been scrutinised by scientific experts through the system of peer review, cannot be validated, and therefore, no matter how widely they are reported, such claims remain, essentially, just an opinion.

References

Web References

  • w1 – I don’t know what to believe… is freely available from the Sense About Science website.
    • Hard copies can also can also be requested from this page. There is a small charge for sending print copies outside the UK, to cover postal costs.

Resources

Review

Even if the topic is not as exciting as the most recent discoveries in cutting-edge research, this article is relevant for secondary-school teachers and students in science education. In fact, the focus on peer review, in addition to the usual sequence of ‘observation, hypothesis, experiment, theory’, is necessary for a correct epistemological approach to what makes science science.

For these reasons, Ellen Raphael’s clear and concise article is suitable for teachers interested in updating their knowledge and for students motivated to deepen their study of the scientific method. Moreover, the invitation from Sense About Science to test the didactical materials (printed guide and related website) is also a good opportunity for teachers interested in the societal impact and perception of science. The full article could be used for classroom activities or read by students interested in the topic.

Giulia Realdon, Italy

License

CC-BY-NC-ND

Download

Download this article as a PDF