Do we need peer review on the 'Net? [CMAJ

Do we need peer review on the 'Net?

CMAJ 1998;158:1203

In cyberspace, all that glitters is definitely not gold. The American College of Gastroenterology discovered recently that a significant proportion of the sites it studied offered unproven therapies for treating gastrointestinal diseases. The remedies on offer included acupuncture, herbal medicines and nutritional supplements.

The review of 100 Web sites was undertaken by researchers from Case Western Reserve University in Cleveland, who looked for information on 4 major GI diseases: pancreatic cancer, duodenal ulcers, hepatitis C and irritable bowel syndrome.

They found that 10% of the sites proffered treatment advice that is not consistent with standard medical practice. In the case of duodenal ulcers, a condition for which satisfactory medical treatment exists, no sites included unproven therapies.

However, in areas where medicine has less to offer many sites touted unproven therapies for pancreatic cancer (12.5% of sites), hepatitis C (17.6%) and irritable bowel syndrome (23%). The researchers are worried that this represents a serious problem for physicians and patients alike.

The rigorous standards of peer review that support scientific publishing have yet to migrate to the Internet, and this makes sorting the science from the snake oil a real problem. Clinical epidemiologist Alejandro Jadad of McMaster University has studied how medical information is rated on the Internet and has found the system wanting.

A number of self-appointed rating systems already exist. Best of the Web or Top 5% are common stamps of approval that guide online searchers. Many of the systems offer judgements on health care-related sites, but Jadad and colleague Anna Gagliardi found that the criteria used by most of them are either inadequate or, in most cases, nonexistent.

Of the 47 rating instruments identified, Jadad found only 14 that published their criteria; of these, only 5 explained how the criteria were applied. Two included information about authorship and attribution, and 1 stated that the rating was reached from a consensus among reviewers; none said anything about formal evaluations.

Jadad's study of search aids available on the Internet quickly ruled out 33 of the 47 possible rating systems because they posted no criteria. Of the 14 that did, 8 appeared to be designed specifically to rate health information. However, even these contained little information about how their choices were made; disturbingly, 9 of them no longer provide access to their rating criteria. Jadad stresses that this does not mean all rating systems are poorly developed, but this lack of information simply makes it impossible for physicians or the public to tell. "My advice is to use [the Internet] with extreme caution," he said. "As with any other tool, it can be misused. The amount and variety of information is such that most consumers, and perhaps most health professionals, are bound to experience confusion, anxiety and frustration."

The naïve answer is to establish an official rating system based on standard criteria. However, Jadad points out that the 'Net's very nature makes this difficult, if not impossible.

Many Internet users object strongly to any "official" attempts to regulate information. Besides, the Web's interactive format means criteria used for paper-based journals may not be valid online. Jadad sees consumer education as the real solution.

Users, both lay and professional, must improve their ability to evaluate research evidence. As part of the Cochrane Collaboration, Jadad is working with researchers around the world to help develop ways to achieve this. In one study Jadad is looking at patterns of Internet use by cancer patients, work that is being supported by the Cancer Care Ontario Evidence-based Oncology Program (www.octrf.on.ca), HEALNet (hiru.mcmaster.ca/nce/overview.htm) and the Supportive Cancer Care Research Unit at the Hamilton Regional Cancer Centre (hiru.mcmaster.ca/sccru/).

"We are facing a sea change in health care," Jadad argues. "The Internet is creating new opportunities to improve decisions and communication in health care, but it can also generate many unprecedented problems. In the midst of an unparalleled information revolution, good communication, scholarly discussions and rigorous evaluations are more crucial than ever."

Failing to meet these challenges would be a tragedy, says Jadad, who considers the Internet a grand opportunity to improve health care. Not only is the 'Net an ideal tool for physicians and other professionals, he says, but it has already proven itself a boon for a new generation of patients who need support and information. He also believes that it has the potential to improve access to medical care.

He recently published some results of his work in JAMA (www.ama-assn.org/sci-pubs/journals/archive/jama/vol_279/no_8/rv71042a.htm). In that article, he says health care is facing an information and communication revolution.

"We can either miss an extraordinary opportunity to make health care more efficient and equitable, or move instead to a health care environment ruled by confusion, opinion, anxiety and unnecessary conflicts."—© Michael OReilly, moreilly@ cancom.net

Send a letter to the editor
Envoyez une lettre à la rédaction