Sunday, August 9, 2009

What does the Science Really Say?




GOOD: You may hear or read about "evidence based" treatments or practices and the importance of planning and using approaches that have been scientifically tested. This is something that has been around for hundreds of years but more recently is being promoted at the national level to bridge the gap between science and practice in "the real world." With more attention to ASD and more public money dedicated to gathering scientific evidence, we are beginning to build a foundation for scientific evidence related to many dimensions of ASD such as what the likely causes are, the commonalities and differences among individuals with the diagnosis, and how that knowledge can inform our research on effective interventions.


BAD: You may hear in the news about children who were "cured" from their Autism. You may see similar claims being made by individuals with advanced degrees and from institutions with lofty names. You may hear from a friend, family member or professional swearing by such a treatment and from lofty sounding "institutes" about the wonderful things they are doing for children with ASD. You may read that X treatment was found to be effective compared to all other treatments. But you may not want to tell the provider from Y, who disagrees.

Ugly: The truth isn't all that ugly. It is just ugly that the truth is having such a difficult time getting the representation it deserves in the public domain.

The frequently suggested "finding" that there is a well established large body of evidence supporting one treatment as effective for children with ASD is not consistent with the body of evidence. There is also no body of evidence to support the notion that a treatment exists that will restore a child to "normal" while ensuring against lifelong dependency and institutionalization.

I don't know what it is going to take to reign in the misleading claims and propaganda other than to provide "the pieces left out". The body of scientific evidence we need to rely on for "the bottom line" should be based on consensus of the independent scientific community. Producers of a treatment technology or investigators affiliated with a specific treatment may disseminate large volumes of publications, reports etc. extolling the virtues of and superiority of their treatment, accompanies by reams of scientific data, testimonials and endorsements. If they consistently arrive at conclusions that are inconsistent with the independent consensus, results, regardless of how well referenced, should be interpreted with caution and healthy skepticism.

There are "Gold Standards" and there are lesser standards. The lower the evidence threshold is set, the less powerful and valid the conclusion. The more rigorous methods require more resources and tend to lead to more conservative results. Quantity does not equal quality. Independent reviewers know how to judge the merits each study and to also look at them when combined to see if "the whole is greater than the sum of it's parts. Depending on how each was designed and carried out, having a bunch versus a few may or may not make a difference in terms of strength of the data and the conclusions that can be made from it.

No comments:

Post a Comment