SCEPCOP Forum

i have been doing some research on the subject of the much vaunted "peer review " in academia and other places - which has been made quite hard as a lot of dissertations on same are only available to those with either the appropriate "access" or the ability to pay - and like much scientific work with baring on subjects in which i am interested in connection with the paranormal ( for or against ) this has stymied me to a major extent

any ways that aside , we often hear from the skeptics /debunkers and others that their treatise on their standpoint or supposed authoritative "knowledge" has been peer reviewed , this often is an accusation used against we who write anything about the paranormal ,in that our work or thoughts have no merit because they are not "peer reviewed " - which of course is difficult as we have no "experts " in the field in the first place to judge the validity of said work

but even in accepted academia - do the "experts" always agree ?? - seemingly not

Take this assessment of peer review made by Ponsi (2003) as an example.

Despite its wide acceptance, peer review has been subjected to a variety
of criticisms: the evaluation procedures are often inadequately
performed, and in general it can be said that research on the peer
review process does not provide unquestionable evidence of its value.
This time-consuming and resource-intensive process is slow,
expensive, profligate of academic time, highly subjective, prone to bias,
easily abused, poor at detecting gross defects, and almost useless for
detecting fraud. (p. 444)
Ouch! We wonder what Ponsi really thinks about the process. But wait, there’s more. A
decade earlier, Bornstein (1990) decried the review process as “. . . unreliable,
unconstructive, and biased in a number of ways (e.g., biased against non significant
findings, against replications of previous work, against unknown authors and less
prestigious institutions, and against unpopular or counter intuitive findings)” (p. 672). ** emphasis added in bold by myself *

ref : http://scholar.lib.vt.edu/ejournals/JCT ... jewski.pdf

so if fellow "experts" are this critical of the concept , (much of which is done anonymously apparently ) - what value has it ??

now to move on to my main thrust of this piece , which is really one of WHY academia /science is SO bios against the whole idea of paranormal phenomena and the roll of the pseudoskeptic in academia

lets take a hypothetical example , say a student does a well researched and credible treatise on the subject , he sites various evidential examples , experiential examples ,and even anecdotal examples - comes up with a perfectly "valid " theory and HAS done his research - also citing counter arguments etc , but has come down on the side ( given the balance of probability ) of there being something worth pursuing , which is then submitted for peer review . and one or more of the "reviewers " are in fact PsKeptics , do any of you think that that piece of work has a cats chance in hell of getting a decent review ?? - noooo - of course it won't as bios has crept in , bios founded of personal belief - not merit of argument

so if this is the case in "academia" the yard stick of "education" - then what hope for ANYTHING the "ley" researcher tries to produce ?? - heck even amongst the paranormal press , it has been evidenced , i and others have written well argued pieces for "paranormal" magazines , only to have them rejected for being quote : "too negative " ( vis not "fluffy bunny " enough ) - when they where constructive ,but hard hitting reviews of the very reasons WHY the subject is not taken seriously other than by its own adherents

but going back to academia - ,i have no doubt that many ground breaking concepts and VALID research in to areas that don't conform to "accepted wisdom " have been stifled by the peer review process ( much like trying to find credible info on subjects on which to base thinking - see my point re pay to read above ) - so when our skeptical friends cite "peer reviewed " works as "evidence " with which to knock our beliefs , can we actually attribute much validity to those works ??

this i feel is a particularly valid question in the area of "psychology" and "brain sciences " , which some skeptics site as the Begin all and end all of our "delusions " - - however again if ONLY those works that accord with the reviewers own mindset/beliefs reach the publication stage - that's ALL they have to go on to knock us ( mind it helps if the papers say we are all crazy /deluded/mistaken/ paranoid / ya de ,ya de.ya de , as it suits the skeptics purpose ) - however what of any that say - actually there may just be something to what people are reporting ?? - we don't see them !! - because they are suppressed

i suppose at the end of the day the question has to be asked of teaching in its entirety and "academia" in particular - as who teaches the teachers ?? - other "teachers" and if students are not encouraged to research their subjects because "perceived wisdom" stymied them with this whole review process - open to personal abuse by those with a pseudo skeptical disposition - what value has the system ??

Quid custodiet ipsos custodes ( of knowledge ) - methinks

( as usual my apologies for the somewhat inelegant argument - but hopefully you get the general drift )

I've been involved with peer reviewing journal articles. Part of the process is reading the article before it is published and trying to clean up the article before it is distributed to everyone.

In one case I was very impressed with the work in its detail and analysis. The problem for me was that the paper never stated why the work was done. The goal of the paper was missing. Other peer reviewers pointed out that the paper never properly addressed its hypothesis or that some of the statistical analysis was good, but could be done in a better way.

In another case the work involved a mathematical reconstruction of a biological microstructure. I skipped over the biology and went for math to make sure it was properly written and that the steps were complete.

I have been discussing homeopathy in another forum. The other debater is posting articles. I am finding that many of the articles posted in lessor journals do not seem to have gone through the same rigor. A scientific article ought to describe the process in sufficient detail that someone else can attempt to replicate the steps or at least see where the steps differ from another experiment. List of materials are often good. It's this homeopathic treat from this source and prepared in these steps. That's good. That's an essential part of the process. What is bad is that these experiments require people. You sign up subjects (aka guinea pigs) and run the experiment. People have lives other than being test subjects. They drop out. Hardly any of the articles in these lessor journals report attrition rates. Some of the experiments are not double blinded. In fact, in some experiments the participants were allowed to choose the group they were to join and knew what they received. This would never have been published in a real journal.

Let's say for arguments sake that you do respectable work in the paranormal. Do you want to see your hard done work published next to some charlatan in it for the money? Peer review allows the scientific community to set standards for the quality of work.

When all is said and done remember that peer review is done by slave labor. I was asked to review some articles and I did. I was not paid for my efforts.

Peer review is not perfect, but it does improve the quality of the papers that are published.

some good points there ND - but hopefully you will have also taken my basic points on board - that of bios of the reviewer , oh and don't get me wrong here the biggest enemy of paranormal research is the way "evidence" and "findings" are presented , "anecdotal" or personal experience reports are as you and others have pointed out just that - can they be much else ??

- but in a field where there IS little serious research in the first place , mainly because most involved in it are there for either the "motive" ( as i have mentioned else where ) or what i term the "shit and giggles " , rather than any REAL effort to get to the bottom of supposed cases + WHO ?? is going to fairly review any works produced ?? - scientists ?? - but the majority don't believe in the basic premise ( that of the existence of paranormal phenomena ) in the first place , self appointed "experts" in the paranormal ?? - show me one , believers ?? ( too much bios ) . sceptics ( note spelling ) - too much bios ?? , so even if i was to write a proper paper on the subject - who WOULD fairly review it ??

also another problem IS that of repeatability , scientific endeavor requires that others can repeat experiments - but when one is dealing with one off and anomalous phenomena - HOW does one do this ?? , OK IF and big IF here someone COULD actually build a device that would lift the Vail and allow us to SEE ghosts on demand , then fair do's ( the holy grail of paranormal research ?? ) then it could be built by others and the experiments repeated , and if any thing the nearest we have gotten so far to even coming close is in the field of EVP , BUT even there there is little agreement as to any technique or standards , I have tried replicating the spiricom devices ( without much success ) as it seems from detailed reading of the research done that many require the input of "medisitic " types , so not repeatable by many - but then again HAVE picked up anomalous "voices" on standard video cameras - when there where only 2 persons present and both where identifiable - so again it leaves one with the problem of repeatability

but ( and this puzzles me somewhat ) science is ready to accept the premise of "quantum physics " ,which from my reading is mostly "theoretical" - as no one has yet produced many of the supposed particles mentioned , nor proven the existence of many of the concepts by sciences OWN standards of requirements of proof , oh sure the maths add up ( apparently according to those who can understand them ) - but again is that enough to PROVE the concept ?? , yet science on the basis of "theories" can seemingly command research budgets of billions ( the hadron collider is a good example ) to try to prove the theory - but if i where to try and get a research grant to try and prove the existence of ghosts?? - yea right - but i question who is chasing the bigger phantoms sometimes ?? ( but then as i have pointed out , i am just a ley researcher who lacks any "academic" credibility - and thus can be dismissed as just another crank "believer" ) - but hold hard , er surely the first people to "believe " in QT where derided by many in the scientific community ?? - and what IF the person who did so was not a "recognised" scientist or academic - would it ever have been given a hearing ?? - i suspect not - so you see there is a bios against any one not in the "academic or scientific " club - but does that mean necessarily they are WRONG ??

i always love to quote the case of the wright brothers , those "damned bicycle mechanics " as they where denounced by one( of many ) "experts" of the time who berated them for even daring to think heaver than air flight was possible - BUT as we all know - those "believers" proved everyone wrong - because they did not listen to perceived wisdom - but just kept going despite all the flack they got from the scientific community ( and i bet there where LOTS of SKEPTICS who reading reports of their efforts - came out with the same sort of reasoning that we get today )

anyways - this is going slightly off at a tangent - but hey why not ?? - as discussion is sometimes as "nebulous" as the concept being discussed