CSMMH has now answered some of their critic's opinions about CSICOP/CSMMH experiment with Natasha. But the answers sometimes are more embarrassing than the original problems...

CSMMH (i.e. Mr. Andrew Skolnick...) should make clear that it is not only "Natasha's believers" that have made criticism to their test. I am not a Natasha believer, nor should be so any true scientist. I just think the test was invalid. As far as I am concerned, Natasha may have no "paranormal" power at all. Only good research will tell.

Although the Discovery Channel documentary, The Girl with X-ray Eyes, has yet to be shown in the United States and Canada, its broadcast in Europe and Asia has already generated a lot of criticisms and protests about CSMMH-CSICOP's investigation of Natasha Demkina, the 17-year-old "medical psychic" from Saransk, Russia. Almost all of the criticism involves groundless accusations or mistakes and miscommunications that mostly were beyond the control of the investigators. Here are our answers to some of these charges:

Let's be positive and hope that by using the term "almost all of the criticism" CSMMH (i.e. Andrew Skolnick...) is still open to criticism of the very many flaws that this test had...

Accusations: The debunking investigators deliberately produced a documentary for the purpose of humiliating Natasha Demkina and destroying her brilliant international reputation as a gifted psychic.

Answer: Neither the Commission for Scientific Medicine and Mental Health nor CSICOP had anything to do with the production of the Discovery Channel documentary. The program was produced by Monica Garnsey of Shine, Ltd. in the United Kingdom. Neither Monica or Shine, Ltd. or Discovery Channel have any relationship with CSMMH or CSICOP, whose investigators designed and conducted the test.

Agreed! The documentary was as fair as could be. It is amazing to see that a media corporation can produce a higher quality work, and far less biased, than our top skeptic world organizations can.

The Commission and CSICOP first heard of Shine, Ltd. earlier this year when the producer/director contacted CSICOP to ask its help in conducting a scientific test of Natasha Demkina's claims. The Commission's and CSICOP's only role in the documentary was to design and conduct the test. They had no input, influence over, or involvement in any other part of the production. As Monica Garnsey stated in email sent to two of our critics:

The rest of the accusation is similarly false.

Accusations: The investigation had been set up with a view to ensuring that Natasha Demkina would fail it.

Answer: Contrary to the accusations, the test was set up to assure that anyone who truly can see abnormalities inside of people's bodies would easily pass.

Not so. The test can be said to have a rather strong bias towards Type One Error (i.e. Rejecting the hypothesis being tested even though it is true, and therefore accepting wrongly the null hypothesis that Natasha has no power). This is not a sin. As a matter of fact this is even a virtue. But the researchers need to acknowledge it, and not misreport it. The researchers failed then, and keeping failing even up to now (despite the fact that I fully explained it to Skolnick...), in understanding that any "vision" has its own internal logic, and has also its powers and limits. There is no way for us to know beforehand if the "special vision" that Natasha has (if true) will fare well in conditions like those of the test. Not until we run the test.

All aspects of the test -- including what score would warrant further study -- were agreed to by the program's producer/director and by Natasha Demkina and her agent five days before the day of the test.

These two parties, I guess, are unsophisticated in scientific matters (at least Natasha and helpers are), and could not know the unfavorable statistics involved in it. And the researchers did not bother to explain it either...

The test was set up to see whether Natasha's claimed psychic abilities warrant a more carefully controlled study. It was designed so that anyone with x-ray-like vision, as Natasha claims, should be able to pass by matching all seven medical conditions to the correct subjects. To give Natasha some latitude for errors, we lowered the required number of correct matches to only five.

Again: if the test was meant to look for x-ray-like vision, why not use a Geiger counter? (Natasha would be either emitting x-rays or receiving them). The researchers failed to take into account the available data about the limits of Natasha's vision! They simply did not do their homework.

What we asked Natasha to do was easier than what she does during her normal readings.

This is what we call "begging the question". Also, Skolnick is saying above that test conditions are easier than true-life conditions. I think it goes strongly against what is known both in parapsychology and in psychology as well...

For each reading, she examines a person from head to toe and describes all abnormalities she sees. In our test, Natasha didn't have to decide what medical problems are present -- we told her. We asked her to look for six different medical conditions that should be easy to detect with x-rays. We clearly explained what the conditions look like and described exactly where to look. It should have been easy for her compared to her usual readings.

Accusations: The debunking skeptics never bothered to find out what Natasha actually claims to be able to do.

Answer: The CSMMH-CSICOP test design was based on a variety of reports of Natasha's healings plus a report from the producer/director Monica Garnsey's of her interview with Natasha Garnsey on April 6, 2004, during which Natasha described how she does her readings and what she is able to see. The purpose of this interview and report was to help us design the test.

Accusations: Natasha never claimed she can see at a cellular level. The investigators are misrepresenting her claims.

Answer: When the producer/director Monica Garnsey questioned Natasha for information to help the investigators design their test, Natasha claimed she "can see right down to the cellular level if she concentrates," Monica reported.

Accusations: The test was invalid because it required Natasha to find healed medical conditions that were no longer causing health problems. Natasha only diagnoses current medical problems, and the investigators knew that.

Answer: Natasha's fame has grown in part from news reports of how she has correctly identified long-healed bone fractures, a removed kidney, implanted metal pins, and other medical conditions that are no-longer causing health problems. Furthermore, the test design and rules were presented to Natasha and her agent five days before the test. They didn't object to the test or indicate in any way that Natasha can only see current health problems. One of Natasha's supporters is the Russian journalist Igor Monichev, who reported how she convinced him of her abilities by seeing the well-healed evidence of a broken wrist that he fractured as a child 37 years ago.

I think I might have been the first one to adress this accusation. I have already withdrawn it. Indeed, Monichev's report makes it clear that this accusation is groundless. I only got to know Monichev's report when I saw the documentary.

Accusations: The experimenters didn't allow Natasha to examine the subjects while they are standing, as she usually does in her readings.

Answer: Natasha viewed the subjects standing up, from their front, back, and sides.

Accusations: One of the test persons had a metal plate covering part of his brain, and Natasha failed to identify this. But why should she? The man was not suffering from a medical condition. Whatever he had suffered had been treated, and Natasha did not claim to be a metal detector.

Answer: Natasha has on a number of widely reported occasions identified metal screws, plates, and other surgically implanted metal devices. In addition to failing to see the metal plate, she also failed to see that the subject's skull has a large hole, which had been made to remove a large brain tumor.

It seems to be true, but I think Skolnick shoud give some feedback about a complaint from Natasha's agent, Will Stewart. He said, in an email, that he had made it clear to Monica before the test that sometimes metal implants may be hard to see (for Natasha). If Monica confirms this, then either she failed to report it correctly to the researchers, or she did report it but they failed to probe deeper into that. Skolnick has to contact Monica again. Do your homerwork, please. By the way, Monica was the producer/director of the program. She had thousands of things to do, thousands of things to worry about. That is one more reason for the researchers to have tried to get a direct contact with Natasha, for example, having her on a telephone line together with a translator. It is so easy to do it! Why didn't they do it?

Accusations: The investigators raised the bar so that Natasha would fail. The odds of Natasha getting four correct matches out of seven was 1 in 50 (or 2 percent). Those are odds that are statistically significant and widely regarded in science as passing.

Answer: Those odds are not considered statistically significant for testing extremely unlikely events.*

But Skolnick is failing to realize one very important thing: what determines the cutoff value is not the "subject's claim" (in this case, Natasha), but instead the "researcher's claim". For example, Targ and Puthoff had papers published on Nature about Uri Geller even though they did not find evidence for his metal bending. That is because metal bending was not the claim of Targ and Puthoff. It was the claim of Uri Geller. So what is the claim of the researchers (Hyman/Wiseman/Skolnick) in their preliminary evaluation of Natasha? Their claim is clearly stated in the protocols: further studies are warranted! That is what they would be claiming if Natasha had got 5 or 7 hits. That is astronomically less extraordinary than claiming that "Natasha has indeed X-Ray-like vision". So one has to take that in consideration when making bayesian decision to set the cutoff value!

More importantly, the 1-in-50 odds of getting four out of seven correct matches would be for someone blindly guessing. But Natasha was not blindly guessing. She had access to a great number of normal sensory clues that could have helped increase her number of correct matches.

Skolnick is again engaged in half-truths here. He is concluding, based on faith (or worse...), that the flaws of the test could only help Natasha. Now, picture yourselves in the place of Natasha: the researchers, right at the beginning of the test, tried to exclude your mother. The researchers presented to you two conditions that seemed clearly alien to the test rules. Now, you see an old man among the subjects. Is he the one that has metal staples in his chest, or is he only a decoy? What would you answer?

We were not able to design a truly blind test. We had wanted to test Natasha with subjects behind a screen of opaque cloth -- to blind her to normal sensory clues. However, we were told that, for unknown reasons, Natasha can see through cloth when it is worn by a person but not when the cloth is hanging in front of the subject.

And here Skolnick admits that he came to know some of Natasha's limits (the limits of her special vision) and he simply did not probe deeper into that. That is why I say that they did not try to understand the logic of the claim. And therefore ended up designing a flawed test. Now, they are desperately engaged in face saving...

Therefore, any test we could devise had to allow Natasha the opportunity to study her subjects and pick up clues about their health problems through normal senses. While we could not prevent this, we could try to reduce the problem by choosing subjects who were of similar age and appearance.

Unfortunately, we did not have the time to recruit ideal subjects for the test; the subjects we were able to recruit were not demographically similar and several of their differences may have provided important clues. For example, the subject who had had open heart surgery was the oldest and was male, and the majority of patients who undergo open heart surgery are elderly men. The subject who had none of the target conditions was the youngest and looked athletic and in good health. Indeed, no one who knows anything about hip transplants would have picked him as the subject with an artificial hip!

Interesting... I am a biologist, with M.A. in clinical bacteriolgy, with many relatives and friends working as doctors and nurses, and in no way I would ever guess this seemingly athletic guy had no artificial hip. Maybe 17-year-old girls from Russia have deeper background on those conditions... (By the way, hasn't Skolnick ever heard of the Paraolympics and of the para-athletes?)

In addition, several violations of the test protocols could also have helped Natasha increase her score. Contrary to the test rules, Natasha and her mother, sister, friend, and agent were able to watch at least two of the test subjects climb a flight of steps to enter the test building.

Will Stewart (Natasha's agent) claims that no one saw anything. And further, he says that they waited where they were instructed to. At least for the sake of ethics, Mr. Skolnick should adress these feedbacks from Stewart.

I hope Skolnick is not comparing his subjects to dumb horses... (he has complained so much about Victor Zammit having compared him to Ku Klux Klan activists). Again he is implying that the only possible reaction would be a give-away. But it could also be a concealing reaction.

There is no way the investigators could estimate how such factors may have improved

Improved or... Hindered!

Natasha's chances of getting four correct matches. But one thing is almost certain: it was very likely greater than 1 in 50 and statistically not significant.

"Almost certain" only if you are engaged in unscientific face saving. If you are engaged in honest scientific debate, your conclusion must be that the test was inconclusive.

Personally, I am very supportive of higher demands from "extraordinary claims", and that involves using bayesian statistics to analyze this kind of material. So I don't really reject the cutoff value used by the researchers (0.4%). As a matter of fact, even if Natasha had got all the conditions right, I still would consider the test inconclusive (i.e. in need of furthe studies). But the fact is that, so far, the researchers have not presented the bayesian interpretation of the data. So I would very much like to know what a good statistician would have to say about the cutoff value that they used (0.4%) in light of bayesian statistics... And I hope the researchers will tell the statistician that the conclusion to be drawn from Natasha's success in the test would merely be that "further studies are warranted"!

Yes, and since Discovery Channel decided not to include that line, CSICOP and CSMMH decided to jump onto the bandwagon and start claiming that they have closed the chapter on that issue, etc, violating that very test rule. So convenient.

Accusations: The CSMMH and CSICOP investigators have unethically used the media for propaganda purposes.

Answer: The CSMMH-CSICOP investigators were contacted by the documentary's director/producer Monica Garnsey, on behalf of Natasha Demkina, who wanted to be tested by skeptical scientists -- so the charge is clearly false. In addition, the researchers designed and conducted the test with careful consideration of all ethical considerations, whether in regard to Natasha. the volunteer subjects, documentary production company, or the general public.

The unethical use of the media happened after the test, with reports to the Guardian where Wiseman implies that Natasha could have cheated, and saying that a failure is a failure. Also, at CSMMH site there is plenty of unethical statements on this issue.

Accusations: The investigators had used their economic power over Natasha coercively, insidiously, unfairly, and inequitably.

Answer: CSMMH and CSICOP had absolutely no economic power over Natasha. They had no financial involvement either in testing Natasha Demkina or in the documentary program. Nor did they offer or provide Natasha any financial support or compensation. The Commission and CSICOP were invited by the producer to test Natasha's claims and in the interest of advancing science, public health, and public education, they agreed and did so. In a recent statement to critics, the program's producer/director wrote:

Agreed. Unfair charge.

I don't really agree with this accusation. Let's turn to the answer to it now:

Answer: What the test required of Natasha was much easier than what she normally does for her regular readings.

Again: there is no way to know if it was really easier or harder than her true-life readings. It is unscientific to claim so.

Instead of having to scan the entire subject from head to toe and describe all the abnormalities she sees, the test only required her to look at seven people and indicate which of them had six different specified medical conditions. These target conditions are easily seen on an X-ray or CT-scan.

Are these two conditions really easily seen in an X-ray, I mean, removed appendix and resected esophagus? It is hard to believe, but I will check that out.

She didn't have to figure out what is and is not normal. She was told exactly what to look for, what each condition looks like, and exactly where to look.

Actually she had to figure out what was normal or not. She was presented two men with no appendix (the metal plate guy also missed his appendix) but was told that only one had a missing appendix. If indeed Natasha has any power, it must have been quite confusing to her...

For example, one of the subjects had had a large brain tumor removed and a large metal plate now covers the hole in his skull. If Natasha's claims are true, then it should have taken her only a few minutes to look at the heads of the seven subjects to see the big hole and metal plate. This was a lot easier -- not more rigorous -- than what she normally does during her readings.

That is begging the question. Also, this guy seemed to be unblind. And had ultra skeptic attitude towards Natasha. And he (conveniently) forgot that he had had his appendix removed... Weird, isn't it?

Accusations: The experimenters were unfair in not giving Natasha a chance to rest after her 6000 mile flight from Russia. They required her to take the test even though she had no time to recover from jet lag or to adjust to the 8-hour time difference.

Answer: We were not in anyway involved in planning Natasha's travels. The timing of her travel was decided by Natasha, her family and supporters, and the program's producer. Natasha arrived in New York two days before the test. The day before the test, she gave six people full readings, which most described as very impressive. So both Natasha and her claimed abilities appeared to have been in good form the day before the test. And immediately after the test, Natasha went on to give individual readings, which strongly suggests that she was not overly taxed or tired.

In the documentary Natasha seems perfectly ok for the test. She was only anxious, very anxious. But at the moment she got to know about the appendix and the esophagus, she got clearly depressed.

We don't get to know her actual words in the documentary (translated in real time), only the narrator's general comments about her words. She definitely complained about those two conditions, and only about those two conditions. This is very important, but Skolnick keeps downplaying it. Also, Skolnick is being a hundred percent denotative in his interpretation of "I see everything". If he was a policeman, and he came across a man who claimed to have seen "everything" in the bank during the theft, he would surely start to ask about microbes and atoms and molecules in all corners of the bank, etc...

Accusations: Natasha got five of the matches correct, not four, and therefore passed the test. Natasha was told to find the subject who had a scar on her tummy from an appendix operation. Because she considered the scar on a woman's tummy from a gynecological operation more important, she pointed out that scar instead, but was not given any credit for that correct answer.

Answer: Natasha was not asked to find tummy scars or any scars to correctly identify the subject without an appendix. She was instructed to look for the subject who didn't have an appendix. She was shown pictures of an appendix. In addition, the test briefer held up his pinky and told her, "You should look for the appendix, which is about the size and shape of the person's pinky. And the way to find it is to look at the spot where the large intestine joins the small intestine. You'll find the pinky-shaped appendix right near there -- or you won't, if the appendix was removed." She was not asked to look for scars on the tummy or anywhere else.

It seems ok. But I believe that appendix sometimes are not the shape of a pinky, for sometimes they are collapsed (just as some parts of the intestines sometimes are). So they would be thinner then. Much thinner.

Accusations: Despite the fact that Natasha told the investigators at the beginning of the test that finding a missing appendix and a surgically shortened esophagus was outside of her abilities, the investigators insisted that she find them.

Answer: Natasha did not tell the investigators that finding missing organs or large parts of organs was outside of her abilities.

First, appendix is not exactly an organ, but rather an "appendix to an organ" (an appendix to the intestine). That is the reason why it is called "appendix". Second, there were no "large parts of organs" missed. Skolnick himself told Natasha that he did not know the size of the resected esophagus.

Natasha did offer explanations why she was having difficulty finding the subjects with these two conditions. She complained about "post-operative scar tissue" confusing her and she said that she thought the surgically removed appendix may have grown back, which of course, it cannot.

The scar tissue explanation was not plausible, because even if the subject with the missing appendix had post-operative scar tissue blocking her view from every angle, she still would have been able to identify him by eliminating the subjects who still had their appendix. In addition, the presence of post-operative scar tissue around the site of the appendix should have been an powerful clue that the subject had had an appendectomy!

Also, her excuses are self-contradictory: If she can see post-operative scar tissue blocking the appendix area, then she should not have had any trouble seeing the post-operative scar tissue around the surgically shortened esophagus -- the very clue that she was instructed to look for to find the resected esophagus!

Interesting answers. But, are they real? Skolnick told me repeatedly that Natasha claimed in the documentary to see at the molecular level, to see at the cellular level, to never commit a mistake. In the documentary, nowhere could I find her saying so. And nowhere could I find even a narrator saying so. So he'd better have all this above recorded in video, and make it available to us, for he is surely a man not to trust when his observations are concerned. In the documentary, it is indeed said that Natasha claimed that post-operative scars may be confusing (regarding the appendix), and that the esophagus may be different lengths in different people. But it is not sure if this is all that she said. What is certain is that those two conditions were the only ones that she complained about! It is something that must not be downplayed.

Accusations: Contrary to what the investigators led us to believe, Natasha never claimed she never makes a misdiagnosis.

Answer: A number of news reports have reported she never makes a misdiagnosis.

Hearsay!

In addition, the documentary shows Natasha's mother saying, "Natasha never makes a mistake. In six years, she hasn't made a single mistake."

Mother's say...

Accusations: The CSMMH and CSICOP investigators subjected Natasha to a highly stressful situation by changing the protocol and refusing to allow her mother to stay with Natasha during the test.

Answer: The investigators only changed protocols Natasha requested in order to make her as comfortable as possible. They introduced no changes to the test rules that she didn't request. This false accusation is based upon an incorrect statement in the Discovery Channel documentary.

At the beginning of the test, a couple of the investigators misread the rules and created confusion about who was allowed to stay in the test room.

To me, before I got to see the test, Skolnick told it was only one person that created the confusion (Joe Nickell). Now, he is admitting that it was two persons. Half of the personnel from CSICOP/CSMMH!!!

Accusations: The investigators also tried to change the rules to disrupt Natasha's concentration by substituting their own interpreter.

Answer: The investigators did not provide their own interpreter. There were two interpreters, one hired by the producer and her teenage friend. Natasha was allowed to use her friend as an interpreter thoughout the test.

Unfair charge.

Accusations: One of the investigators (Andrew Skolnick) misrepresented his credentials by claiming to be a "medical doctor" when he is not.

Answer: Andrew Skolnick never made such a claim. He is the executive director of the Commission for Scientific Medicine and Mental Health. He is a medical journalist, who worked nearly a decade as an associate editor at the Journal of the American Medical Association, but he is not a physician. That error was made by the documentary script writer.

The program's producer/director, Monica Garnsey, regrets the mistake and says it will be corrected in future broadcasts.

Unfair charge.

Accusations: The documentary clearly shows evidence of cheating by the investigators. At one point near the start of the test, when the camera is aimed at the man who had a metal plate in his head, you can see him hm raise his eyebrow, which shows that he could see through the supposedly opaque glasses and was not truly blinded.

Answer: The documentary shows CSICOP research fellow Joe Nickell preparing the sunglasses immediately prior to the test. They were made opaque simply by covering the lenses with duct tape. This allowed the subjects to be able to see to their sides and above and below them -- so they would not get disoriented and dizzy -- but not straight ahead.

The opaque glasses prevented them from seeing when Natasha was looking directly at them and, more importantly, it prevented Natasha and others from seeing their eye movements -- which might provide unintentional clues. The subject's eyebrow movement was more likely a reaction to something he heard than to what he saw going on in front of him, since it was impossible for him to see straight ahead.

Accusations: The skeptical investigators went after Natasha Demkina to discredit her as a phony.

Answer: The CSMMH and CSICOP investigators did not go after Natasha Demkina. Earlier in 2004, the investigators were contacted by the program's producer/director and were asked to scientifically test Natasha's claimed abilities. The investigators agreed.

I don't think they did. But I think they are terribly misreporting events, and that is serious enough. They are supposed to be respected academicals. They should live up to their credentials.

Accusations: The investigators took advantage of an underage girl by inviting her to come to New York for a test where they ambushed and tricked her.

Answer: CSMMH and CSICOP did not invite Ms. Demkina to come to New York. The investigators were invited to come to New York to test Ms. Demkina by the program's producer/director. Although 17 years of age, Ms. Demkina was accompanied and represented by her mother and an English journalist acting as her agent. The test, which was straightforward and fair, was agreed to by all parties prior to meeting in New York for the examination.

Same as above.

The Investigators' Most Noteworthy Mistake

There is one actual mistake in the test that few if any critics are complaining about.

Quite wrong. I have already complained about it!

Although the test design required no more than one subject to have any of the target medical conditions, the investigators discovered after the test was over that the man who had metal plate in his head following brain tumor surgery also had had an appendectomy.

That is: they did not check for it properly before...

He had forgotten to mention this

So convenient. Forgot that he had been opperated on...

when he had been recruited but brought it up after the test because a missing appendix had been one of the test target conditions.

The reason Natasha Demkina's supporters may not be complaining about this error is that it hurts rather than supports their case.

Only from your point of view, Mr. Skolnick, since you did not do your homework of asking Natasha what she could see and what she could not see.

It means that Natasha had twice the chance of correctly guessing which subject was missing his or her appendix -- roughly 28% rather than 14%. Despite this doubling of odds, she incorrectly chose a woman who still has her appendix.

Natasha also missed a golden opportunity to convince the researchers of her claimed abilities, and perhaps most other skeptics in the world. During the test, she had an opportunity to say, "Something is wrong here! There are two people who are missing their appendix. But you told me there's only supposed to be one." The investigators would have been embarrassed by the unintentional mistake. But they also would have been convinced.

Yes. She also could say something about cells and molecules, whatever you think are her actual claims... But, one has to live up to his/her actual claims, and only to the actual claims.

I don't think the researchers were responsible for all the problems in the test. I don't even think they had bad intentions. And I do agree that they are respected academicals, and academicals that deserve to be respected (that is, they are not only respected academicals, but respectable academicals too). But I don't think they have been acting accordingly in these last months after the test, regarding this Natasha issue. There is still a chance to revert this situation. All they have to do is to stick to the test rules. And to avoid counter-productive face-saving strategies!

Julio Siqueira
email: juliocbsiqueira2012 and then @ and then gmail.com