mjbat7 t1_iv8rky6 wrote on November 6, 2022 at 3:58 AM

Reply to comment by Grisward in BRAIN SEROTONIN RELEASE IS REDUCED IN PATIENTS WITH DEPRESSION: A [11C]Cimbi-36 PET STUDY WITH A D-AMPHETAMINE CHALLENGE. - Biological Psychiatry by chilladipa

I used the term 'error bars' to describe the +/- values in the listed results. Frustratingly so far I haven't been able to access the original article in full text, so I can't be sure exactly what those 'error bars' represent. Presumably they're quoting the standard error in the samples, in which case, there appears to be a lot of error in their samples, which suggests that each estimate would have wide confidence intervals, which may overlap. Obviously overlapping confidence intervals don't necessarily mean that the difference isn't real, but if they overlap widely you'd certainly be quite skeptical of the results.

More specifically, let's look at some of the data in the abstract.

In the pre-treatment groups, the baseline BPND was 1.04 +/-0.31 vs 0.97 +/- 0.25. Is this a statistically significant difference? I don't know, but my guess would be that it isn't. Post exposure, the two groups' BPND was 0.87 +/- 0.24 vs 0.92 +/- 0.22. is this a statistically significant difference? Once again, I don't know, but my guess would be that it isn't.

If depressed people weren't measurably different from healthy controls in the baseline or post-treatment measures, then the fact that there is a within-group difference in the healthy controls but not in the depressed patients is much less convincing and seems more likely to be a statistical fluke.

Grisward t1_iv8ubsu wrote on November 6, 2022 at 4:24 AM

I get what you’re saying, and to be frank I appreciate the skepticism, and the drive to reality-check the numbers. Sometimes the numbers are over- or under-stated, or not reasonable effects.

In this case the abstract actually says it’s a significant change:

> “was significantly reduced in the HC group (1.04±0.31 vs 0.87±0.24 , p < 0.001)”

They include the P-value. You’re right it’s hard to know what the +/- means, typically that’s standard deviation, and not reflective of confidence intervals. Literally the confidence interval at 95% by definition would be smaller than the difference, that is if it used the same model used by the test.

It’s a whole thing about reporting and displaying confidence intervals, versus standard deviations, etc. Sometimes it’s straightforward to report standard deviation, but is not reflective of whatever statistical model was actually used. Frustrating, but not usually an author issue, also a journal guidance issue.

Oh, and the reason standard deviation does not indicate whether effect size is significant is in part that it doesn’t account for the number of individuals, nor the actual statistical model. Standard deviation is a simple university summary.