Saturday, November 8, 2008

Comment on a comment --publication of data that doesn't fit



Okay, I really don't blog much about science here. But in response to what I had thought of as a light-hearted, amusing post (albeit one that also sums up some very frustrating things about my new workplace), ambivalent academic posted a very interesting comment.


The authors are to be commended...if science is about figuring it out (rather than making it up) as we go along then it's important to include stuff that doesn't make any sense when communicating finds...then someone else might see how it fits or changes the working model and *presto* get it figured out. Unfortunately that only works if the stuff that doesn't make sense ends up in the publication so other people can see it...but it won't get published with stuff that the authors can't explain...I think that this is a major flaw in the way we report our findings.

And I started to respond to this in the comments, but then it got so long that I just decided to throw it out as a post. It's my blog, I can do that. And it's a very interesting point.


The data that's confusing, that doesn't fit a paper's hypothesis, usually isn't published. No suprise--why would any author include data that contradicts or confuses the story she/he is trying to tell? Negative results usually also aren't published. That transgeneic mouse with no phenotype? Will probably languish unknown. But if the experiements were rigorous and carefully controlled, then even puzzling and negative data is valid data. And when that data is not communicated, it can be to the detriment of the whole scientific community, as researchers waste time and money heading down blind ends . . .


But by convention, the scientific paper isn't a "data dump." By convention, it's a place to tell a clean, coherent, succint, and hopefully compelling scientific story. As with any story, extraneous and confusing details only (well, usually only) detract.

This isn't to say that puzzling data that dosn't fit the main hypothesis/story line is never published. I have included the odd pieces myself in a paper--sometimes this is necessary, as there may be a major experiment which *must* be done, and so you must report on it even when you don't quite understand all the results. But if you do have odd, confusing results, you better damn well try to explain or at least address it in your text, instead of just throwing it in there and hoping no one notices.

Or worst, not understanding that it's confusing in the first place (as seems to have happened in the manuscript I mentioned in the original post).

It is an odd business. Nature is messy, science is messy, but we try to tie it all up in a neat package for the journals, crafting a condensed, clear storyline out of months or years of frustration, failed experiments, trial and error, and sometimes entirely serendipitous discovery. And then we reframe the whole thing to make it appear as though blind luck was really brilliant foresight all along.


I agree that there needs to be a mechanism to more effectively communicate those puzzling results and negative data that don't get published in the peer-reviewed journals. I don’t know what the answer is. I suspect (hope?) the answer will have something to do with the growth of online scientific communities and of increased sharing of raw data in online databases. . .


Thoughts, anyone?


The science philosopher is now off to sleep. . .

6 comments:

Ambivalent Academic said...

Thanks for continuing on these lines - I think it's an important topic. I also started to respond and it grew too long. Post will be up soon over at mine if you're interested.

Cath@VWXYNot? said...

There should be a Journal of Negative Results. Imagine how many CVs would be boosted.

The bean-mom said...

I'd be on the fast track now, with that Journal of Negative Results...

Ambivalent Academic said...

Bean-mom - thanks for your kind comments on my blog.

You're a science writer? This sounds really appealing. I will check out the AltSci blog and see what I can learn...would you mind if I emailed you re: questions about breaking into the field? Still trying to get a handle on other career options at this point.

Thanks again for your support!

Ambivalent Academic said...

Tag! You've been hit with the "5 things meme"!

I can't get the damn hyperlinks to work in comments so here's where to go to find it:

http://ambivalentacademic.blogspot.com/2008/11/5-things-meme.html

Sorry so ghetto

Mberenis said...

I like the idea of this, I have a similar blog about posting comments.

Post a Comment


:) Thanks in advance