I spent a long time this afternoon thinking about the model upon which the set of experiments I have been doing for too many months is based. And I have come to the conclusion that the model is pants.
Which on the one hand is a shame, because I feel that I've wasted half the past year. On the other hand it does explain why the experiments are not showing what we expected — that is of course why we do experiments, to test models. The thing is that I can not make the model yield the result that was expected of it, even theoretically. That is, even if the model is correct it can not give the answer we were looking for. The best I can do is make half of the pre-messenger RNA transcripts behave and the remainder have ruddy great lengths of intron in them.
That is somewhat annoying, especially seeing as the effect I have been trying to reproduce was published a few years ago and appears to be accepted as true. That published work explains (but does not justify) why I had not previously assessed the model so critically.
I have looked at the original papers again and again, and the effect is a very tiny one. If I saw such a band on one of my gels I would be tempted to pass it off as artefactual. I had been warned that the group that published this effect had trouble seeing it, but had not realized that it was not the difference between "good enough to convince the postdoc" and "good enough for publication" causing them trouble, but "good enough to convince the postdoc" and "not bloody there at all". Which is where I've been for six months.
The one figure that supports the (intrinsically flawed) model is poor. It is a thin horizontal slice through a gel of a PCR reaction, barely wide enough to display the bands between 300 and 700 base pairs that supposedly tell us there is an effect. There are no appropriate controls. The difference between 'best' and 'zero' is less than ten percent. I can not reproducibly get even that. And given that the model is mechanistically flawed anyway I have to say, myth busted hypothesis disproven.
Perhaps I can now concentrate on finding out what this protein really does. I have another model, and all I have to do is think of an appropriate experiment. And that, my friends, is not as easy as it sounds.
Comments
Oh, it seems like it can be a good thing even if I do understand the frustration about spending half a year doing things not working... On the other hand if oyu hadn't done the experiments, how would you have known?
What really bothers me though is if you can't tell anyone else about that not working... so other people won't be trying for themselves. All this would go into the "Journal of Tried it and sorry it does not work" that I been looking to find for a while.
Good look with protein and the experiment! I spent a good year trying to find binding to a protein I discovered and really, a smart plan helps.
(I was just a phd student... and the smart plan was more like looking in a heystack for a needle.)
Posted by: challenge | November 23, 2006 04:47 AM
Shades of my PhD. But I had an additional layer of politics to deal with. I will omit the names to protect the innocent, but a simple PubMed search will reveal who the cast are.
I had two PhD supervisors. One was a Scottish Italian (yes, from Glasgow, and yes, something to do with ice cream) and the other was an ex-Cambridge Dunn labs Professor who was head of the division in the Scottish Office institute I was working in.
Back in the day, my supervisor had a particularly well-known biologist working with him who is famed for her vigorous and prolific research. She is also quite a formidable character. During that time, the pair of them were getting very excited about uncoupling protein (UCP) in brown adipose tissue. For anyone who remembers, this was supposedly the great white hope for burning excess triglycerides and making America thin again (Britain was not that fat in the 70’s). They did a range of experiments, but the defining method of determining UCP activity is its ability to bind radio-labelled GDP. Sure, you can do expression experiments once you have provided strong enough links to allow surrogate Westerns, but you have to start with GDP.
The pair found a ton of ways to activate UCP, the most remarkable of which was cold exposure (put mouse in fridge for 4 hours). However, after messing around with UCP for a while, everyone began to realise that this was a peptide exclusively found in a tissue that only exists in any meaningful amounts in rodents, babies and hibernating animals. No anti-fat drug here, please move along and repeat all of the above with leptin in 20 years.
The result, from my perspective, was that UCP was scratching around for a role other than warming up sleepy hedgehogs. The said biologist hit on the idea of seeing what was going on in fever. Is UCP a major part of the thermogenesis needed to mount a febrile response? After a few GDP experiments in rats, it seemed that yes, UCP is activated in experimental fevers … in rats. No-one tested it in human babies … obviously.
Twenty years later, we were in Scotland with the Prof who had developed a cast iron method of doing chemiluminescent northern blots (cast iron in that they always, always worked as long as a certain lab tech did them using a non-variable combination of an iteratively designed protocol and plain, ordinary black magic). Of course, the lab what churning out UCP northerns like Fleet Street does titty pics. “Nige, can you do an entire PhD on febrile thermogenesis and UCP mRNA?”. There was my mission.
Of course, I had to start with the GDP experiments to see if I could repeat the 1980s/70s stuff. It took me more than 6 months of repeated radio-labelling experiments before I could convince anyone that A. UCP is NOT upregulated in fever, and B. rodent fever models are inherently shit and give wildly varying results if you don’t spend the money to set up a handling-free radio thermistor system. The subsequent northern blots simply confirmed exactly what we saw with LSC counts.
You can imagine my frustration with this as the Prof had the research of the biologist to hand and owing to the success he had seen her have with all other areas of her research, it may have been difficult for him to accept that this PhD scrote might be right. The funny thing is that at my viva voce, we were lucky enough to secure another senior leader in this field who said that he had pretty much the same result as me and “Would we like to collaborate on a paper?”
Posted by: Nige | November 24, 2006 02:03 AM