Tuesday, February 21, 2017

Who's responsible for flawed studies?

In his blog post about case of data-fabrication in which the (alleged) fabricator was not included as an author, Neuroskeptic notes that „Authorship means responsibility“. I agree!!

In this case, the grad student who collected and manipulated the data remained (so far) unnamed, because he is, rightly or not, not listed as an author. But while it should be clear that authors have to take responsibility for what is published under their name, I think that responsibility reaches further.

In any complex study naturally a lot of people are involved. Of course it’s not the fault of the person at Siemens who wrote the scanning sequence if we then go and use it incorrectly and – in my opinion – neither is it the fault of a supervisor if their PhD-student “cleverly” manipulates the data.(1) After all it’s not kindergarten; not everything can be checked.

But it should be (though I know that it not always is) by “surprise” if an aspect of a study turns out to be problematic after publication: You shouldn’t be able to know prior to publication that, say, the statistics are somewhat dubious; papers with fishy statistics shouldn’t be published. However, it might turn out later on that “consistent signals” may just be artifacts produced by the imaging techniques - and the fact that brains need a constant supply of blood. In such cases it would be useful to know which sequence was used and how – exactly - the resulting data were analyzed. To figure that out the people who chose the sequence or did the analyzes should be known – which in the current mess of unregistered studies is much more likely if they are (co-)authors. Of course they also deserve to be on the paper if they had (substantial) work with the study but for them to become important at some point it doesn’t really matter how “scientific” their work was. The purely technical aspects can be just as important: How long did it take before blood was frozen, how (exactly) where the patients immobilized in the scanner, which arm was the blood pressure apparatus on, did the optic cables reliable transfer the signal and how was that made sure, …? One might say, that all these kind of information should be part of the publication but often it isn’t.

Another point is which responsibilities people involved in research have beyond their specific duties. While I generally think that the responsibility doesn’t extent (much) above ones area of expertise, I do think that everyone does have the responsibility not to knowingly do or support unethical stuff – and any unjustified or flawed study is unethical in my opinion. In this sense funding agencies share a great deal of responsibility and, once known, the media are also responsible for reporting scientific misconduct. However, of course it can be difficult to judge which studies are unethical or flawed.


(1) If the data are manipulated “uncleverly” I do however think that the supervisor has some responsibility because he or she should be able to “see” that.

(Unfortunately English is still not my native language. I'm sorry about that!)

Sunday, February 05, 2017

A (very) critical "review" of CBT: activity planning

I have been doing a cognitive behavioral therapy (CBT) program for depression. In yesterdays post I described the relaxation classes we had to do as part of the program. Today I’d like to talk about the “activity planning” group.

The group was based on the notion that actions, feelings and thoughts influence each other. Therefore – we were told – we just had to get doing (pleasurable!) stuff again in order to feel better and have more pleasant thoughts. We were handed a list of over 200 potentially pleasurable activities and had to do one such activity alone and one together with other people for each class. Each such activity furthermore had to be rated on a 1 to 10 rating scale in the dimensions “mood”, “drive” and “ease of mind” before, during and after the respective activity.

While I understand that it is important to get doing stuff – cos otherwise with absence of or low drive you’d just lie in bed and eventually die of thirst (or lack thereof) – I find that it is asked a bit much, that those activities should be perceived as pleasurable. For me, it just made me feel even more like a loser, because I couldn’t get any pleasure out of going for a walk, even though I tried. (“But it’s nice out here… I just have to tell myself… It’s nice… Sun and wind and…. But it sucks so much, why do I have to do this… I don’t have the energy, I’ll never arrive at the other end of the street…. And how do I know if it’s nice, the world is f*cked pretty much. It’s NOT nice. Climate change, and plastic in water, air pollution…. Why do I have to walk here, it’s so exhausting…. And the food we get here is wrapped all in plastic…. Which pollutes the seas….. But the river is NICE…. I’ve to tell myself…… But I don’t want to lie, not to others and not to me…… it’s not nice…. But maybe liars are the better people, at least they make others happy and aren’t such failures….. I hate this.” And so on….).

But what’s more important than that potentially nice activities didn’t seem to work for me is how to group was lead: It was instructed by whichever nurse had duty that day and they seemed to have very different opinions about what constitutes a pleasurable activity (independent of the list). While some accepted everything others had very specific ideas and would even get mad at you if you did an activity which they didn’t find pleasurable, for example tidying up or learning something (as an activity alone) or skyping and chatting or tweeting (as an activity with others). In their mind you had to get a massage or take a bath or color in something as a nice activity alone and an activity with others had to involve more than virtual contact (going to the cinema or a cafe would do). Now, I don’t find tidying up that pleasurable either, but at least something’s done afterwards, i.e. there’s less mess. If I’d get a massage I wouldn’t enjoy it and my place would be just as messy afterwards, so therefore nothing would be gained. Probably I’d feel guilty for spending the money on something I didn’t even enjoy, and like a failure for not enjoying something which should be enjoyed. Therefore I prefer tidying up – it has the better consequences. However, as I said, not everybody shares that opinion and while of course everybody is entitled to theirs I find it unhelpful decry the activities of people who tried their best to do (and enjoy...) them anyways. But of course not every nurse was like that, some didn’t really give a shit (so that the group ended within 5 minutes after everyone had recited their ratings) and others were trying to be helpful and encouraging (e.g. by suggesting in which other situations the said activity could be helpful or how the activity itself could be improved or what could be done instead).

Overall, is it helpful having to be doing a “pleasant” activity? Personally, I don’t think so, because it’s just not pleasurable and not feeling anything nice made me feel bad. *Just* doing some stuff, without any aspiration to finding it pleasurable *might* be helpful: Not eating or not drinking (water ;) ) doesn’t either, nor does not showering or not brushing teeth etc. But, in my opinion, it can be unhelpful to hope for any pleasure doing these things. Because if that’s the point and it’s too far away you might as well just not do it.

Saturday, February 04, 2017

A (very) critical "review" of CBT: Progressive Muscle Relaxation

I did an inpatient CBT program for depression (as a patient) and I’d just like to express my opinion about it (because in a lot of ways it sucked … and apparently I’m not well enough to fill out their feedback form, so…). Of course it’s just that, just my personal opinion and just based on personal experiences at one time and at one place. Other people will have different experiences.

Whatever…

The program consisted of relaxation, activity planning, a “pleasure/enjoyment”-group, music (and/or art) an “Info-Group” and of course the CBT itself (in a group and individually with a therapist).

In the relaxation group we did Progressive Muscle Relaxation according to Jacobsen (PMR). There you have to tense a specific muscle (like you hand, your upper or lower arm, your foot, etc.) for a few seconds and then relax it according to instructions. Then you should focus on the difference between the tension and the relaxation as well as the difference between (say) the left and the right hand. This relaxation technique is popular in Germany (I don’t know about elsewhere) however it does not work for me. (I tend to tension and relax my hands/fingers a lot anyways, especially when I'm not well, but I don't do it according to PMR guidelines; so therefore during the lessons I had to (try to) stop that.)  That’s not their fault, obviously, however I had to do the relaxation class during the entire time of my stay there, even though it had – if any – the opposing effect on me. Most of the time the instructor read the instructions from a piece of paper, but occasionally they just played a tape. Still, the fact that you had to go there may be helpful for people for whom the PMR works but who don’t have the motivation to do it on their own, when they are not forced to. (I find it hard to find any empirical evidence that it works for depression, though.) 

However, I think they vastly overestimate the amount of people from whom it works, because they did not accept any kind of negative feedback. At the end of each lesson there was a feedback round where you had to say how you are now and most people stated that they felt more relaxed. However, if you stated that you weren’t relaxed you still had to make up something which supposedly was better (which reminds me of research). My room-mate suggested I’d just say I’d me more relaxed to have my peace (which REALLY reminds me of research).

However, the one thing that was helpful about the PMR for me was having the appointment (where I had to go) itself; it structured the day a bit. But over all, I had preferred if I hadn’t had to go there.



I may or may not describe the other parts as well. 
Don’t know yet.

Wednesday, January 18, 2017

My dreams 2016


Hope you don't get nightmares...

1.

I have to do an MRI experiment, of course. For that purpose I have to poke my participants with needels, lots of them, cause it’s fun to investigate suffering. But – oh shit, I think - the needels are magnetic. They’ll slice my poor participant into thousand pieces while being scanned. I lean into the MRI-scanner to my screaming participant and try everything to grap as many needels as many needels as I can reach but this hurts my participant even more. 
Versions of this dream include me trying to convince my colleagues that they can’t put the participant in the scanner prior to him being sliced up or added fun with two participants simultaneously. 

2.

My supervisor wants me to build an inside fantasy world on top of the research building. With parks and waterfalls and stuff. And of course an airport, cos s/he is busy so how else should s/he get there. I keep thinking that that’s impossible, an airplane can‘t land inside the top floor of a building. Furthermore the groundplan of that planned top floor is much greater then that of the building itself. I mention that, but my supervisor just says that if I’m too dump for that I shall just ask the experts. Unfortunately I’m having a hard time finding experts on top story fantasy word building. However, to my surprise one day the the site is finished and it looks good. However, I know that it isn’t, I know that it’ll turn into a disaster and of course I’m right. The plane flights through a tunnel, and which is connected to that top floor to land on the airport and first it looks like it might just work out but then there’s a wind or something, the plane bounces from one site of the tunnel to the other and everything falls into pieces. 

Versions of that dream include a lot of stress because the human recouses depart has disabled my card to enter the building and especially the top floor or me going for a check in the fantasy dream world before my supervisor visits and discovering plies of dead (park-) workers in a sideway. 

3.

I try to get somewhere. Home, to work, where ever. But of course I don’t manage to get there. 

I may take a plane, which flies way too low between the buildings of a random city. The plan has a steering wheel and somehow, I’m the one who has to control it and try to avoid hitting any buildings. But it doesn’t react to what I do. Furthermore I have to run back and forth between the uncontrollable steering wheel and the back of the plane where people want something from me, but I don’t understand what, so they get angry and everyone screams at me. It’s windy, too. Sometimes strong winds hit the plane from street canyons, and the damn steering wheel doesn’t react. Eventually the plane crashes, with its nose into the asphalt of a street. People are not pleased with that.
Or, I may try to take a train (maybe a good idea after all these crashed planes, one would think). But of course, it doesn’t work out. Purchasing a ticket can be a complicated matter, especially in (my) dreams. The ticket machine may calculate a ridiculously long journey with several overnight stays at scary rail stations for an actually short distance. Or the ticket machine explodes when I try to purchase my ticket. Or, what could be a nice dream but isn’t, the ticket machine gives me back way too much money. I don’t know what to do with it i.e. how to handle the situation and get the money I falsely received back to its rightful owner(s). I can’t leave it there, because then obviously someone will take it. I can’t take it myself because I don’t have enough pockets and I’m clumsy and drop it everywhere. While I try to pick one coin up again I lose 10 others. People chase me for the money. I forget the rest in the ticket machine. My trains don’t arrive and the prediction on which track they’ll arrive changes constantly. After some chasing over some tracks I decide just to wait on the track I’m at till my train arrives there. (Good plan, no? :D). But police is after me for the money I just wanted to bring to the lost-properties office. (Another great plan, right? …. That reminds me of a real situation in Munich where I found a 50 Euro note in a train (S-Bahn) station on the moving stairs. I was so confused by that, that I turned to the person next to me and asked him if it was his…..). I get arrested, my attempts to explain myself fail.

Alternatively I have a ticket, but I need to bring my little sister with me and like always the announced tracks change constantly. We chase through station hall after station hall without getting anywhere. I realize we need to get underground but the underground part of the station is all flooded. It’s supposed to be like that, it’s just me and apparently my sister who can’t breathe under water. Not only that, but there’s also an underwater obstacle course which we have to solve in order to get somewhere. Unfortunately I fail and my little sister drowns.

I spare you the versions of that dream in which I accidentally kill my grandmother, my father or other random people.

Sleep well.

Wednesday, May 11, 2016

Bullshitting.

A few weeks ago Richard Morey wrote a blog post[1] about how undergraduate students in psychology learn to bullshit (i.a.) by criticizing peer-reviewed papers (which are difficult to criticize because they shouldn’t contain any severe and easily detectable errors anymore). 

When I was studying we didn’t have to write any essays or to criticize any studies. So maybe I didn’t learn about the art of bullshitting well enough. However, I believe that the problem that bullshitting is rewarded is even greater than that: Guessing an answer in an exam and bullshitting a bit around is almost always better than stating that you don’t know. 

My first MSc-exam was in Clinical psychology. I think, I had learned really well[2], but as this was an important – and also basically[3] my first oral – exam, I was very nervous. The first question the professor asked me was: "Why is the prevalence of major depression increasing?" And I didn’t know! I didn’t even know it was increasing! But if s/he asked that, that must be, right? So I mentally went through all the text-books I’d worked through, searching for the part where they explained *why* more people today are depressed then in the past. I mean, it could be anything, or nothing. More divorces, different foods, more or different industries, different expectations or stressors, different lifestyles? Vaccinations? Exams?[4] But my professor hadn’t asked for some hypotheses or my opinion but *why* that is. And I didn’t know! So I told him/her so, while I kept searching mentally in the text-books and all my notes. His/her next question was *if* I *knew* the theories for depression. As I was still desperately trying to come up with a better answer than "I don’t know" for his/her first question, I just said “yes”, because, well, I knew. At this point, I think s/he possibly felt pranked, because apparently s/he wanted me to name (or explain) those theories not be told that I knew them. Upon clarification I of course explained the theories to her/him, but I had real trouble understanding what s/he wanted from me throughout the whole exam.
It occurred later, much later, to me that s/he probably wanted to ask a "nice" first question whereby my task had been to bullshit something together, despite not knowing. Who knows *if* and *why* the prevalence truly is increasing and *if so* compared to when? 

I don’t blame that professor for my inability to understand my task correctly, so please don’t take me wrong. Of course I could have said: I don’t know… for this and that reason… one would need to know this and that to answer this question. Certainly that would have been much better, but I thought she wanted an answer to the question and not some justification why I don’t have it. 

A lot of people will say "Yeah, but you have to think beyond the facts and combine all the knowledge you have. Everybody can learn some facts so there’s no value in echoing some books without thinking about it by yourself!" And it's true, it's certainly an advantage to be able to think even during – or especially during – exams. But then at least frame the question like it. (E.g. "I'm interested in your opinion/thoughts. Let’s assume the prevalence of major depression was increasing. What could be possible factors that may – or may not – contribute to that? And how could one test if those factors actually do contribute to an increase in the prevalence? What would possible problems in testing those hypotheses be?") Otherwise it invites to bullshitting.
 
But then, on the other hand, bullshiting seems to be an essential skill in many aspects of life… maybe that is why they select by that….


[Sorry if that post is to bullshitty itself, obviously it’s just my opinion (and experience) and no knowledge… Also sorry for language-errors.]




[1]  Link: http://bayesfactor.blogspot.de/2016/04/how-to-train-undergraduate.html 
[2] Or at least A LOT! Probably much more than what was good for me… 
[3] Apart from a tiny part of my final school exams. But that was just a tiny part and therefore not nearly as important.
[4] Not really.

Sunday, April 17, 2016

"I should sell tickets to everyone who wants to see my patient"...

A few weeks ago a colleague of mine, who is a psychotherapist, told my colleagues and me about a patient of his, because he needed some help with questionnaires and other things. 

He had to treat this patient because of our supervisors wish. However, he personally thought that his patient was totally crazy proceded to explain why. Due to his explanations some other people wanted to see the patient, and he joked he should sell tickets and get rich by showing the patient to others. When he left he mentioned that it’s just not possible to have empathy with someone like that, because he can’t put himself in the patient’s position. Everyone agreed.

While this may be the most insulting (that’s my opinion, obviously not the opinion of others) thing I’ve ever heard here about a patient, it happens quite regularly that people are making fun of or laughing about patients (not everyone! But some people.). Mostly (again, my opinion) for really minor things like talking too much or misunderstandings. I’ve always felt really uncomfortable with that.

However, what irritates me is that he mentioned that he just can’t have empathy with the patient because of the patient’s behavior. That may be. I can’t understand the patients behavior either, but my understanding was, that that is (in part) why it is a disorder and not just perfectly reasonable and understandable human behavior?

Sometimes I get the feeling that psychologist as well as doctors just "understand" situations (behaviors, thoughts, …) that seems to some degree reasonable to them, and, at best, perfectly fit the diagnostic criteria for a disorder.

I don’t understand any better. But I don’t understand the slander either.

What is your opinion on this?

Thursday, January 28, 2016

What should you *ask* at an interview for a PhD position? (Part 1)

There are a lot of advice books out there which tell you what to answer to all those mean questions that you might be asked at a job interview.

They might be more or less useful. However, they never tell you what to *ask* in order to find out if a PhD-position is or isn’t a good one. (At least I haven’t found any book that tells that. And I read a lot of these books before applying to my PhD-position because relying on my social skills would have been foolish.)[1]

But don’t worry, I’ll tell you ;).

Of course this list is subjective, but maybe it can give some ideas about what could possibly be relevant to someone who is looking for a PhD-position. Because at the time I didn’t ask any of these things. I was just glad they took me... :/

This first part is related to questions about the project you are applying for... further parts will be related to the supervisor/professor and the group you are (potentially) going to work with and the working environment.[2]

Most often when you apply for a PhD-position in a specific project a short description of the project is given in the job-advertisement. But this description is generally very short and doesn't tell much at all what it's really about.

Therefore I think the supervisor or whoever does the interview should introduce you more to the topic, the background theories and how the specific question is planned to be answered.

With how I don't mean answers like "with fMRI" or "with TMS" (what most likely the job-description already tells), but a more specific explanation of the planned study design, so that you can (try to) judge if that is realistic to accomplish or not. For that purpose it would be useful to get a copy of the project-proposal. It would spare the supervisor time and you could assess by yourself if you think it's realistic. Yes, in theory any funded research project has already been through a selection process during which, among other things, the feasibility of the research idea should have been evaluated. However, that is the theory. It seems to me, that in practice no one in these selection committees does some basic reality checks. Therefore you’ve to do it by yourself. [3]However, a lot of researchers might not like to give their proposal out because they might fear that you steal their brilliant ideas. That is fine[4], but in this case I still think they should give enough background information to make judgements possible.

Of course it is possible that the study design is not yet developed (though I think at least a rough study design should be part of the proposal). I think that that should be mentioned as it involves more uncertainty then.[5]

So whether or not you get the research proposal, what are the critical points to check?

First, I think it is important information which population should be examined. Patients with rare disorders might be difficult or impossible to recruit for participation in a demanding experiment.[6] That does not mean that such research should not be carried out. I doubt that difficulty in recruitment is related to the importance of the research, but it should be taken into account: You might end up with a very long recruitment phase and/or a very small sample size. So be prepared to conduct an underpowered study...

The other important factor is how many participants should take part in the study in total in comparison to the measurement time per participant and the available lab/MRI time. In general, the longer the experiment the lower the change to get enough time to do the measurements. Also take into account that one day has 24h available hours at maximum.

As next, I would consider number of methods to be learned. It takes time to learn to analyze an fMRI design, but there might be much more you have to learn. You might have to do additional MRI-recordings and analysis.[7] Furthermore you may have to or want to run additional recordings in parallel to the MRI (like skin conductance, heart rate, breathing, or even EEG) or an additional part of the study might be done with EEG, TMS, eye-tracking, virtual reality or something like that. It might be exciting to learn a lot of methods; but it takes time. Therefore I think it is an important point to take into consideration. A finished PhD with just one or two methods is probably much more worth than an unfinished PhD with 10 half-learned methods. Of course a number of factors might influence how hard it is to learn various methods. I’ll speak about that in a later post.

It might also be important to ask if the project is already running and if so at which stage it is. I'm not saying it's good or bad if the project is already running, but it might be important information. You likely won't be involved in the experimental design then, but on the other hand data (to publish!) might be available faster (and: it's running!). The important question then is, though, to whom the data "belong", i.e. who is allowed to publish with them? This is a question that can lead to huge fights in some groups.

Speaking about already running experiments, I’d furthermore say that it is "dangerous" to start with any new method that has not been applied by anyone in the department/institute before. Even if it should work in theory (it surely should, otherwise no one would use it, right?) it might be that the hard- or software used at the institute is not compatible to what you (or the research proposal) plan. If you can check the compatibility, do so.

Also, whether you got the proposal to read or not, try to read background studies on the subject. If you got the proposal, you can find them by yourself, if you didn’t get it I think you should ask for them. The professor might tell you, that it is sufficient if you read them once you got the position (which may sound nice), but I think that is kind of too late.

Though it's still possible to terminate a PhD later (i.e. once you have enough insight into the project) it's harder because of the time you've invested.[8]
So try to find out if the background studies are sound[9], especially if it are background studies from the same department.[10] And if you have to do a direct or indirect replication of work done previously by your supervisor, run away.[11]



P.S.: Sorry for my english. I deleted about half of my text (bc too long), so I hope it makes still some sense...
And for the next post I'll complain about which handy feature the English language is missing.

P.S. 2: I really hope this is not misunderstood as advice on how to get an "easy" PhD. It's not meant to be! 



[1] [Though, reading those books was already a good start into wasting a lot of time since I was not asked any of those weird questions during the job interviews (for what I should be glad). It seems social skills are not a selection criteria*. Obviously those questions might also be pretty unvalid and unreliable, especially if more people are like me and just learn the “correct” answer from a book, but considering all the unreliability of research I doubt that that is why they are not used to determine who’s a good candidate. *They might be success criteria though. More on that later.]
[2] [This distinction is a bit artificial, but I didn't want to write an even longer post than I already wrote. Especially since not everything applies to everybody.]
[3] [Although it would of course be better if a selection committee of experts did that and not a student who hasn’t even started a PhD-training yet.]
[4] [I guess in some areas it's more realistic than in others.]
[5] [If in need for a list what can go wrong, contact me ;) ]
[6]  [For animal-studies it might be that some animals are difficult to get. For example mice with a specific gene modification. But I don’t work in animal-research, so I don’t know much about that.]
[7] [Like, for example, morphometry, DTI (diffusion tensor imaging), functional connectivity (e.g. PPI), resting state analyses (e.g. ICA), MR spectroscopy, ASL (arterial spin labeling), or another fancy (new) method.]
[8] [Time-investment sound way too nice... I don't want to scare anyone away from a PhD. Just choose the project carefully. Otherwise it might be more than just time.]
[9] [or not less so than an “average” study]
[10] [Which would not be surprising because it makes sense to build new work on previous work, but it can lead to a lot of trouble when they are... not good.]
[11] [I think more replications should be done, because every study can have positive results just by chance. Together with a publication bias towards positive findings, this means that it is hard to know which findings to trust. However, I think that replications should be independent. Or preregistered with signed contracts that any result will get published from all parties involved.]