Monday, September 26, 2016

Something else that doesn't exist: power posing edition

Add power posing to your list of things that turn out not to be things. Who says? The co-author of the study which claimed it to be a thing in 2010:
Now, we can add another researcher to the skeptical camp — one who will be difficult for anyone to ignore: Carney, who is now at the Haas School of Business at the University of California, Berkeley. Late last night, she posted a document on her website publicly expressing, for what appears to be the first time, serious skepticism about power poses.
The key takeaway, which she underlines and bolds for emphasis: “I do not believe that “power pose” effects are real.” But she goes into some really interesting detail about how she came to that conclusion. She notes that while some of her skepticism stems from the recent replication attempts, there were also decisions she, Cuddy, and Yap made as researchers that she regrets in retrospect. For example, she writes that in the original study, one of the outcomes “of interest was risk-taking. We ran subjects in [chunks] and checked the effect along the way. It was something like 25 subjects run, then 10, then 7, then 5. Back then this did not seem like p-hacking. It seemed like saving money (assuming your effect size was big enough and p-value was the only issue).” Elsewhere, she notes that “The self-report [dependent variable about feelings of power] was p-hacked in that many different power questions and chosen were the ones that ‘worked.’”
Let’s translate: “P-hacking” is a prevalent but increasingly frowned-upon method of making one’s results appear sturdier than they are. To oversimplify, it involves taking a “throw everything at the wall and see what sticks” approach to data analysis, running all sorts of different tests and then only or primarily reporting the ones which come back as significant. Carney is admitting that the original power-posing experiments were conducted in a manner that allowed the researchers to, in effect, overclaim the significance of what they had found (Gelman and others had long suspected as such).
Carney also highlights other problems with the way the experiments were run: For one thing, she writes, too many of the people involved were aware of the hypothesis being tested. Generally speaking, this is a bad idea, since this awareness can influence how an experiment is conducted, the subtle feedback given to the participants, and so on. She also notes that when participants won in the gambling task, they were informed they had won, and this could have confounded the experiments’ results: “Thus, effects observed on testosterone as a function of expansive posture may have been due to the fact that more expansive postured subjects took the ‘risk’ and you can only ‘win’ if you take the risk. Therefore, this testosterone effect—if it is even to be believed—may merely be a winning effect, not an expansive posture effect.”
Carney concludes her document with what reads as a decisive statement against the claims Cuddy and others have made about power poses:
Where do I Stand on the Existence of “Power Poses”

1. I do not have any faith in the embodied effects of “power poses.” I do not think the effect is real.

2. I do not study the embodied effects of power poses.

3. I discourage others from studying power poses.

4. I do not teach power poses in my classes anymore.

5. I do not talk about power poses in the media and haven’t for over 5 years (well before skepticism set in)

6. I have on my website and my downloadable CV my skepticism about the effect and links to both the failed replication by Ranehill et al. and to Simmons & Simonsohn’s p-curve paper suggesting no effect. And this document.
So stop assuming that insufficient mastery of the wonder woman pose has blocked your path to power. Start looking at structural oppression instead.

No comments:

Post a Comment