Tuesday, December 12, 2017

Tales from academic publishing: pseudo-science edition

It looks as if reading putatively scholarly journals can make you stupid, given the high and increasing rate of junk being published, even by 'legit' publishers, as Adam Marcus and Ivan Oransky (co-founders of the essential Retraction Watch) show. 
 [e]ven the big, top-tier houses fall victim. Springer and the Institute of Electrical and Electronic Engineers, two leading publishers, in 2014 retracted more than 120 articles that had appeared in conference proceedings after learning that they had been written not by scientists but by a convincing computer text generator called SCIgen.13 The program—a sort of industrialized version of the Sokal hoax14—allows anyone to create a “scientific paper” by simply providing author names. The resulting text and graphics look like a proper scientific paper, but are gibberish. The fact that any were published means that no one peer reviewed the manuscripts.
What happens when, retrospectively, journals try to prune out the junk?
Although one might assume that journals would hold a strong hand when it comes to ridding themselves of bogus papers, that’s not always the case. In 2011, Elsevier’s Applied Mathematics Letters retracted a paper by Granville Sewell of the University of Texas, El Paso, that questioned the validity of the second law of thermodynamics—a curious position for an article in a mathematics journal, but not so curious for someone like Sewell, who apparently favors intelligent design theories over Darwinian natural selection.15 
The journal’s editor, Ervin Rodin, blamed the appearance of the paper on “hastiness” and acknowledged that the article had no place in the publication. “Please accept our apologies for our erroneous judgement in even considering this paper,” Rodin replied to a critic of the Sewell article, which was eventually retracted.16
The affair ought to have ended there. But Sewell sued and Elsevier, the world’s largest scholarly publisher, blinked. Not only did it pick up the tab for Sewell’s legal fees—a $10,000 hit—but it took the unusual step of apologizing to him (although it did not order the journal to reinstate the article).17 The article was retracted, according to the notice, “because the Editor-in-Chief subsequently concluded that the content was more philosophical than mathematical and, as such, not appropriate for a technical mathematics journal such as Applied Mathematics Letters.”18
Beyond the financial remuneration, the real value of the settlement for Sewell was the ability to say—with a straight face—that the paper was not retracted because it was wrong. Such stamps of approval are, in fact, why some of those who engage in pseudoscience want their work to appear in peer-reviewed journals.

Wednesday, December 6, 2017

Another thing that probably doesn't exist: IAT validity

For some time now, we all have come to know that we harbor unconscious racist attitudes, which can be measured by the Implicit Attitude Test (IAT). But it turns out that there is little reason to believe that this test is a valid measure of anything, and a lot of reason to disbelieve it. Olivia Goldhill delves into this in Quartz:
There are various psychological tests purporting to measure implicit bias; the IAT is by far the most widely used. When social psychologists Banaji (now at Harvard University) and Anthony Greenwald of the University of Washington first made the test public almost 20 years ago, the accompanying press release described it as revealing “the roots of” unconscious prejudice in 90-95% of people. It has been promoted as such in the years since then, most vigorously by “Project Implicit,” a nonprofit based at Harvard University and founded by the creators of the test, along with University of Virginia social psychologist Brian Nosek. Project Implicit’s stated aim is to “educate the public about hidden biases”; some 17 million implicit bias tests had been taken online by October 2015, courtesy of the nonprofit.
There are more than a dozen versions of the IAT, each designed to evaluate unconscious social attitudes towards a particular characteristic, such as weight, age, gender, sexual orientation, or race. They work by measuring how quick you are to associate certain words with certain groups.
The test that has received the most attention, both within and outside psychology, is the black-white race IAT. It asks you to sort various items: Good words (e.g. appealing, excellent, joyful), bad words (e.g. poison, horrible), African-American faces, and European-American faces. In one stage (the order of these stages varies with each test), words flash by onscreen, and you have to identify them as “good” or “bad” as quickly as possible, by pressing “i” on the keyboard for good words and “e” for bad words. In another stage, faces appear, one at a time, and you have to identify them as African American or European American by pressing “i” or “e,” respectively.
The slower you are and the more mistakes you make when asked to categorize African-American faces and good words using the same key, the higher your level of anti-black implicit bias—according to the test.
OK, so far, so good. My guess is most of us have taken a version of this online.But many institutions are using this test to structure anti-impicit-bias training.
HR departments quickly picked up the theory, and implicit-bias workshops are now relied on by companies hoping to create more egalitarian workplaces. Google, Facebook, and other Silicon Valley giants proudly crow about their implicit-bias trainings. The results are underwhelming, at best. Facebook has made just incremental improvements in diversity; Google insists it’s trying but can’t show real results; and Pinterest found that unconscious bias training simply didn’t make a difference. Implicit bias workshops certainly didn’t influence the behavior of then-Google employee James Damore, who complained about the training days and wrote a scientifically ill-informed rant arguing that his female colleagues were biologically less capable of working at the company.
Silicon Valley companies aren’t the only ones working on their “implicit bias” problem. Police forces, The New York Times, countless private companies, US public school districts, and universities such as Harvard have also turned to implicit-bias training to address institutional inequality.
The problems with this? Two, actually. One, there is no evidence that these tests measure anything, and second, the workshops and training don't seem to be effective in reducing bias. Regarding the latter:
The latest scientific research suggests there’s a very good reason why these well-meaning workshops have been so utterly ineffectual. A 2017 meta-analysis that looked at 494 previous studies (currently under peer review and not yet published in a journal) from several researchers, including Nosek, found that reducing implicit bias did not affect behavior. “Our findings suggest that changes in measured implicit bias are possible, but those changes do not necessarily translate into changes in explicit bias or behavior,” wrote the psychologists.
What about the tests themselves?
In recent years, a series of studies have led to significant concerns about the IAT’s reliability and validity. These findings, raising basic scientific questions about what the test actually does, can explain why trainings based on the IAT have failed to change discriminatory behavior.
First, reliability: In psychology, a test has strong “test-retest reliability” when a user can retake it and get a roughly similar score. Perfect reliability is scored as a 1, and defined as when a group of people repeatedly take the same test and their scores are always ranked in the exact same order. It’s a tough ask. A psychological test is considered strong if it has a test-retest reliability of at least 0.7, and preferably over 0.8.
Current studies have found the race IAT to have a test-retest reliability score of 0.44, while the IAT overall is around 0.5 (pdf); even the high end of that range is considered “unacceptable” in psychology. It means users get wildly different scores whenever they retake the test.
Part (though not all) of these variations can be attributed to the “practice effect”: it’s easy to improve your score once you know how the test works. Psychologists typically counter the influence of “practice effects” by giving participants trial sessions before monitoring their scores, but this doesn’t help the IAT. Scores often continue to fluctuate after multiple sessions, and such a persistent practice effect is a serious concern. “For other aspects of psychology if you have a test that’s not replicated at 0.7, 0.8, you just don’t use it,” says Machery.
The second major concern is the IAT’s “validity,” a measure of how effective a test is at gauging what it aims to test. Validity is firmly established by showing that test results can predict related behaviors, and the creators of the IAT have long insisted their test can predict discriminatory behavior. This point is absolutely crucial: after all, if a test claiming to expose unconscious prejudice does not correlate with evidence of prejudice, there’s little reason to take it seriously.
Bottom line: while we might all be infected with unconscious racist bias, the evidence does not support the claim that the IAT measures it, or that workshops structured around the IAT help mitigate it.


Wednesday, November 29, 2017

Truth, Lies and Bullshit: Notes from Trump's Kakistocracy

Trump's kakistocracy does not merely attack honesty and justice, but arms itself with bullshit bombs so powerful that it assaults the very concept of truth. Greg Sargent puts this well:
To date, Trump has made over 1,600 false or misleading claims as president. Routinely, the lies are demonstrably false, often laughably so. But this actually serves his ends. It is impossible to disentangle this from his constant effort to undermine the news media, seen again in today’s NBC tweet. In many cases the attacks on the media are outlandishly ridiculous, dating back to the tone-setting assertion that the media deliberately diminished his inaugural crowd sizes, even though the evidence was decisive to the contrary. Here again, the absurdity is the whole point: In both the volume and outsize defiance of his lies, Trump is asserting the power to declare the irrelevance of verifiable, contradictory facts, and with them, the legitimate institutional role of the free press, which at its best brings us within striking distance of the truth.
Referencing the odious James O'Keefe, he reflects:
Trump is not responsible for O’Keefe’s antics, but they are fellow travelers. Margaret Sullivan, summing up the mindset they are both trying to achieve in their followers, quotes Hannah Arendt: “If everybody always lies to you, the consequence is not that you believe the lies, but rather that nobo.dy believes anything any longer.” Others with similar missions have gravitated to Trump. Brian Beutler points to former Trump adviser Stephen K. Bannon’s deep admiration for history’s most successful wielders of the power of disinformation as agitprop.
Trumpers are liars, yes, but more than that, they are bullshitters. As such, as Frankfurt warned a generation ago, they pose more dangers than traditional liars. Liars are careless with truth; bullshitters don't care about it. They don't reject a truth in favor of a convenient falsehood, as liars do; they reject the relevance of truth.

Tuesday, November 28, 2017

Medcrave spams again


 This just came in from yet another Medcrave fake academic publication.


Dear xxxx,

Hope you are doing well.

I am delighted to inform you that
Arts & Humanities Open Access Journal were planning to release Volume 1 Issue 3 by end of November. Hence we require one article to accomplish the issue.

Moreover i am having few more days in hand to accomplish the task, thus I have chosen some renowned people like you to support us for release the upcoming issue in time. Accordingly support us by submitting your 2 Page Opinion/Mini Review for publication.

Hope you understand my concern and your kind attention in this regard highly esteemed.

Await your optimistic response and apologies in advance for disturbing your busy schedule. 

Best Regards,
Alarice Warren
Arts & Humanities Open Access Journal

PS*: In addition, we are accepting the thesis as eBooks and also videos towards publication in AHOAJ to full fill our journal in all aspects.
 
In the past few months, I had been spammed by the Medcrave Sociology International Journal, trolling for papers, and even inviting me to join its editorial board (umm---no thanks). Should I feel slighted that the google-invisible Alarice Warren failed to invite me to her editorial board, and is only trolling me for a paper submission ---before the end of November? like two days from now?

Nah.

Monday, November 27, 2017

News from fake academic publishing: Open Access Text edition

This email, from a Beall listed "OAText" enterprise sends an important message, so I will just quote it in full:
Dear xxxxxxxxxxxx,
Greetings...!!
We contacted you earlier with the concern matter. As we have not received any response, we are taking the liberty to contact you again
We are glad to inform that, Research and Review Insights is considering December month as thank giving Month, for the authors, researchers, practitioners, students etc. for their endless efforts in research field and we selected year ending month is the prestigious month to thank them all.
Thus, as a part of Thanksgiving month, journal is announcing no publication fee for the articles which are submitted on or before December 15th. So, I would like to request you to submit your work (Only Research/Review/Minireviews) before the due date to avail the benefit.
You can send your valuable work to this mail or to editor.rri@oatext.com.
Kindly share your opinion regarding the same.
Regards,
Katie Tia
Managing Editor

Research and Review Insights
Any questions?

Tuesday, November 21, 2017

How to turn a journal into junk

I am not familiar with this journal, so I don't know what its quality was, but if the changes to its direction under new ownership are as alleged, I'm pretty sure that its quality will take a dive into junk:
For much of its 22-year existence, few outside the corner of science devoted to toxic chemicals paid much attention to the International Journal of Occupational and Environmental Health.
But now, a feud has erupted over the small academic publication, as its editorial board — the scientists who advise the journal’s direction and handle article submissions — has accused the journal’s new owner of suppressing a paper and promoting “corporate interests over independent science in the public interest.”
More is at stake than just the journal’s direction.
IJOEH is best known for exposing so-called “product defense science” — industry-linked studies that defend the safety of products made by their funders. At a time when the Trump administration is advancing policies and nominees sympathetic to the chemical industry, the journal seems to be veering in the same direction.
“There are many scientists who work for corporations who are honest scientists,” said David Michaels, the former head of the federal Occupational Safety and Health Administration under President Obama. “What we’re concerned about here is the ‘mercenary science’ … that’s published purely to influence regulation or litigation, and doesn’t contribute to public health.”
“I think the IJOEH articles were threatening to that whole industry,” said Michaels, now an environmental and occupational health professor at George Washington University. While Michaels has never served on the journal’s editorial board, he has published an article in the journal and peer-reviewed others.
The journal was one of the relatively few places that provided an outlet for “scientists whose work is independent of the corporations that manufacture chemicals,” he said. “The silencing of that voice would be a real loss to the field.”
Last Thursday, the journal’s 22-member editorial board, along with eight former board members and the journal’s founding editor-in-chief, wrote a letter to the National Library of Medicine requesting disciplinary action against the academic journal’s new publisher, Taylor & Francis Group. In particular, they asked the Library of Medicine to rescind the journal’s listing in the Medline index, which could drastically reduce its scientific influence.
In other words, the editorial board is alleging that under Taylor and Francis control, the journal is no longer to be considered reputable. Specifically, they claim:
Taylor & Francis has done the following since taking over:
  • Selected a new editor-in-chief, Andrew Maier, without consulting the editorial board. Board members said it’s “highly unlikely” that they would have approved of Maier. Their letter said he had a tendency to reach scientific conclusions “highly sympathetic to parties with an economic interest in favorable outcomes,” which is at odds with the journal’s mission.
  • Withdrew a peer-reviewed article by the journal’s former editor-in-chief David Egilman that criticized Union Carbide Corporation’s efforts to oppose workers’ claims of asbestos exposure. “Suppression of an accepted paper is a direct assault on academic freedom,” the board members wrote to the Library of Medicine.
  • Flagged three additional studies approved for publication under Egilman as “raising potential concerns,” according to a May 8 email the publisher sent to the board.
Not all that is junk starts out that way; rapacious publishers can take something of value and spin it into junk.
Read the whole article here. 

Tuesday, November 14, 2017

More from Medcrave

 I couldn't resist sharing the whole of this adorable email from Sociology International Journal.
Hope you are doing well.

Apologize if I am disturbing you with my email.

I am pleased to inform you that Sociology International Journal is planning to release the Volume 1 Issue 4 by 25th of November and we are in need of two articles to accomplish this issue. But I am afraid as I am now on 14th November. Hardly I am having few more days in hand to accomplish the task.
 
Hence I have chosen some illustrious people like you to support us for release the upcoming issue. So will you please help us by submitting a Research/ Review/2 Page Opinion/Mini-Review/Case Report for publication.
 
Your single article sustains us a lot and impacts my ranking in end of this month.

Await your precious submission.
 
Best Regards,
Levi Martin
 
Somehow, I doubt that "Levi Martin" is the real name of the author of this solicitation from a fake academic journal spawned by Medcrave. When you check out the editorial board, however, you find that at least some of the names listed appear to belong to accomplished academics. I don't know whether these folks are really attached to this "journal" or whether their identities have been hijacked.

Monday, November 6, 2017

Mass shootings and the legitimacy of the state

Not that it matters in the larger scheme of things, whether I contribute anything to the sporadic conversations triggered by a mass shooting, suspended in a few days in the wake of other events, and picked up again after the next shooting, not that it matters in a world that sees entire families wiped out in seconds and offers prayers in response, blind to the realization that prayers provide ineffective protection against bullets, that these families were in fact praying at the very moment of their destruction, not that any of this matters.

But I was thinking that the advice and conclusions of some government officials (Paxton of Texas, of course and others) that citizens need to be armed and prepared to protect themselves and others with these arms against gun wielding attackers undercuts the very rationale of the state they lead. States, after all, and the governments they institute, gain and retain their justification for limitations of individual free range of action by creating and maintaining zones of relative safety in public and private spaces for citizens to work, play, love and live their lives, reasonably secure in the persons and property. You don't have to be a social contract theorist to hold a view that sees this as an essential function of a legitimate state.

For Paxton and his cohort to now devolve that responsibility to individuals is for them to surrender one strand of the argument legitimating the power of the state. Each mass shooting, each official offering thoughts and prayers and no policy proposals to use the law to lessen the carnage, each person, school and church pushed to resort to private arms and security services, is a sign that the state has surrendered its legitimate authority and one of its main rationales for existence.