The Mold in Science’s Bathroom

About once a month I notice the black smudge in the corner of my shower—it’s mold that I keep telling myself I need to take care of. But 29 other days, I don’t see it. If the same smudge was in a hotel bathroom, it would probably be the first thing I noticed. Familiarity drapes invisibility cloaks on our daily lives including the rot we immediately notice when step out of the familiar.

My life is usually happy and normal but as I travel around, every once in a while I have this sudden jolt of “stepping out”. How strange is it that in the 21st century, in one of the most advanced countries, we take it for granted that if you end up in prison there is a decent chance that you will be raped? To the extent that we have a common urban legend of “pound me in the a$$” prison. That is, somehow we’ve normalized the idea of violent prisons as part of the social contract.

What else? How about the normal expectation of absolutely shitty schools in urban or economically disadvantaged areas?  Or, the recent events reemphasizing the banal normality of police brutality, guns in every fool’s hand, people without health care, and so on. I don’t mean we don’t recognize these things as problems, we do. And, some people dedicate their lives to solving these problems. But, for most of us, like the mold smudge in my bathroom, we are  horrified when we see them, but we live our daily lives without seeing them. Instead of being intolerable social conditions, which they should be for any advanced country, they are part of some intractable tangle we accept as part of normal life. Twenty nine days of invisibility is okay with me.

Things that should be abnormal cancers to a decent society have somehow become, dirty, but normal part of life. One thing that especially stands out to me is the rather uniquely American acceptance of the soft corruption of the rich and powerful. Admittance of children of rich donors, legacies from pre-historic colonial times, and other “holistic” policies for fancy schools are accepted part of society. So are foundations for the powerful that are really collection agencies; and, the idea of a White House full of friends and family advisors (this isn’t just a Trump thing) is okay. I mention these because those three things were the “crimes” under which the previous Korean president was impeached. Of course, all of these corruptions exist in other societies, but what is unique here is that we don’t really even think of these kinds of things as smudges to be noticed once in a while. Oh, studying my faculty handbook for some other committee work, I found that our university president has the power to unilaterally appoint anybody as tenured faculty. Very nice.

Actually, to back off a bit, the soft corruption I mentioned is not necessarily a bad thing. While it might be immediately unfair, there are long-term societal benefits to having flexible subjective college admissions, even trading admissions for donations. Allowing a university president, or the country’s president, to exercise subjective judgment can allow more effective governance. But, there is one big caveat. Having a social contract that allows a large degree of subjective judgement requires having transparent processes and for each actor to act in good faith with ethical principles. We are now seeing, with great astonishment, what happens when at the very top, principles of ethics and morality breakdown in an open society. We need the Pope to be holy and all of us to follow suit or the mold will find its warm niche.

What mold are we unseeing in the open society of science? One thing especially stands out. When I’ve tried to talk my collaborators into submitting to bioRxiv, they are hesitant. Some people who do submit, leave out information. In fact, some people never give talks on unpublished data. Why?They, we, all fear,

somebody might steal the idea/data.

We all have stories about when that happened to us or to friends. Just the other day a junior faculty came to my office about being scooped and how they will never again give a talk with unpublished data. Okay, why is stealing other’s work a thing? Why is “pound me in the a$$” science community a normal state of life?

I admit “stealing ideas” is an amorphous thing. A lot of times we do it inadvertently–some vague idea we had didn’t really click until we heard it from somebody else. What is an idea compared to actual work, anyway? Also, we all have source-amnesia. Well, sexual harassment used to be an amorphous thing too. Maybe she/he kind of wanted it; it might have been consensual; what do you expect in Hollywood, etc, etc. Yes, it used be a corner smudge we chose to unsee most of the time, which we are finally beginning to point out for the malignancy it is. Something that seemed to be “locker room” shenanigans, when we step out of the familiar, is revealed for what it really is and we can educate ourselves to be vigilant. In the meanwhile, although we don’t feel so honor bound about other’s scientific work, strangely, we are extremely hygienic about plagiarism. That is, this idea from humanities that prose is sacred (because that is the creative output in that domain). Even the idea of self-plagiarism is a thing. You better reword that methods section for the method that you used for the 10th time. But, hey, if you saw somebody’s poster with a good idea and you decided to do the same thing, its all fair competition and whoever gets there first wins.

Of course, stealing ideas is not the only normalized corruption in our scientific lives. Deliberately leaving out attributions (yes, you know who you are), torpedoing a grant for competitive reasons, trading soft favors, threatening junior people in the field, withholding data and materials, on and on and on. One of my favorite gadgets: I publish a derivative work with proper citations first time, but then from the next set of papers only reference my derivative paper making this whole body of work mine. We know these are “bad” things but not so bad that we would think of them as bad as, gosh, plagiarizing.

Following norms of our greater society, the scientific society especially tolerates (unsees) corruption at the top, the privileged. Pretty much every one has stories of misbehavior by big-time scientists. I have my suite of stories, all the way from stealing grant funds to stealing authorship positions, to actual criminal behavior. But, we stay silent or sort of chuckle over the stories over beer–until somebody actually gets caught with hookers and drugs. The powerful get a pass. Recently, there was a hullabaloo over some alleged plagiarism (actually, idea copying) case by some Chinese authors of a paper by a Johns Hopkins professor. A whole bunch of people either resigned or threatened to resign from the board of Scientific Reports for their (mis) handling of the case. Of course, correcting misbehavior is a good thing. But, I couldn’t help think about whether the same people would have acted so forcefully if the accused were a chair at, say Penn. In fact, when possible misbehavior is called out, especially of powerful people, there is a lot of head shaking and accusing the accuser (sound familiar?) or let’s all be collegial calls.

We are finally moving past blinding ourselves to the Weinsteins of the world. Why are we not treating unethical behavior in science, the real unethical behavior–not the canned things we teach in Responsible Conduct of Research sessions, as we treat sexual harassment? Abominations antithetical to the open society of science? Despite Retraction Watch, scientific fraud is a far rarer problem than simple unethical social conduct of scientists. Or, maybe ethics is the wrong word, setting too high a goal. We all think we are ethical after all. Let me put it differently:

Why do we accept jerks and selfish assholes in science?

This isn’t just about being nice. Remember that because we think science is full of selfish assholes that will steal our ideas, sabotage our grants, and be vengeful with reviews, we act in ways that are not conducive to open conduct of science. Imagine a world where we know for sure nobody actively steals ideas and you just met somebody who is working on the same project as you. Cool! I can’t believe you had the same idea too. Let’s work on this together!

Why do we not teach students a code of behavior where the respect of other scientists and their ideas are as automatic as our fear of copying some prose? Over my 30 years in science, I’ve seen (admittedly only once a month)  the mold in science’s bathroom growing inexorably, normalizing soft corruption until we see it as part of science itself. It time to bring out the bleach.

Posted in Business of Science | 3 Comments

1st week teaching

Every year, I know less of about teaching—and, know less, period.

I always give my students these slides, LearningtoLearn, from my wife Ingrid Olson on the cognitive science of learning.

Posted in Teaching | Tagged , | Leave a comment

Don’t let the journals win

Recently, two somewhat different topics on the business of science came across my Twitter feed.

The first was yet another push for “reproducibility in science” by 72 authors whose latest prescription was to set p = 0.005 (under Neyman-Pearson hypothesis testing) as the new threshold for “significance.” This paper was picked up by the usual Nature et al. press and, of course, generated lots of thumb-time. Without irony, an accompanying Center for Open Science blog post suggests:

“the fact that this paper is authored by statisticians and scientists from a range of disciplines—[…]—indicates that the proposal now has broad support.” (my italics)

That is, the fact that 72 self-selected individuals out of hundreds of thousands of researchers signed on indicates that the proposal has broad support.

Well, regardless, reproducible science, like apple pie, has to be a good thing (maybe). The paper is careful to point out that the proposal is not about publication standards or policy but standards of evidence in science. That is, it is about science.

So then, who in science needs this proscription on the use of the word “significance?” Suppose there is a risky and expensive experiment or maybe even a new dissertation project. I am trying to imagine a student arguing (against advice of caution)

“But, but, the article said it was significant!

In fact, even for very well executed ground-breaking studies, typical journal clubs are exercises in skepticisms, take-downs, and “what-about-isms” (probably to the detriment of the discussants). Do grant review committees take at face value an applicant’s claims of “significant” preliminary results? Do hiring committees?

Q1: To whom is regulating the use of the term “significant” valuable?

I want to bring up the second topic before trying to answer the above question. This involved another (often repeated) discussion on whether one should cite papers in preprint servers like the new bioRxiv. Without regurgitating the various of pros and cons, much of the argument against citing a paper in a preprint server came down to “giving validation” to something that might not deserve such validation. That is, it was again supposed to be about standards of evidence in science. Science will benefit when we only acknowledge that which has been okayed by three people—including the notorious Reviewer 2.

This was interesting as I always thought the role of citation was to establish the “source of information” from which I was deriving my own thesis. I thought the worry on citations was about missing possible relevant sources (and pissing off somebody) or citing sources that might be too ephemeral. We used to cite “pers. comm” which I think is okay as long as the “citee” doesn’t die. Preprints do have some possibility of the second problem, but then it is probably as durable as any online journal. So, why worry about citing non-peer reviewed papers?

Q2: To whom is regulating what is cited valuable?

Any human activity, regardless of whether it is art or science, acquires an economic structure. Efficient operation of the economy requires exchange tokens; we would rather not cart around bushels corn to exchange for milk so we make up credit papers (i.e., money). In science, the economy is supposed to be organized to enable exchange of ideas. But, we also need efficiency, so rather than read the candidates’ papers, we look up their h-index. Time saved. But, use of money or tokens like citations requires establishment of valuations (how much is that puppy in the window). So, one might think scientists who only want to cite “peer-reviewed” work are attempting to create accurate valuations—despite the Dutch Tulip prices created with drive-by-citations (see Lior Pachter’s discussion of this wonderful term from Andrew Perrin). But, what value is being assessed by citations as the valuation mechanism?

This brings me back to the above “significance” issue. Who cares about this language use? As I mentioned, I have never experienced anybody changing their negative opinion because some authors stated that their results were “significant”, statistically or otherwise, that is, if they actually read the paper. If indeed well-regulated use of the term significance leads to reproducible results in print, it should save people’s time. Well, but the original 72 authors of the p = 0.005 paper state that they are not talking about publication standards but language descriptors (valuations) and suggest adding other descriptors like “suggestive.”

But, hey, why even try to convert a metric scale (real-valued probabilities) into some vague ordinal scale? Because, the journals—more specifically the non-expert editors proliferating in current commercial journals, care. Significance is the bouncer behind the velvet rope they use to enshrine the (high impact factor) journal corpus. In fact, many of the journals explicitly ask the reviewers about “significance”. And, those polite rejection letters mentioning “more specialized journals” always mention “significance”. Yes, I know statistical significance is not the same thing as these uses of the word “significance”. Or, is it?

No one can argue against the idea that science demands everybody to do their due diligence. But, the specific concern has been focused on journal publications. Who made the printed words in peer-reviewed journals the matter of record? That is, when did putting scientific work to print become canonization instead a form of communication? And, who decided citations of such printed works should be a valuation token? I don’t know who decided all this but I do know who benefits (the answer to above Q1 and Q2): The Journals.

For the journals, there is a clear self-interest in establishing themselves as “all that is fit to print” and the “matter of record”. That is, nothing would make the journals happier than being the gate keepers of Truth. But, should this be also true for science and scientists? If we look at papers from 10, 20, 50 years ago, what percent of them hold up? Should Newton have been prevented from publishing Principia since he didn’t take gravitational curvature into account? Science is the never-ending search for, and refinement of, understandings of nature–which we pursue through exchange of ideas. We communicate these ideas to each other through the printed medium because it adds precision and distributional efficiency. We established the tradition of peer review because it helps increase (in some undefinable manner) the quality of the communications. Into this economy, commercial journals and journal empires came and, mirroring the rise of the financial sector in the real economy, established a derivative market where the number of publications, citations of publications, impact of publications, are made to replace the actual value of science itself. More they can convince us that “validated citations” are important and that only “significant” results get published in significant journals, more they solidify their position and substitute publication for science. They would like to see you bite down into Credit Default Swap papers of “Altimetrics” than an actual apple.

We are all human and to some extent efficiency and expediency makes us all admire a CV with a large list of Nature, Science, and their baby critter journals. But, we need to remember that the interests of these journals are not the same as the interests of science. Journals literally bank on our asking them to validate us.

Don’t let the journals dictate our values. Don’t let them win. Resist.

 

Addendum: A more serious question is what percent of published papers should be reproducible? If you think 100%, you have never thought about the problem of optimization over a rugged landscape. And, Nature is rugged indeed.

Posted in Business of Science | Tagged , | 1 Comment