Elisabeth Bik is a microbiologist and scientific integrity consultant, who is known for her work spotting errors in research papers. Having quit her job in 2019, Bik now investigates scientific misconduct and regularly posts on her blog Science Integrity Digest. Bik also boasts a Twitter following of over 100,000 followers, which allows her to share her detective work and engage with the community.
Please note the transcript has been edited for brevity and clarity.
FLG: Hello, everyone, and hello, Elisabeth. Thank you so much for joining me today for the latest interview in the A Spotlight On series. Today, we are joined by Elisabeth Bik who is a microbiologist and a scientific integrity consultant. So, Elisabeth, if you can just introduce yourself and tell everyone a little bit about what you do as well.
Elisabeth: Sure. So, I’m Elisabeth Bik. I originally did my PhD in the Netherlands. I then moved to the United States and have lived there now for 20 years. I’ve worked 15 of those 20 years at Stanford as a microbiologist doing research on the microbiome of people and of dolphins! And about two and a half years ago, I quit my job. I also worked a little bit in start-ups. But I quit all of that. And I turned what was my hobby into my new job, which is looking at scientific papers and looking for misconduct. This includes things like plagiarism and images that have been duplicated or photoshopped. So, that is now my new job. I earn some money by consulting, but I do most of my work as a volunteer.
FLG: How did your career in academia begin?
Elisabeth: Like most careers in academia, I did my PhD at the Dutch National Institute for Health and the Environment. I worked on Vibrio cholerae, which is the bacteria that causes cholera. I worked on molecular identification of different strains. I also did a short postdoc in that same lab and then I moved to more the clinical side of microbiology. So, I worked in a hospital, setting up a molecular lab. We were trying to see if we could identify different strains that different people were infected with and doing detective work to see if they were infected with the same strain. So, whether one patient maybe had contaminated another patient. I worked on that for about four years and then moved to the United States and first did a postdoc, and then after, a staff position at Stanford for 15 years. And then I worked for two years in biotech.
FLG: How did you become a scientific integrity consultant? And for those who don’t know, what exactly is this?
Elisabeth: Well, there’s not many of us. So, I am a consultant. I’m mostly hired by scientific publishers or universities who want to do an investigation on a set of papers or a particular researcher that has been suspected of misconduct. So, I would serve as an expert witness or something like that, where I can look at images that have been published and compare them. For example, with original images that a university or a publisher might have obtained and see if the originals match to the published ones to see if there’s a duplication. So, I will be hired by one of these institutions as an expert person who can look at these images, and then I can write a report. And that helps these publishers or universities to make a case or to dismiss a case i.e., maybe there was no evidence. So, it works both ways. And so, I started this work as a hobby. I heard about plagiarism, heard about image duplication, and just started to look for it and by a series of coincidences found several examples. That became my hobby for many years before I quit my job.
FLG: If we take a step back, why is scientific integrity so important?
Elisabeth: So, for me, science is about finding the truth, finding what is going on in biology or in chemistry, whatever field we work in. We – scientists – are doing experiments and measurements and will report upon that in scientific papers. And then other scientists will base their work on those papers. So, whenever we start a new research project, we will look at previously published work. So, if one of those papers contains false data or fake data or even if it contains an error, that means that other scientists might waste a lot of time trying to reproduce that. So, for me, science is this continuum of published papers, and we need to make sure that these published papers are as close to the truth as possible. So, I care about science integrity, because it might waste a lot of other people’s work if we don’t care about that.
FLG: You started a blog in 2014, called Microbiome Digest. What made you set that blog up?
Elisabeth: So that is about my work as a microbiologist. So, Microbiome Digest, started first internally. I worked in the lab, we all worked on the microbiome of animals or humans. And it was just a new field, lots of papers were coming out. And I just felt overwhelmed sometimes by all the papers. If you just did a search for the word ‘microbiome’, there would be so many papers. So, I started sending out emails to my co-workers, like, ’Oh, I think this is a paper that you should read, it fits your research’. I just thought that was helpful. And it turns out, other people found that helpful. And then they said, ‘Why don’t you send this out on a larger scale to all people working in the microbiome field and curate these papers’. And so, I started doing this in a blog form. I was curating papers on the plant microbiome, or the environmental microbiome, etc and picking up the pearls and the nice papers that I thought would be worth sharing. This was the blog, it turned into a daily thing. And now it’s being run by a team of volunteers. So, I don’t run this myself. But I also run another blog, which is called, Science Integrity Digest. That is the blog that focuses more on my science integrity work. So, I have two blogs, both digests. The Science Integrity Digest blog is the one that I run myself on topics of science integrity that I’ve worked on.
FLG: When did you set that up? What was the process like setting that one up as well?
Elisabeth: The Microbiome Digest I set up in 2014. The Science Integrity Digest I set up in 2019, around the time that I quit my regular job and started to focus on science integrity as my career. I just wanted to write about the cases I had worked on myself. So, it’s not really a digest in terms of what happens in the world, because most of that is covered by Retraction Watch, and I didn’t really want to interfere with that, because that’s a great blog. I wanted to focus more on my findings on particular sets of papers that I have posted on sites like PubPeer or discussed maybe on Twitter. But I wanted to make it for all people who are not on Twitter or who do not really follow other blogs. So yeah, it is a site where I summarise my work on the different cases.
FLG: What was it like quitting your job to pursue this?
Elisabeth: I worked in a start-up company about the microbiome and whenever I was talking about my work in the start-up, and also my work as my hobby on science integrity, I started to notice that I sounded a little bit more enthusiastic when talking about my hobby on science integrity than about my real work. So, I thought maybe that’s what I should be doing. It was a little bit scary because I had a stable income and so to quit that and become a consultant was a scary step. Because you don’t really know if it will pan out financially and things like that. So, I did quit my job and of course, I wished them all the best. I started doing this full time and because I had some Twitter presence, I felt I had built up enough of a reputation of working in this field that I could take this step and hope that some people would find me and hire me as a consultant. And they did.
FLG: What is the current landscape of scientific misconduct and photo manipulation? How often is this occurring? What type of things are people doing? Could you give us an overview?
Elisabeth: Right, so when I started this work, I had the same question. I first focused on plagiarism. And that seemed like a very tiny slither of science papers, because most scientific publishers will screen for textual similarities. There is, of course, stealing somebody else’s ideas which is very hard to detect. But I was wondering how many duplications or maybe photoshopping type jobs were going on. So, I did research where I scanned 20,000 papers that contained the word ‘Western Blot’. So that enriches for papers in the molecular biology field that have western blotting or immunohistochemistry. So, it’s a particular subset of papers, but they usually have photos, and I can screen for those. I found that roughly 1 in 25, so 4% of these papers had duplicated images within that paper. Not even across papers, but within that paper, there was a problem. And so, I don’t know if that means that 4% of papers are fraudulent. Some of these are errors. But we estimated that roughly half of those, so 2% of all papers, were suggested for science misconduct. I actually think that the real percentage of misconduct is much higher than 2%. Because it’s very hard to screen for papers that have fraudulent data in a table or a sequencing data set or a line graph. It’s very easy to commit misconduct in there. If you do it with photos, it often leaves a trace. Unless you’re a good Photoshopper. So, I would guess that around 10% of papers contain science misconduct, contain fraudulent data or falsified data. So that’s actually very scary and a high percentage.
FLG: That is high! Day to day, how would you screen for these errors? Also, on average, how many errors would you find in a single day?
Elisabeth: It’s hard to say because nowadays. I work mainly from tips that other people have given me. So those are already papers that somebody else thinks there’s a problem with. So, my percentage of problems that I’m finding could be very high on a given day. But I’ve also worked on very complex cases. The ones I’ve written a blog post about, those are very complex where lots of money is involved, because one was a company that is at the Nasdaq’s and there’s people who buy stocks. So there, you have to be very careful. And I could work on a case like that for a week and maybe there are only five or 10 papers that I’ve found that are suspicious. But on other days, I found up to 50 problematic papers when I have focused on one particular author who might have duplicated their western blots or has overlapping photos. So, I might have a really good day where there’s lots of duplications that I found. But most days, I search and search and don’t find anything. So, it depends a little bit on the dataset that I’m working on.
FLG: How do people inform you about their suspicions?
Elisabeth: It’s through all channels. So, I might get an email from an anonymous group of people who have created an anonymous email account just for that purpose. They say, ‘We’re a group of worried researchers and we’ve come across this particular paper and we think there’s a problem with it’. Or I might get an email from a person, I can see who it is, and they say, ‘I think this particular professor is committing misconduct. Can you check all of their papers?’ Or I get flagged on Twitter or a poke on Microbiome Digest because they think there’s a problem there. So, through all channels, I might get requests to look at data. And it’s either anonymous or I know who it is. I try to protect the identity of the people who have flagged things to me as much as possible. And usually, I will post things on a website called PubPeer and just say, ‘An anonymous reader has flagged this paper to me, and I agree with this problem’.
FLG: What are some of the potential reasons why someone might commit scientific misconduct?
Elisabeth: It’s all kinds of scenarios that I think might play a role. So, I could imagine a person working for a professor who is a bully and says, ‘I want these results by Friday. I want this PCR to work, otherwise, I will fire you and I will hire another postdoc’. And if you are a postdoc on a visa in a foreign country, then that is a scary thought. If you’re, let’s say, a person from India on a visa in the United States, and you are fired from your job, that means that you don’t have a sponsor, and you have to leave the country within five days (I’m talking about pre-COVID, obviously). But those are situations where people are scared to lose their job. And so, if you have a family or there is money involved in a salary and you’re in a situation like that, you will commit misconduct, because you feel that’s your only option. And I could also imagine a person who was extremely successful as a postdoc, published an amazing discovery in Science or Nature. Then, got a tenure track position, but switched their research topic a little bit and then the research doesn’t work. And we’ve all been there, where we have these phases of big successes, and then months or years of hard work, but no real experimental data. So, I think in a situation like that, where a person has tasted success, has published in Science and there’s great expectations for this person to do really well. I think might be another scenario where they could start to commit misconduct.
There are also people who are forced to publish papers. I’ll give a very specific example of medical doctors in China. They need to publish a paper in order to get a position at a clinical hospital. So, they are not interested usually in research, they want to help patients and cure patients. And that’s what their education was all focused on. But now they have to publish a paper, yet they’re not given time off to the research, they’re not working in a research institution and there’s just no way they can publish a paper. And so those people might buy a paper because there’s companies offering these papers. And so those are what we call paper mills – companies or entities of some kind that produce papers, fake papers, and sell them to authors who need them. So, it’s usually incentives or expectations that will drive people to do misconduct. I don’t think there’s a particular person who starts their career thinking, ‘I’m just going to cheat’. I think most scientists will start from a point of where they’re really honest, but they just feel the only way out, the only way to get further in their careers, is by doing this misconduct.
FLG: I didn’t know that these types of companies exist. It is crazy!
Elisabeth: It is organised crime almost. And that is what worries me the most, that these paper mills are pumping out hundreds or even thousands of papers that are hard to recognise, they look surprisingly real. And it’s only when those companies make mistakes, like using the same figure twice or having overlapping images that we can find these papers. But most of the time, there’s no smoking gun, there’s just a suspicion that it is a paper mill. And it’s really hard as a reader to know what is wrong with these papers or if they’re real or not. And only the publishers or the journal editors can then ask the authors, ‘Can you provide some real data?’ And in many of these cases, the authors will say, ‘Okay, just retract the paper’. And you just know if this was a real paper, you would fight for your paper, right? You would make sure that your paper was not retracted, you would provide all the evidence to show that this is a real paper. But these authors just say ‘Oh, just retract the paper’. So that tells me that these papers are indeed fake.
FLG: You are very active on social media and on your blog as well. Many would argue that these types of issues should be flagged internally, directly to the journals or research institutions. What would you say to this?
Elisabeth: Right, I can understand that point of view. And that is how I started this hobby and this work on science integrity. So initially, I did that scan of 20,000 papers, and I found around 800 papers with duplications. And I reported all of these to the journal editors privately and then waited five/six years. And after waiting so long, you would have expected all of these papers to have been taken care of. But, unfortunately, only 40% of those papers had been either corrected or retracted and 60%, so the majority of these papers, had not been addressed. So that’s a lot of papers that have potential problems and that could still be cited. And I wanted to give the editors a chance to address these issues privately and ask for evidence, but I was frustrated that the journals didn’t respond. And I actually care more about readers and other scientists who might cite these papers and who might base their research on these papers and wanted to alert them that there’s a problem. So, I’ve sort of switched.
I still will report these papers to the journal editors or the publishers. But I will also flag them on PubPeer, because people can install a plugin that will work with your literature searches. So, if you do a literature search, you can see which papers might have a potential concern and you can base your future research on what PubPeer might tell you is a trustworthy paper or not. So, I care now more about the readers and the other scientists than I care about the privacy of trying to resolve this matter behind the scenes. I feel this needs to be shouted from the rooftops. And that’s the only way to almost force journal editors to take action. It feels like when you try to address these things privately, they don’t feel the need to respond. When you flag things on Twitter or PubPeer, it forces them to take action. And it’s really based on what I’ve done in the past. I didn’t start this way, I tried to play it nice. But I sort of gave up.
FLG: In 2020, you published a blog analysing a paper claiming the antimalaria drug hydroxychloroquine was effective in treating COVID. Since then, strong evidence has been found that this drug has little to no impact on COVID illness severity or likelihood of death. You received a lot of abuse from this, and the authors of the paper even filed a complaint to a prosecutor in Marseille, France, against you – would you be able to expand on what happened?
Elisabeth: Well, so I wasn’t pro or anti hydroxychloroquine, per se. This particular study had been tweeted by President Trump, and so was suddenly World News, and a lot of people tried to get hydroxychloroquine because they believed it was the wonder drug against COVID. And this was March 2020, where we were at the start of the pandemic, nobody really knew what to do and there was panic everywhere. How can we solve this? So, this was World News, literally. I just thought this study was poorly set up. It had a very small number of patients. It had many flaws where particular patients who did not do well on hydroxychloroquine were left out of the study. There were all kinds of red flags. So, I wrote my blog post and obviously, the authors of the paper were not pleased with me, and I can totally get that. They called me out with some weird names on Twitter and in the French Senate. But in the meantime, I looked at more papers from this group. And since I specialise in images and duplications, I just screened a bunch of their papers for image duplication and found several papers with severe problems or signs of not just duplications, not just errors, but signs of perhaps deliberate manipulation of photos. And so, I posted all of these on Twitter, and I also found a bunch of papers with problems with ethical approval. There were all kinds of problems with this particular research group.
So, in total, I flagged about 60 papers of that group on PubPeer and so they got 60 emails saying, ‘Elisabeth Bik had a comment on your paper’. They did not reply to these concerns. Some of the papers got some replies but not from the big professor names. And most of these concerns could be taken away by showing original blots or showing approval numbers. Instead, they filed this complaint against me for harassment and blackmail and things like that. So, this is a complaint that was filed to the prosecutor in Marseille, which is where this research group works. I am still not quite sure what stage this complaint is in or if it will turn into a lawsuit or not. I hope obviously not. But instead of answering the scientific questions, they threatened me with a lawsuit. And I think that means they don’t have answers to my scientific questions. It’s still unknown what will happen. I don’t think if the prosecutor drops the case that they will tell me, obviously. They like to keep me not in the know. And I think it’s just a way to threaten me and to try to silence me. I didn’t let them silence me because I kept on searching for more papers and finding more.
FLG: What has this abuse been like for you?
Elisabeth: Yeah, I mean it’s scary, because I think I just raised some legitimate concerns about papers, which again, could be easily solved. And I had not expected this to turn into a lawsuit or a threat of a lawsuit. So that was very scary. I’m not employed, I don’t have a big-name University with a big legal team that can help me. So, I would have to hire a lawyer and probably lose a lot of money just trying to defend myself. And luckily, there was lots of support that I got from people online. I think this is one of the advantages of having a big Twitter account is that you also have a lot of supporters. But usually, the people who are negative and who are critical will make more of an impact, and obviously will make me sad. I did have some sleepless nights and questioned myself, ‘Should I continue this?’ But I thought, ‘No, I should continue this’. Because I do think I have asked the right questions. I tried to remain polite. I did not insult anybody. I just raised some questions. Instead, they were insulting me and threatening me with a lawsuit. I know I’m right. And I will keep on doing what I do. And so luckily, I had lots of support from other scientists. There were two petitions with hundreds of signatures from people who were supportive of me. But it’s usually the negative commands on Twitter that will somehow count more than the positive comments. But I did get lots of positive comments, as well.
FLG: What do you think needs to be put in place for people who do this type of work?
Elisabeth: Most are doing this work anonymously. So, I’m one of the few people who does this under their full name. And so that makes me very vulnerable for lawsuits. But I also think it’s important because I can talk about my work and I do get a lot of support, while the anonymous people who work really hard behind the scenes don’t get those words of encouragement. So, it’s a decision I’ve taken to become public and to write these blog posts on my full name. But it also makes me very vulnerable. I’m not sure how to solve this. I think there should be some like anti-SLAPP law, which we have in California where I live. So, in California, if you try to sue somebody just to silence them or raise fake lawsuits by having fake arguments in an attempt to intimidate the person who criticises you, there are strong laws in California against that. And I’m not a lawyer at all, so I hope I have explained that right.
But I hope that other states or countries will have similar laws like that. Because I think this work needs to be protected. It’s sort of whistle-blower work, even though I’m not an internal whistle-blower, I’m sort of an external whistle-blower. This work needs to be protected. As long as I raise valid concerns and I don’t insult people, I should not have to be afraid that I’ll be sued. Again, I’m not a lawyer, I’m not sure how to solve this, but I hope that there will be stronger laws and more support for scientific whistle-blowers to protect their right to ask questions and not be sued for that.
FLG: How do you think we can improve scientific integrity as a whole?
Elisabeth: I guess part of it is driven by the culture that we have as scientists that we need to publish. We’re so held accountable for the numbers of papers and the impact factors. And I think that is part of what drives science misconduct. We feel this urge to publish and to have positive results. And so, I think part of it is we need to have smaller publishable units, where if you’ve done a couple of experiments and the results are negative, there’s no grandiose positive PCR band, or the drug doesn’t work, we need to have better ways of publishing those results as well. So, we need to have more journals accepting negative results, not just focussing on the shiny, beautiful results. Because there’s a lot of research that ends up in the trash can, because it doesn’t yield anything. But it’s worth sharing so that other people don’t have to repeat these experiments and try to replicate it. So, I would be in favour of very small publishable units where you can just publish one particular figure, and then other people can comment on it and replicate those results. And where replicating other labs results could count towards your resume in some way. Like you get bonus points for that. And that’s a completely different way than scientific publishing now, which is focusing on really big studies with lots of figures that, once they’re published, nobody really seems to care about reproducing them, because it doesn’t count towards your resume if you’ve tried to replicate somebody else’s results. It’s not publishable because it’s not new, and it won’t be accepted by another journal. So scientific publishing focuses too much on positive and novel results and doesn’t focus yet on reproducibility. And I feel there needs to be more focus on that.
In general, I think there’s already enough policy at every university teaching people about what cannot be done. I think there’s way too many of those classes that we all have to take. There’s a lot of overhead and a lot of people working on these things. There’s lots of websites with regulations, PDFs and fancy words, like stakeholders, which I don’t quite understand. But I think we need to focus more on real life examples and on solving these cases. Because I’ve seen many universities with beautiful PDFs and guidelines about research integrity, and whole websites dedicated to that. But when I write to them, first of all, I cannot find a contact person whom to write to. Then, I write to somebody at the university and I don’t get a reply, or I get a vague reply and then I hear years of nothing. So, it’s nice to have all these beautiful regulations and classes and courses and certificates set up at a university, but we need people who will act upon my allegations or other people’s allegations. And there’s too much protection of big-name researchers at universities, where the research integrity officer doesn’t want to rock the boat or tries to silence the whole thing. There are too many conflicts of interest. So, in general, I would hope for a more international or maybe country wide research integrity institution that could investigate these cases without having to worry about things like people who get a lot of grants for universities or getting rid of these conflicts of interest and having independent organisations investigate these cases.
FLG: What are you overall thoughts on the peer-review process?
Elisabeth: I don’t think peer review can always catch misconduct. In most cases, it cannot. Peer reviewers are volunteers who dedicate a couple of hours to peer review somebody else’s paper for free. And they don’t really know what to look for to detect misconduct. So, there are several ways of improving that process. I think having a tick box on your peer review form that you fill out after you’ve peer reviewed your paper saying, ‘I’m suspicious of some of the data here or I think there’s no sign of misconduct’. I think having that box will help people look at the paper and give it another five minutes and think about it from a different perspective. That might catch some of the problems in figures, for example. But in general, it’s really hard to detect that. So, I do try to educate people on Twitter by giving a lot of examples. And I think that will make people more aware of what to look for. But, in general, you would hope that publishers would have educated staff who can just screen papers for these types of things like they already do for plagiarism. They have automated screens for that.
I also think that peer review, for me personally, is getting harder and harder because papers are getting bigger. I feel no longer qualified to peer review all aspects of the paper. So, I’m still sometimes doing microbiome paper peer review and I can review parts of the paper. But there’s other parts like statistical parts or particular clinical data that I just don’t feel qualified to peer review. So, I feel papers are getting too big and too multi-disciplinary for one or two peer reviewers to completely look at a paper from all angles. So, I wish there were smaller parts of the paper that I could focus on, that I feel comfortable with. In general, I do feel that peer review is a good process. Maybe people should get paid because we do it all for free. And then somehow the publishers make money out of all of this. I feel we, scientists, do everything for free. So, there’s many things wrong with peer review. But I also think we cannot abandon peer review completely, because without that it would be a wild west, and anybody could write anything. If you look at some preprint servers, you will see that having no peer review in place will result in a lot of garbage being published.
FLG: Are they any efforts to create an automated algorithm to improve integrity detection?
Elisabeth: Yes, so there is software to screen for textual similarity. So, plagiarism. But it’s much harder to look for duplicate images. People are working on writing algorithms and tools to find that. It’s still very hard and not perfect. But I’ve worked with two of these tools that look like they’re doing a great job. And they’re not perfect yet and they’re not really ready as far as I know for primetime, where you can just put in hundreds of manuscripts that come in at a publisher’s desk and then screen for duplications. But in the end, it’s all solvable. This is just computational things, which are still hard, but I do think this will be solvable in the end.
And the advantage then is that we can screen for manuscripts that are being sent to a publisher, not just for duplications within that paper, but also across all other papers that have been published. So that is something a human could never do. And I think that we’ll find lots of cases where people are just reusing somebody else’s Western blot, inserting them in their own paper or maybe flipping them or rotating them a bit to make them look different, then labelling them as something completely different. I think that is very hard to scan for as a human and I hope software like that can become commonplace in a publisher’s toolbox to scan all their incoming manuscripts.
FLG: What would be the one message that you would give to people who are concerned about whistleblowing?
Elisabeth: Unfortunately, I would have to tell them that if they are in their early career, they are graduate students, for example, and they want to blow the whistle on suspicious things happening in their lab, maybe by their professor, then it’s probably not a good thing to blow the whistle because they will be fired. Unfortunately, this is the situation I’ve encountered too many times where a person did raise a concern with the professor themselves, or with the Dean of research or the research integrity officer at their institution. But the professor then got cleared of misconduct and the junior researcher had to leave because they were labelled as hard to work with or were revoked access to the building or weird things like that. They’re just bullied and slowly forced to resign. And so, unfortunately, if you’re very junior, you have lots of things to lose, and you have very little chance of winning the case or being regarded as the person who’s right. And when you are the big PI who brings in a lot of money, you will be protected by the university. And we’ve seen lots of these cases in the ‘Me Too’ movement, where certain professors were known to be sexual harassers, and everybody knew it. They were reported multiple times, and nothing happened. And so, I feel lots of similarities to those cases.
I’m always happy to screen papers once they’re published. I can actually raise them on PubPeer and raise concerns. But inside knowledge, where a whistle-blower just knows that a particular figure was manipulated, where it’s hard to look at the figure from the paper and see it, I can’t really help them because I would expose their identity, as only an insider would know it. So, I wish it was different. I wish I could tell people to report it no matter how junior they are. But that’s unfortunately going to turn out bad for the junior whistle-blower. But yeah, get out of the lab, I would say if possible. It’s hard. Because it’s such a hierarchical position to be a junior researcher, you need so much from your PI. You cannot usually leave the lab and find another job. That’s not how academia is set up. So, it’s tough advice. And I wish I could word it differently.
FLG: It is a tough time! Thank you so much for joining me today, Elisabeth. It’s been great. I think this work is so important and often undervalued. Thank you so much for joining me and sharing your important insights, it has been great. Thank you.
Elisabeth: Thank you for having me on, Shannon. It was great to be here.
