Leslie McIntosh, co-author of a new report into improving scientific research, answers questions on reproducibility, falsifiability and methods.
What is good research?
This is a complex question. We must look at (and separate) the research from the communication of that research, as “good research” will be defined differently for each. While defining good practices varies across scientific disciplines, there are common best practices needed across all disciplines. In biomedical sciences, this means capturing a significant amount of ancillary information to the research process. For example, best practices would include having the entire compendium of scientific research documented and reproducible. This will include ensuring clearly articulating the research objective, citing your data sources, and sharing code. But not all things (e.g. private data) can be openly shared. So good research should be a commitment towards transparency, even when accessibility of all pieces is not possible.
How much current published research would you say is good research?
There is good research represented across scientific journals and globally, yet, almost all published research could be improved. It is frustrating to read what appears to be good science, yet the publication lacks a robust methods section to back up the work. As mentioned in the Making Science Better report, if we start from a simple construct of what good research is then it starts with a well-defined study objective or hypothesis. While I believe most researchers have a hypothesis, those are not clearly stated in the publications.
Do many researchers consciously or unconsciously skew their results in search of the conclusions they want?
Researchers definitely have views that influence their work; we are all human and cannot escape some biases that may unconsciously skew results. However, these views differ by research. And as science is a collective endeavour by individual actors, any skewing that may occur could be accounted for. There is certainly evidence some researchers consciously skew results through such means as p-hacking (i.e. inflation bias), which is analysing resulting data until one finds something significant.
Your report says reproducibility is very important - why is this?
Ultimately, having reproducible research builds (or weakens) the trust in the scientific work. Reproducibility lies at the heart of the scientific method. This method depends on the verifiability and reproducibility of findings crucial to the construction of a scientific heritage. This allows the construction and validation of findings from others’ work.
Researchers definitely have views that influence their work; we are all human and cannot escape some biases
Is big data and data access (for example, that produced by Genomics England) going to have much of an impact on research?
Both big and small data will definitely have an impact on research. Having available data – particularly data that are well documented for better understanding – offers immense opportunities to make new discoveries without needing to collect new data. But big data has the power of offering views that smaller data often at times cannot, for instance with rare diseases. It will be important, however, as the growth of reusing large datasets continues, that transparent research processes are communicated. This will help to highlight assumptions that can then be tested and challenged.
Should all registered research be published?
From my perspective, all research should have transparent processes available even if all research results are not published.
Why is failure something that is increasingly important in modern research?
Failure has always been important in research; success is built off of failure. Yet, science is more complex than ever before due to many advancements (e.g. computational capabilities) thus obscuring the true causes of success or failures. It has been easy to assume that we truly test what we think we are testing (e.g. a new drug), when in actuality there is a nuance of a computer programme influencing the calculations. Additionally, failing in the process of conducting research is different than having an experiment fail (e.g. not support the hypothesis).
What is the one key message you would like people to take away from your report?
To make science better, we need to make better science easier. Better science is reproducible and falsifiable. Making science easier means embedding good practices and checks for good science into our scientific ecosystem.
Dr Leslie McIntosh is one of the lead authors of the Making Science Better: Reproducibility, Falsifiability and the Scientific Method report. She is a consultant, speaker,and researcher passionate about mentoring the next generation of data scientists.
Image credit | iStock