The PhD Illusion: Are You Becoming a Researcher or Just Collecting Data?
You can spend three to six years in a laboratory, collect thousands of observations, complete hundreds of experiments, and publish research papers—yet still struggle to answer one fundamental question: What have I actually discovered?
Picture two PhD candidates in the same lab.
The first candidate has collected hundreds of samples, prepared many graphs, completed all the experiments, and even submitted a research paper. However, ask the candidate simple questions to assess their actual understanding of the research. For example, why did a particular result occur? Could there be any other reason for the same result? What new knowledge has this research added? What do the findings actually mean? If the candidate struggles to explain the results or provides unclear and uncertain answers, it may indicate that they have collected data and completed experiments without fully understanding their research.
The second candidate has fewer samples but can speak to the rationale behind each experiment, the meaning behind the results, what limitations impact conclusions, and what research should be done next. This candidate can evaluate the research design, defend unexpected results, and identify what other scientific questions the data raise.
Both candidates can ultimately have the same PhD.
But are their scientific abilities equally developed?
This phenomenon is the PhD illusion: believing that research activities are equivalent to the development of research thinking.

The aim of a PhD is to cultivate the ability to question the unknown, analyze evidence, elaborate knowledge, and generate contributions that are original to the field. It is easy to lose sight of the purpose of research and become consumed with experiments and checking boxes (thesis chapters, deadlines, and publications).
Consider this carefully as a PhD candidate. Reflect on your role as a supervisor of PhD candidates. Reflect on your role as an early-career scientist.
It is not necessarily the case that you are collecting data. The real question is whether your data are teaching you to think like a scientist. A widespread misconception about doctoral research is that the goal is to collect ample data, complete prescribed methodology, and write a thesis. While these components are essential in many types of research, they are not the primary focus of doctoral education.
The goal of a PhD is to help a researcher build their ability to carry out independent research by assisting them to understand the breadth of the research domain, articulate a significant research problem, identify and apply the appropriate methodology and resources, assess evidence, and interpret research outcomes.
Let us consider an environmental science researcher who is investigating the water quality of a river.
The researcher samples at various locations to assess pH, turbidity, dissolved oxygen, electrical conductivity, biochemical oxygen demand, nutrients, and microbial indicators. The researcher enters the results in a spreadsheet. After a period of time, the researcher has hundreds, if not thousands of observations.
Does the researcher have enough data to make a significant contribution?
Not necessarily.
How does the researcher explain the observations? Why is the water quality the way it is? What are other potential explanations? Is the sampling frequency adequate? Are the analytical and statistical methods adequate?
The principle of science is the same across all disciplines. In biology, what kinds of observations must be made to make inferences about biological systems? In social science, what must be observed to draw a causal relationship? In engineering, what must be observed to determine if an intervention is reliable?
Using Reason to Turn Material into Knowledge
Why the Ph.D. Illusion Matters More Than Ever
According to the OECD’s Research and Innovation Careers Observatory 2026, OECD countries conferred more than 300,000 doctoral degrees in 2023. Between 2015 and 2023, doctoral graduations in OECD countries increased by approximately 23%.
These numbers show the volume and increase in advanced research training. They do not speak to the quality of doctoral programs or the skills of graduates. However, they do show the need for real research capability.
The aim of a doctoral degree should be to prepare individuals to contribute to the body of knowledge in their field through a range of potential career pathways, including an academic/research career, industry, government, or other careers that require advanced research or analysis.
The challenge is that academic output is just one facet of research.
Evaluating critical thinking, intellectual independence, and methodological judgment is more subjective. Recognizing incorrect assumptions is difficult to evaluate objectively. This means a scholar can excel at research tasks but still be unable to evaluate whether those tasks answer the right questions.
Being productive and getting things done is vital for research. There has to be sustained effort and disciplined data collection to meet research deadlines. The problem is when completing research tasks becomes the focus, and understanding becomes secondary.
A researcher who appears to be the busiest is not necessarily the most effective. Being a productive researcher doesn’t always mean progress and innovation are occurring in a scientific field. A large amount of research data doesn’t always mean research has led to a major breakthrough.
Doctoral programs aim to develop researchers. One of the outcomes of doctoral research should be the ability to not only identify vital questions but also to evaluate whether the research questions are answerable using the available methods and evidence.
| Research activity | The data-collection mindset | The scientific mindset |
| Literature review | Read as many papers as possible. | Identify patterns, contradictions, limitations, and unanswered questions. |
| Experimental design | Follow an established protocol. | Understand why the protocol is appropriate and what alternatives exist |
| Data collection | Obtain the required number of observations. | Ensure that the observations answer the research question. |
| Statistical analysis | Statistical analysis should identify significant differences. | Evaluate effect sizes, uncertainty, assumptions, and practical meaning. |
| Results interpretation | Explain why the hypothesis was correct. | Consider competing explanations, including those that challenge expectations. |
| Research writing | Produce a manuscript | Build a transparent, evidence-based scientific argument. |
| Publication | Add another paper to the CV. | Communicate a defensible contribution to knowledge. |
| Future research | Repeat the same procedure with more samples. | Identify the next question arising from the findings. |
The two methods do complement each other. A researcher must be able to collect data and analyze it in an intelligent manner.
Distinguishing the two really depends on the intent behind the actions.
A scholar focused on completing the steps in the process may be thinking, “What is my next step?” A researcher developing his or her process thinks about the reason and justification for that step and its outcome: “What am I trying to determine?” and “How will I determine if it is correct?”
Asking questions regarding reliability shifts the basis for judgment and scientific determination about the process.
The Most Dangerous Research Habit: Following Instructions Without Understanding Them
All researchers at some point need to depend on and take direction from more experienced researchers. As a student, I must learn established scientific and research methods from more experienced colleagues. Blindly taking direction, during the process of research and scientific determination, does not help in the evolution of scientific thought.
A scholar replicating an experiment may be unable to explain why or assess if the replication meets the research goal. Doing the task doesn’t develop an understanding of the significance of scientific judgment.
There are several examples of researchers focusing on similar questions and varying hypotheses, in which case using the same or closely related statistical tests as those utilized in a previous study would not be scientifically appropriate or sound.
Abandoning established research practices and protocols is not the solution.
There’s a reason for every major choice.
PhD candidates should be able to justify each choice by answering the following questions.
- What is it about this choice that demonstrates its appropriateness for the research question?
- What does this choice presuppose?
- What are the limitations of this choice?
- What are other possible choices, and why was this particular choice preferred?
Just because you are not able to immediately provide the answers to the questions above, your response should not be interpreted as a demonstration of inability to undertake research. The information should be taken as an indication of what you need to know to be able to provide the answers to these questions.
The aim of research supervision is not to train research candidates to be overly reliant on supervisors. It is to equip them to make appropriate choices by themselves.
A Literature Review Is Not a Summary of Literature
A collection of reviews of published works is a clear demonstration of research maturity. While still a novice in research, a researcher may be able to collect and summarize a large body of literature. The researcher may be able to identify key elements of the works (i.e., name of author, year, objectives, methods and findings, etc.). However, the researcher may be unable to answer the key research question: What is the main conclusion of all these studies?
A researcher is considered to be research mature if he/she is able to evaluate and synthesize published works and identify the research gaps, controversies, and the overall significance of the research that is being carried out.
The studies report a decrease in dissolved oxygen, no relationship with DO levels, and seasonal variations. A description of the studies may simply provide these observations.
A critique of the studies may address the reasons for the result variability. Were the samples collected at the same locations? Was the same analysis and/or statistical technique performed? Was the same range of environmental and biological variables considered? Were the climatic conditions the same? Were the techniques and instruments adequate for the study?
The failure to integrate various lines of evidence may yield the most valuable research opportunities.
Ignorance of a specific body of research may stem from various factors, including inadequate explanation and integration of evidence, methodological approaches that do not adequately consider the problem, neglect of certain populations, and limitations that prevent reliable implementation of recommendations.
These warrant further exploration. If a researcher addresses the above, then she may exceed the boundaries of the reviewed research and formulate new knowledge and close research gaps.
What has not been researched?
The most interesting and valuable aspects of scientific research often occur when results do not support expectations. Prior to the start of a study, researchers assume that results will support a hypothesis. Collections of organisms and other materials and the performance of lab assays and analysis are efforts and expenses that represent a significant investment of time and money.
It is natural to expect that such efforts will support the original idea.
However, this is not always the case.
We expect a given intervention to ameliorate water quality. It is likewise possible that such an intervention alters the growth of other flora or affects the quantity and quality of microbes or other forms of biotic and/or abiotic contaminants. After the experiment, the results showed no statistically significant difference between the treatment and control groups.
What is often done in this situation?
Analyses are repeated until differences in the groups are reported. Negative results may be disregarded. Conclusions may be drawn prematurely. Each of these strategies compromises the trustworthiness of the research. It is more appropriate to consider what the data are telling us.
For example, was the sample too small? Were measurements too coarse? Was the treatment delivered incorrectly? Was there unknown variability in the sample? Is the sample being tested the correct one?
The absence of a statistically significant difference between two groups may mean that the difference is negligible. However, it could also mean that the difference is really small and the evidence presented by the research has very little power, and the true difference is in a larger range.
Just because researchers report a statistically significant difference does not mean that the difference is large or important. Statistical analysis emphasizes the evidence presented to answer the research question, rather than its insignificance. Researchers should expect and report negative results. Surprising and unexpected results may be the most valuable findings of a research study. Furthermore, results should disprove a hypothesis if it describes an untenable or untestable state.
Finding no difference between treatment and control groups may actually lead researchers to better explanations. This could be the case during a PhD, for example, if the research processes were designed correctly.
Results must be presented in a way that allows readers to examine the evidence and the research process behind them.
Inability to adequately reproduce or replicate results has stimulated discussions and formed working groups across the spectrum of academic disciplines. Why? Publicly available results may prompt further research. However, researchers may be unable to replicate the results and evaluate whether previously published research was warranted without a sufficient description of the methods, data, assumptions, and details of the analysis.
One study published in PLoS ONE in 2025 explored attitudes toward reproducibility and replicability with a sample of 452 faculty members working at universities in the USA and India. More than 91.5% of the faculty members surveyed in the USA and 83.8% of the faculty members surveyed in India had knowledge of the reproducibility crisis in science. Misalignment of incentives and constraints in the research process was one issue the authors mentioned. Again, the faculty members surveyed do not represent the whole of the researchers in either country.
It is essential for researchers to understand the methods that allow for the reproducibility and replicability of a given result. PhD candidates must particularly pay attention to understanding the scientific process by considering the following:
Is my data in their raw and untouched format documented and orderly?
Can I describe and justify the procedures and steps taken during the analysis of the data?
Are the procedures and decisions made during the analysis, processed, and formed into evidence, fully explained and described?
Have the assumptions and hypotheses upon which the procedures and methodology are based been evaluated and/or assessed?
Are the outcomes/results of a statistical analysis and procedures logically, methodologically, and rationally aligned?
Is the information and data provided sufficient for an individual knowledgeable in this field of study to replicate the study?
Laboratory research is evaluative in nature and may require the maintenance of an accurate and exhaustive account of each experiment, reagent preparation and quality control, and the environmental conditions of the laboratory.
Field research requires similar meticulous records. Such records may include the same regard for environmental conditions and procedures and steps undertaken during sampling.
Research that is of a computational nature may require similar procedures to ensure accountability and assessment of trustworthiness, accuracy, and reliability. These procedures are not in addition to research. Rather, they are an indispensable part of research. They serve to assess the fidelity of research.
There are several reasons for the repetitive nature of a research study. One assumption may be that additional data collected will strengthen the case made in the research. Additional data have strength and merit if they provide additional information pertinent to the research question.
Before collecting additional data, consider what knowledge gap the additional data will address.
Would additional data collected at a different point in time strengthen the case made in the research?
Would additional data provide insight and facilitate comparisons between two or more groups?
Would additional data gathered at a different point or over a longer duration provide a sufficiently strong case to support a relationship?
Would additional data help validate a previously made finding?
How to Become a Genuine Researcher During Your PhD
Developing the habit of questioning, evaluating, reflecting, and critiquing yourself is what will finally liberate you from being an employee and a data collector to being a genuine researcher.
You can achieve this goal by adopting the following practices.
Ask at least one more profound question every day.
1. Read less, read better: Critical evaluation of academic literature must eventually lead to a theoretical understanding of the fundamental questions posed in that literature, the methods used to answer them, and the implications of those findings for your research. Emphasis should be placed on the completeness of evidence and rationale, rather than on how a study concludes.
2. Comprehend the thought process behind methods: Achieve a basic to intermediate understanding of the essentials of your field, including measurement and research methodology and limitations, the degree of confidence in study results, the population to which results generalize, and other study assumptions. Participate in planned, structured training in research methodology and related disciplines rather than depending on automated or machine-learning methods and outputs.
3. Keep research decisions in a diary: Remain aware of the rationale behind significant choices made in the course of your research and record them. Interpretations made, alternatives considered, assumptions relied upon, reasons for choosing one method over others, and new information that might necessitate revision of your decision.
4. Exchange Ideas Instead of Simply Updating Your Manager: When speaking with your manager about work progress, don’t simply cover your sampling and experimentation activities. Share your ideas and suggestions about your findings. Communicate uncertainties in the evidence, your assumptions and conclusions, and your planned next steps. Identify the issues and constraints that you have encountered.
5. Learn to Justify Your Ideas and Opinions: A competent researcher understands the concepts and assumptions upon which his or her findings are based; understands and expresses the limitations of the evidence that supports his or her conclusions; understands and explains whether the findings are based on a causal relationship or an association; and understands whether the sample represents the population of interest and whether other, alternative conclusions are plausible. A competent researcher is able to present reasoning and a conclusion, as well as defend them against a reasoned challenge. In other words, know when to stop and where to stop.
6. Monitor Your Progress: Assess your progress in understanding science and research, including the next research decision you can make without depending on others.
The purpose of the supervisory relationship is to develop a research participant’s ability to find gaps in research, eliminate uncertainty in science, and respectfully challenge the researcher’s beliefs and assumptions to enhance research independence.
A PhD should produce these changes
A PhD student may not make a career-defining scientific discovery. A PhD student’s experiments may not be path-breaking. PhD students’ research may not be pioneering or published in the highest impact journals.
However, PhD students must be able to develop the skills that give them the ability to do reliable research. Being able to research a question and assess the answer is as important as determining what remains unresolved and requires further study.
The goal of a PhD is to help you become an independent researcher. The goal of a PhD is not to train you to respond to questions with the ‘correct’ answer.
You may not have to make all the big decisions on your own at first, but by the end of your PhD, you should be able to make more decisions about your research. Your supervisor should help you understand the rationale behind the big decisions you make along the way.
Your PhD research, when published, is a record of your findings.
The way you think scientifically affects every investigation you will ever participate in.
Before you begin a new investigation, take a moment to reflect on your assumptions.
When you want to gather new data, take a moment to reflect on what you want to know and what the data can legitimately address.
When you want to construct a new diagram, take a moment to consider what it will represent.
When you are about to communicate your findings, take a moment to reflect on the value of your work.
And when you want to measure your progress by the number of tasks you complete, take a moment to consider your true motivation.
Are you striving to gather the skills required to be an academic, or are you seeking understanding of the research frontier?
The difference may impact the way you write your doctoral dissertation and the type of scientist you become.
Do not leave your PhD with only a collection of data. Leave it with the ability to think scientifically.
My publications or visit my LinkedIn
