Research bias can (unfortunately) be found everywhere in academia. The term refers to anything that distorts our “paper” models of the real world. Excellent studies seek to remove the malign influence of anything that might distort and warp their intricate models, findings, and conclusions.
Eliminating confounding factors, flaws, and faulty conclusions is essential to producing quality work that’s truthful, accurate, and applicable. Minimal bias can also vastly increase a work’s chances for publication and later citation.
Definition: Research bias
Research bias is a broad umbrella term covering many negative factors and flaws that distort empirical studies. Anything that might actively damage validity and reliability falls under the definition.
Research bias can negatively affect qualitative and quantitative (i.e. statistical) findings.
Research bias can result from accidents, unrepresentative data, subconscious attitudes (e.g. preconceived prejudice), or, more rarely, deliberate lies. Authors, publishers, and editors can all exhibit research bias. Qualitative studies are particularly vulnerable.
Types of research bias
Some classic mistakes crop up more frequently than others, causing research bias. Here’s what all good researchers dread to find.
Confirmation bias represents damaging over-investment in a system, theory, school, or idea. At its worst, it may lead otherwise reasonable people to ignore objective reality when making conclusions.
It may additionally lead researchers to highlight selective evidence that reinforces their current beliefs at the expense of that which doesn’t. It’s also sometimes invoked by researchers who have an ideological or personal incentive (e.g. funding) to prove a thesis.
While developing a pattern of evidence selection can strengthen a hypothetical argument, examining crucial contradictory evidence may lead researchers to create better contextual theories.
Mixed evidence doesn’t automatically preclude a proven null hypothesis.
Information bias is when something is fundamentally untruthful with the data collected for the study. We can identify four main causes (types).
People generally don’t have perfect memories and ideal self-awareness. If asked to remember vague, distant events? Respondents may reply with inaccurate, irrelevant, or partial information. “Lenses” (e.g. nostalgia, shame, pride, propaganda, hindsight, “presentism”) may also distort how we remember our past.
Over time, people form general expectations about how the world around them behaves. When new evidence contradicts those basic conclusions, the observer may reject discussing it, as it doesn’t fit the (supposed) broader picture.
Performance (The Hawthorne Effect):
Sometimes, participants will unconsciously treat being observed as an exam and upgrade their behavior accordingly. Metrics and feedback gathered from tests may not accurately reflect day-to-day baseline performance!
Regression to the mean (RTM):
General probability rules a nominal value will likely follow an (extreme) outlier. Regression to the mean often causes natural statistical normalization to be mistaken for a radical causative due to a simple “right place, right time” coincidence.
Actor-observer bias is a type of prejudice. How we recall and judge events may depend on our levels of self-belief and whether one was an invested participant.
Certain people may attribute personal gains to themselves and personal setbacks to bad luck or others. Observers may also be much more critical of rival actors than they would be of themselves. People may also be motivated to give socially desirable (but inaccurate) answers.
Are we left unchallenged? Selfish a priori assumptions can heavily distort our bias understanding of causative factors. Seeking other opinions and logical queries of the observer’s interpretation establishes truth.
Researchers are only human. Academics are (usually) highly motivated people with vested interests and agendas who may bring their baggage, feelings, preconceptions, and ambitions to what they’ve chosen to study.
Sometimes, this may overwhelm or circumvent strict objectivity. This can result in:
- Leading questions and statistics
- False assumptions
- “Boxed in” conclusions
- Poor framing
- “Bad faith” arguments
- Severe distortions of actual findings
There’s also the extensive issue of driven confirmation bias in qualitative work.
Researchers may, willingly or unwillingly, actively guide subjective conversations to get ideal responses.
How to identify research bias
Systematic questioning works. Start by looking at the study’s design, questions, data points, and any conclusions made before running through the checklist below.2
- Is the study reasonably fair, objective, and comprehensive?
- Are any questions, statistics, or segments unfairly biased toward proving a hypothesis?
- Are there leading qualitative questions or conversations present?
- Does the author reject or over-promote evidence due to ideological or theoretical investment?
- Does some cited evidence (or literature) seem too good to be true?
- Are any statistical arguments made selective or “cherry-picked”?
- Are there any notable conflicts of interest?
- If the paper is worthwhile – is there any broader social bias preventing publication or citation?
How to avoid research bias
If you’re conducting a study? Follow the checklist below to help minimize your research bias.1
- Set out a clear and detailed methodological design and (peer) review it for objectivity and fairness
- Randomize your respondents to limit anomalies and “cluster” bias
- Use neutral and open-ended questions from a fixed schema in interviews
- Avoid judgmental, leading, and intimidating language
- Repeat tests as much as possible to check for sudden changes
- Use single and double-blind reviews and testing (e.g. numbered questionnaires)
- Apply common sense (i.e. is this feasible?)
- Ask other experienced researchers for their opinion(s)
- Read relevant secondary literature to compare
- Apply statistical controls and normalization
No, you cannot. It is impossible to avoid observer bias entirely, especially in studies where data is collected manually, but there are ways to minimize it as much as possible.
Researchers can minimize observer bias by standardized training of observers, using blinded protocols, identifying any potential conflicts of interest, and doing continued monitoring of objectivity in observers.
A researcher who has taken no steps to minimize this bias is more likely to misinterpret data. Observer bias has been shown to affect the validity of the results of studies significantly.
The Hawthorne effect describes a type of observer bias where the participants’ behavior changes after realizing they are being observed.
1Formplus Blog. “Research Bias: Definition, Types + Examlpes.” July 27, 2022. https://pitt.libguides.com/citationhelp.