Mitigating Bias in Scientific Evaluation and Decision-Making Processes
No person is to be prevented or excluded from pursuing an academic career because of non-scientific factors.
In order to ensure the best research projects receive funding, it is important to make the most of all the potential and talent available and to arrive at funding decisions in a fact-based and non-discriminatory manner.
Both positive and negative bias can potentially come into play in the process, for example with regard to sex/gender, origin, health status, family responsibilities or an applicant’s research institution. Every individual is biased in different ways and to differing degrees. This also applies to research and to researchers who strive for objectivity and evidence.
In order to ensure that the assessment of proposals submitted to the DFG is based purely on scientific criteria, and to mitigate bias as far as possible, a short film and an e-learning course are provided along with recommendations for action and more background material.
Note:
The film is also available in an accessible version with a spoken audio descriptio(externer Link) of the visual content.
General Framework Conditions Established by the DFG to Mitigate Bias
Transparent procedures and the observance of predefined standards help reduce bias. In particular, the defined assessment criteria are to be applied equally to all individuals involved.
The DFG attaches particular importance to ensuring that its processes leading up to funding decisions do not discriminate based on non-scientific factors. Accordingly, the assessment of project proposals and research achievements is to be science-led. When conducting a review, the individual life situation (where applicable) is appropriately taken into account, and this is done exclusively in the applicant’s favour: unavoidable periods of absence or periods of restricted research activity should not impact negatively on the review, evaluation or decision. Applicants are advised to notify the DFG of special personal circumstances. Read mor(interner Link)
The DFG’s procedural steps of review, evaluation and decisio(interner Link) are designed to be carried out separately and independently of each other: this in itself ensures that the review and evaluation undertaken by the DFG’s various statutory bodies is subject to quality assurance.
Before a decision is made, the DFG Head Office also carries out a quality assurance of approvals and rejections so as to ensure procedures have been followed correctly. If bias or discrimination is identified here, the appropriate action can be taken.
The general instructions and the specific instructions for the review proces(interner Link) lay down the assessment criteria for project proposals; they also emphasise the confidential nature of the proposal documents to be reviewed and describe how to avoid the use of non-scientific criteria. Votes are to be based on the content of the proposal documents, the curriculum vitae, and also – in the case of on-site and panel reviews – on the presentation of the applicants and their qualifications for the project.
There is also a set procedure to rule out conflicts of interest(interner Link) The Framework Rules of Procedure for Review Board(interner Link) set down uniform working principles for each review boar(interner Link).
The categories of the DFG’s CV templat(interner Link) clearly convey that research achievements in their entire breadth (without reference to metrics and without a photograph) are relevant to the review and that an applicant’s individual life situation should also be taken into account in their favour where relevant. (Please also read our FAQ on CV(interner Link)).
Good research practice includes being aware of and dealing with bias. Any potential factors that may give rise to the impression of conflicts of interest must be reported to the relevant department of the DFG Head Office (see guideline "Guidelines for Avoiding Conflicts of Interes(interner Link)").
The DFG’s philosophy states the following: “The DFG’s core task is to competitively select the best research projects undertaken by researchers at universities and research institutes and provide funding for them”. The votes obtained on the proposals are evaluated “exclusively according to scientific criteria” by reviewers working in an honorary capacity. But how does the DFG Head Office proceed when non-scientific criteria are mentioned in externally obtained reviews and these are included in the evaluation? Read mor(interner Link)
Researchers in the German research system ultimately need to adhere to the DFG Code of Conduct “Guidelines for Safeguarding Good Research Practic(externer Link)”. The guidelines set out standards for research work and research integrity addressed at the entire German research system.
Mitigating bias is central to the Code of Conduct as an expression of good research practice and is relevant in different contexts.
For research work units, processes need to be implemented that help to avoid “implicit bias” (Guideline 3). In general, precautions must be taken to mitigate bias (examples can be found in the film and Guidelines on Mitigating Bias in Scientific Evaluation and Decision-Making Processes above on this web page).
In the context of the revie(interner Link) of funding proposals, universally valid regulations for avoiding conflicts of interest ensure the neutrality of reviewers, which is central to a fair review process (Guideline 16). Any (appearance of a) conflict of interest must be disclosed. The avoidance of conflicts of interest on the part of reviewers and committee members is ensured on an individual basis by the Rules of Procedure for Dealing with Scientific Misconduc(interner Link). Any failure to disclose circumstances that could give rise to the appearance of a conflict of interes(interner Link) can be sanctioned as scientific misconduct.
Furthermore, assessing performance adequately requires drawing solely on an appropriate scope of objective assessment criteria that are adequate for scientific purposes, in addition to considering individual career paths and personal circumstances (Guideline 5).
Further Information
- NWO, Inclusive assessmen(externer Link)
(with film and guideline document on written reviews and review panels) - FWF, Unconscious Bias in Decision Proces(externer Link)
- NWO, Written assessment, 202(externer Link)
- NWO, Interaction and group dynamics in evaluation committees, 202(externer Link)
- Royal Society Understanding Unconscious Bias, 201(externer Link)
- CERCA, Recruitment Bias in Research Institutes, 201(externer Link)
- Université de Lausanne, Eviter les biais de genre lors de nominations professorales, 201(externer Link)
- Communication and perception: How easily are we influenced?, 2022
- CIHR, Online Training Modul "Bias in Peer Review(externer Link)
- Charta der Vielfalt 2022, Online-Toolbox "Anti-racism(externer Link)
- Gender Diversity Impact 2018, Gender Diversity Index – How inclusive is your team? Policy Brief 6/(externer Link)
- Trinity Center for Gender Equality and Leadership, Trinity College Dublin, Unconscious Bia(externer Link)
- Further Recommendations for Action FWF, Points for Action when Reviewing Scientific Project Application(externer Link)
- Gender Diversity Impact 2018, Implications for Research Group Leaders, Policy Brief 3/(externer Link)
- LERU 2018, Implicit bias in academia: A challenge to the meritocratic principle and to women’s careers – And what to do about i(externer Link)
Annotated bibliograph(Download) (in German only) for the DFG’s e-learning course, compiled by Melanie Steffens, Professor of Social Psychology at the Rhineland-Palatinate Technical University of Kaiserslautern-Landau, and Roland Imhoff, Professor of Social and Legal Psychology at Johannes Gutenberg University Mainz.
| E-mail: | edi@dfg.de |