The rise of citational justice: how scholars are making references fairer

Christen Smith was at a conference in October 2017 when she felt a familiar jolt of frustration. A presenter showed a slide with passages that had been paraphrased from one of her books — and, to her dismay, had failed to credit her.

Smith, an anthropologist at the University of Texas at Austin, decided not to call the speaker out on the spot. She had learnt from experience that such actions were typically met with retaliation. But this wasn’t the first time that Smith had seen her work used without proper citation, and she was fed up. “It was really the straw that broke the camel’s back,” Smith says. “I remember calling one of my best friends and telling them what happened and saying, ‘The next time I go to a conference, I’m gonna wear shirts that say, ‘Cite Black Women, period’, and dare people to do this to me again.’”

In November that year, Smith arrived at the conference of the National Women’s Studies Association (NWSA) in Baltimore, Maryland, with a pile of T-shirts that displayed “Cite Black Women.” in large, bold letters across the front. The shirts were a huge success — they sold out at both the NWSA and the American Anthropological Association meeting two weeks later. “What I discovered through that experience was that I wasn’t alone,” Smith says. “There were so many Black women who had very similar experiences — that’s why everybody recognized themselves in those T-shirts.”

From there, the effort gathered steam. Smith took the movement to social media, creating Twitter, Instagram and Facebook accounts to continue the discussions that had started with the T-shirts. That has now grown into the Cite Black Women collective, which includes a blog, website and podcast. “What Cite Black Women really did was encourage and popularize a conversation that was already happening within academic communities, especially among women of colour and Black women, around citational erasure,” Smith says. “Ultimately, we want to create a conversation that encourages people to actually stop and think about their citational politics, the choices that they’re making and the consequences of those choices.”

Citations are not just a way to acknowledge a person’s contributions to research. Because funders and universities commonly consider citation metrics when making decisions about grants, hiring and promotions, citations can have a significant impact on a scholar’s career, says Cassidy Sugimoto, an information scientist at the Georgia Institute of Technology in Atlanta. “Citations, in many ways, are the currency of the academic market.”

The giant plan to track diversity in research journals

Yet studies in bibliometrics have revealed persistent biases in citation patterns — women and people of colour, for instance, garner citations at lower rates than men do. An increasing number of researchers are calling on academics to acknowledge the inequities in citational practices — and, by paying more heed to work from groups that are typically under-cited, take action to reduce them. Some are referring to this idea as ‘citational ethics’ or ‘citational justice’. Initiatives include computer code that helps academics to estimate the balances of gender and race in their papers’ reference lists, a push for ‘citation diversity statements’ in research papers, and websites dedicated to highlighting papers from under-recognized groups. Journals, too, have started to take action, with some introducing guidance and tools for authors to highlight and address citational inequities in their own papers.

These ideas have critics, but many say that such a reckoning is long overdue — both for scholars whose works have been under-recognized, and for the broader benefits to academia.

“To me, citational justice isn’t only about justice,” Sugimoto says. “It’s about doing robust, rigorous science, where you are truly exploring all the potential areas of research and what has been conducted before to accelerate the progress of science.”

For many decades, scholars have noted the uneven nature of recognition and credit in academia. The ‘Matthew effect’, which describes the snowballing advantage that accrues to scholars who are already successful, was popularized in the 1960s. And in the 1990s, the ‘Matilda effect’ was coined to describe the phenomenon in which women’s contributions were undervalued, or attributed to men1.

Over the past decade or so, bibliometric assessments have shown how citation rates for men are, on average, higher than those for women across a wide range of fields, including economics2, astronomy3, neuroscience4 and physics5 — even when controlling for other factors that might influence citations, such as author seniority, or the year or the journal in which a paper is published (see ‘Overcited, undercited’). Men also cite their own work more often than women do6. A gap exists among racial and ethnic categories, too, with white scholars being cited at higher rates than people of colour in several disciplines7.

Source: Ref. 4

“We have really strong evidence that women are undercited and that people of colour are undercited,” says Perry Zurn, a political philosopher at American University in Washington DC.

Researchers have also shown that there are cumulative disadvantages for people who are part of more than one under-represented group. In a study published earlier this year8, Sugimoto and her colleagues examined citations in more than 5 million articles published by US-affiliated first authors between 2008 and 2019. The team inferred these authors’ gender and race or ethnic origin from their names, using data from sources such as the US Census to assign each name a probability distribution of belonging to different categories. (This method has limitations, the researchers note, but is useful for looking broadly across groups to identify disparities.)

The results were striking, especially for people belonging to what the researchers call minoritized groups, such as women or Black and Latinx individuals. Not only were these authors more likely to publish on less-cited topics — such as racial discrimination, gender-based violence and immigrants — but, even within those topics, their publications were less likely to be referenced than was work from other authors. White and Asian authors were over-represented among citations, whereas Black and Latinx authors were under-represented. Across all racial groups, women were less cited than men8 (see ‘Citation inequities’).

Source: Ref. 8

To Thema Monroe-White, who specializes in technology, entrepreneurship and data analytics at Berry College in Mount Berry, Georgia, and a co-author of the study, these findings were not a surprise. They confirmed her own experience as a Black woman in academia. Still, the degree to which the stratification mirrored historical biases in the United States “was astonishing”, Monroe-White says. “It further validates the social constructed nature of our institutions — if everything wasn’t so socially constructed, you would not see this very clear delineation from white males all the way down to Black women.” The team has since launched a website to raise further awareness about these intersectional inequalities (see https://sciencebias.uni.lu/app).

There are also indications that, even as fields become more diverse, such gaps are growing rather than shrinking, notes Dani Bassett, a physicist at the University of Pennsylvania in Philadelphia. For example, a 2020 study4 by Bassett and their colleagues — which examined reference lists in papers published between 1995 and 2018 in the top five journals in neuroscience — suggested that, although the proportion of papers with women as a first or last author was slowly increasing with time, the citation gap was growing larger.

Thema Monroe-White (left, with researcher Rosa Brenes Rodriguez) is part of a team that has launched a website to raise awareness about intersectional inequalities.Credit: Matthew McConnell/Berry College

There is evidence that these citational disparities are not related to the quality of the work. In a 2018 analysis9, Béatrice Milard, a sociologist at the University of Toulouse in France, and her colleague Ludovic Tanguy, a computational linguist, reported that authors tend to cite people they know, such as co-workers, colleagues or those with whom they have had professional interactions. In addition, Bassett and their colleagues have found that men more often cite other men, and white authors more often cite other white authors — and that this behaviour was partly explained by their tendency to co-author papers with individuals of the same gender4 or race7.

Scholars also use convenient heuristics when finding sources to cite, such as the language a paper is written in, the researcher’s institution and journal reputation, Sugimoto says. Such practices can inadvertently silence certain voices — because individuals from some populations are more likely to write in a particular language and less likely to work at prestigious institutions or publish in high-impact journals. Other biases in the publication process, and in academia more broadly, also contribute. For example, papers that come out of international collaborations are cited more often than are those from domestic collaborations, but women are less likely to hold leading positions in the former, Sugimoto notes. “You see these compound disadvantages just by looking through the lens of gender in each of these spaces.”

Little is known about any citation-rate influence for scientists from sexual and gender minorities, or for those who have disabilities, says Zurn, largely because of a lack of data. “But I would hypothesize that there are imbalances there as well,” he says.

Geography could play a part, too: citational biases might contribute to under-acknowledgement of works from authors in certain countries in both the global north and the global south, says Neha Kumar, who studies human-centred computing and global development at the Georgia Institute of Technology. “Even scholars in, for instance, Australia, Japan and parts of Europe feel that scholars in the United States are cited much more than they are,” Kumar says.

Over the past few years, many researchers have launched efforts to raise awareness of citation biases and to mitigate them — and some scholarly journals have taken action. One of the exciting things about citational justice is that “every researcher has an opportunity to contribute”, Bassett says.

Since publishing their assessment of references in neuroscience journals in 2020, Bassett, Zurn and their colleagues have developed several tools to help academics to examine representation patterns in their own references. They have written code that scholars can run on their papers to evaluate the gender and race of the authors in their reference lists (see go.nature.com/3qqdo3j). This assigns probabilistic assessments in a similar way to Sugimoto and Monroe-White’s approach. (The code assigns only ‘white’ or ‘non-white’ for race.) And they have developed a browser plug-in that adds gender predictions to first and last author names for search results on Google Scholar or PubMed.

The researchers have also advocated for the inclusion of a citation diversity statement10 in scholarly articles to raise awareness about citation inequalities, and for papers to include the proportions of citations in terms of the gender and race or ethnicity of the referenced authors. So far, a Nature analysis suggests, 91 papers in more than 50 journals and preprint servers in the past two years have included such statements; Bassett is an author of nearly half of them.

Physicist Dani Bassett (left) and political philosopher Perry Zurn, who study citation inequities.Credit: Tony & Tracy Wood Photography/Perry Zurn

Others have developed tools to increase the visibility of papers by authors from groups whose works have been undercited. Jakita Thomas, a computer scientist at Auburn University in Alabama, and her colleagues created the CiteHER Bibliography, a database of work by Black women in computing (see go.nature.com/35pwqzt). This effort is part of blackcomputeHER, an organization dedicated to supporting Black women in the fields of computing and technology. Similar communities of under-represented researchers, such as 500 Women Scientists and BlackInNeuro, have also emerged in the past few years.

There are also efforts to recognize contributions beyond conventional citations. Lorisia MacLeod, a learning-services librarian at the Alberta Library in Edmonton, Canada, devised a new type of citation in 2018 to more appropriately document and recognize oral teachings from Indigenous communities11. Standard citations for such personal communications include verbal exchanges, but generally only mention an individual’s name. As a member of the James Smith Cree Nation, one of Canada’s Indigenous groups, MacLeod was frustrated that this method did not provide the means to fully acknowledge those in Indigenous communities such as her own. “We have amazing knowledge keepers,” MacLeod says. “And yet when I look for articles about us, using our nation’s name, I pretty much just find news articles about oil spills or news articles about our death rates — and that’s not the community that I know.”

MacLeod’s template for citing oral teachings includes the person’s name, their nation or community and other applicable information, such as where they live. Since she introduced them, the templates have been included in the citation guides of some 25 institutions across Canada and the United States, MacLeod says. “I think this is an important step, because for the people who need this citation style, a sufficient template is there for them,” she says. “It’s also an important reminder to people who might not have considered this as a source that Indigenous knowledge exists, too.”

Men cite themselves more than women do

Some journals have taken notice of disparities in scholarly publishing more broadly. Many publishers have begun or are planning to ask researchers to provide information about personal characteristics, such as their gender, race or ethnicity and any disabilities, when submitting or peer-reviewing manuscripts. These efforts are aimed at understanding more about representation among authors, reviewers and editors — and could help to analyse biases in the publication process.

When it comes to citations, only a few journals have begun to take concrete action. In 2021, the Journal of Cognitive Neuroscience introduced a gender citation balance index, which uses software to track how closely gender proportions in reference lists match rates of authorship in the journal. In December, it reported that 30 author groups had used the tool12.

The same year, the publisher Cell Press (part of Elsevier) invited authors to fill in standardized ‘inclusion and diversity’ forms when they submit articles, in which scientists can share information about aspects of their research that took diversity into consideration — including gender balance in reference lists. This February, the publisher reported that 26% of author teams chose to fill in the forms; among those who did, almost one-third ticked a box stating that they “actively worked to promote gender balance in our reference list”. (A smaller proportion, 9%, chose to include a statement in their research paper built from this form.) The Biomedical Engineering Society in Landover, Maryland, also introduced an optional citation diversity statement for its journals in 2021.

Other journals have published editorials alerting readers to research on bias in citation patterns and urging authors to read more widely and aspire to more-equal gender ratios. The publisher Wiley says that it is implementing measures to raise awareness of citation bias, such as recommending that its journals adopt guidance for reviewers to focus on aspects of inclusivity, including citation diversity. Other publishers, including Springer Nature and Sage say they are considering such changes. (Nature’s news team is editorially independent of its publisher, Springer Nature).

Meanwhile, scientists are continuing with initiatives to increase the visibility of these issues — and to brainstorm other solutions. Kumar and her former student, Naveena Karusala, who is pursuing a PhD in computer science at the University of Washington in Seattle, have been working with a collective of researchers to run workshops on citational justice at conferences in their field. The group has particularly focused on meetings geared towards scholars in the global south, such as the Latin American Conference on Human–Computer Interaction. “We thought this would be a good way to bring people in, have a sustained discussion and start building a community around this idea,” Karusala says.

Neha Kumar at the Georgia Institute of Technology in Atlanta has run workshops on citational justice focused on scholars in the global south.Credit: Georgia Tech/Allison Carter

Most scientists involved in citational justice efforts say they have experienced some backlash. Bassett says that after presentations, there are always a few people who provide a mix of critical comments. Outright racist or sexist comments — such as “white people or men just write better papers” — are rare, but people often say that they just cite good science or that they don’t see race or gender. These arguments are problematic, Bassett says, because they indicate that people are not actively trying to address their own explicit and implicit biases — or that they are not willing to dig more deeply into the literature in their field to diversify their citations.

Some critics of the movement say that, although they recognize that inequities exist in citations, they are troubled by the implications of pushing to include more citations from individuals in certain groups. Some are also worried about the idea of inferring the gender or race of people in reference lists.

Among those with such concerns are Mahdi Khelfaoui, a science historian and bibliometrician at the University of Quebec at Trois-Rivières in Canada. He worries that adjusting reference lists to be more equitable could lead to an increase in perfunctory citations — nods to similar work — as opposed to real engagement with under-cited research. It’s more important to provide under-recognized scholars with access to high-prestige networks, he notes, to enable these networks to diversify. Khelfaoui also says that citational justice is, for the moment, a “very US- and Anglo-Saxon-centric issue” that is less present in, for example, French-speaking regions such as his own. “I don’t mean that there aren’t citation distribution inequalities, but they are mainly discussed in the US context because it reflects the broader social issues that are going on in that country, especially those related to racial discriminations,” he says.

Fewer citations for female authors of medical research

Khelfaoui and others, such as Milard, argue that an underlying problem is that citation counts figure so strongly in researcher assessments and grant applications. Because it is clear that cultural and social factors influence which works get cited, Milard says, the challenge is to reduce the primacy of citations as indicators of the quality of a particular researcher.

Many of those involved in citational-justice efforts agree that the current use of citational metrics needs to be overhauled. But because people do use citational metrics in evaluations, encouraging individual scholars to change their practices is a worthwhile endeavour, says Bassett. “Doing the right thing to change the number feels more important than telling people to not use the number at all,” they say.

Advocates of citational justice also emphasize that it goes beyond simply bringing more diversity into reference lists. When the Cite Black Women collective talks about citations, “we’re not just talking about bibliographic references”, Smith says. “We’re talking about acknowledgement. We’re talking about engagement. We’re talking about recognition, and the valorization of ideas.”

Just changing the proportion of authors from minoritized groups in reference lists is not enough, she adds. “If you’re only inserting Black women onto a bibliography, but you’re not actually allowing those ideas to influence the way that you think about and see the world, then what you’re engaging in is superficial diversity work, and not true, fundamental change.”

In the end, citational justice requires a multi-pronged approach, with efforts to seek equity in all parts of the scholarly communication system, Sugimoto says — from funding and peer review to hiring and the allocation of awards. Citation is “one part — but an important part — of the system that has to be done in conjunction with all of these other changes around equity and scholarly communication”.

Ultimately, elevating the work of under-represented scholars will not only benefit those individuals, but will also have broad effects in the wider population, says Monroe-White. Studies have shown, for example, that women inventors are more likely to develop products for women13 — so the more representation there is across disciplines, the more people are likely to benefit from the fruits of that scientific labour.

“If you truly believe that science would benefit from the best minds contributing to the scientific enterprise, you have to let all minds have a chance to contribute,” Monroe-White says. “If you’re not allowing all minds to contribute to the science, then you’re not getting the best.”

Nature 603, 568-571 (2022)

doi: https://doi.org/10.1038/d41586-022-00793-1