Journal publication · 2023
Burton-Jones, A., & Wang, G. (2023). Editor’s comments: What makes a journal significant? From the tyranny of metrics to true impact. MIS Quarterly, 47(2), i–xiv. https://aisel.aisnet.org/misq/vol47/iss2/2/
Explore the original contribution
Interactive paper explorationStart
Aggregate citations make journals easy to compare, but hide how those citations were generated.
Reconsider
The authors' recalibrations change the rankings substantially when the sources of citations change.
Implication
A favourable ranking still does not show how a paper was used or whether its effects were desirable.
A rising Journal Impact Factor is often read as evidence that a journal's research has greater impact.
The authors' analysis of 19 IS journals shows how publisher-platform patterns and the sources of citations can change JIF rankings. Recalibrating the same indicator by its citing community changes the story; even a favourable recalibration does not establish meaningful impact.
Concept explorer
Select a concept to see its role in this paper’s contribution.
The analysis finds disproportionately strong within-platform citation patterns for Elsevier journals. This is an association, not proof of a platform's causal effect.
Separating IS from non-IS sources and higher- from lower-ranked sources substantially changes the JIF comparison.
Restricting the numerator reveals which communities contribute citations, while retaining the limitations of a citation-count measure.
The original comparative analysis connects journal-platform patterns with citation-source composition, opening a research agenda on how publication infrastructure shapes apparent journal influence.
PDF pp. 3–5; printed pp. iii–v.
MISQ ranks highest under several recalibrations, yet the authors explicitly reject treating this as evidence of true impact. The analytical result and the evaluative inference are kept separate.
PDF p. 5; printed p. v.
Editors and research leaders can inspect citation sources and substantive use before interpreting aggregate changes as progress toward their intended audience.
PDF pp. 4–5; printed pp. iv–v.
Use metrics to inform learning, alongside community judgment and evidence of effects within and beyond academia, rather than making a ranking the objective.
PDF pp. 5–6 and 11; printed pp. v–vi and xi. This applies the editorial's argument; its stakeholder vision builds on earlier work.
Why this matters
As digital discovery platforms shape what researchers encounter, understanding the provenance and meaning of citations matters alongside counting them.
Discuss this researchAn editorial, not a newly validated theory of impact. Its citation analysis is exploratory and associative. General critiques of metrics and MISQ's stakeholder vision predate this paper; they are not presented here as inventions of the authors.
Nineteen journals, five-year JIFs, platform patterns and citation-source recalibrations; causal inference explicitly limited.
Recalibration is not impact; formative rather than target-driven use of metrics.
This exploration is based on the author-supplied full text. It distinguishes the paper’s own contribution from established concepts it draws on. For formal citation, consult the publication.