Citation counts and author-level citation-based metrics
Let’s return to that basic building block of research impactResearch impactA way to describe and measure the ways in which research causes some sort of positive impact on a community. Research impact can be measured through quantitative or qualitative data.
measurement, the citation count. As mentioned previously, several databases track comprehensive citation counts, although many other platforms track for a more limited range of resources (such as subject-specific databases). Two of these sources require a subscription to access: Web of Science, and Scopus. We will focus on two other sources that can be freely accessed: Google Scholar and Dimensions (though for Dimensions, a subscription is needed for some advanced analysis features).
Before we do that, however, we need to discuss the appropriateness of citation-based metrics for academic librarians. A 2022 study by Borchardt, Bruce, Click, and Roh found uneven indexing of academic librarianship journals, meaning that journal-level metrics relying on database indexing will always be fairly incomplete when it comes to reflecting the academic librarianship literature. The two subscription-based resources, Web of Science and Scopus, each index roughly half of the academic literature journals identified in the study. On the other hand, Google Scholar has more complete indexing but also has the most basic journal-level metric. Journal-level metrics measure the impact, reach, or prestige of a journal. They are designed to measure the aggregate impact of a publication as a whole. This is one reason why journal-level metrics are not generally recommended as an accurate reflection of academic librarian research, though as we’ll discuss later, institutional expectations do not always reflect this reality.
Turning now to citations, we start with Google Scholar, a major tool that is widely used, particularly for social science and humanities researchers. Its coverage is the most comprehensive but also contains the widest range of peer-reviewed and other sources, including journal articles, conference proceedings, books and book chapters, theses and dissertations, preprints, bibliographies, and more. As you might expect, this means that Google Scholar’s citation counts are routinely higher than in other resources. In librarianship, this difference is often significant due to the lack of standardized indexing for library research outputs. While you can look up individual outputs in Google Scholar, creating a Google Scholar profile is an efficient way to both gather relevant scholarly impact metrics and lead others (including evaluators) to the full body of your research.
Dimensions, a Digital Science database, primarily indexes journal articles, although it also contains more limited indexing of books and book chapters, proceedings, preprints, and datasets. As with Google Scholar, individual publications can be searched for to find citation counts, and many researchers also have their own research profile. These profiles have a similar graph showing both publications and citation counts over time and can be downloaded as a PNG, JPG, PDF, or as a CSV or XLS file to make a customized chart or graph. The profile also contains some advanced data, such as Fields of Research and Concepts. However, as noted earlier, many librarians will find their research profile either missing or heavily incomplete compared to Google Scholar. Finally, it’s worth noting that Dimensions does not include any author-level metrics like the H-index or i10-index, though they can be calculated relatively easily as needed.
Once created, you can collect citation counts for individual research published and some author-level metrics. You should be able to see the sum of citations across all outputs and a graph that shows the distribution of counts by year — this graph can be a great way to show whether your citations are “on the rise,” indicating that your scholarly impact is increasing. Other metrics are also included, but these are not commonly used in librarianship, including the H-index, a metric designed to measure the overall impact and productivity of researchers with large written portfolios, particularly in STEM fields. We’ll discuss the H-index again later. The i10-index, the other author-level metric Google Scholar provides, is also rarely used — it is simply the number of publications with 10 or more citations.
An H-index matrix, Wikimedia.
Journal-level metrics
In the world of scholarly metrics, article-level metrics are often presented alongside journal-level metrics, at least for many disciplines. Journal metrics are designed to measure the prestige or relative impact of a journal within a subject or field. We’ll discuss two tools for finding these metrics.
Google Scholar contains a separate database, Google Scholar Metrics, which can be found in Google Scholar’s menu of options in the top-left corner. By clicking on the magnifying glass at the top, you can search for individual publications — if the journal has been indexed, it will give you two journal-level metrics: H5-index and H5-median.
The H5-index is the same formula as discussed for the author-level H-index metric but applied to a journal within a 5-year period. So, an H5-index number of 8 means that the journal has published 8 articles in the past 5 years that have each received 8 or more citations. The H5-median calculates the median (or middle) number of citations among those (in this case, 8) articles.
However, the main usefulness of any journal-level metric is in its comparison. So, H5-index metrics can be directly compared, as long as they’re for journals similar in scope. Most journal-level metrics should not be compared across disciplines or even subdisciplines because citation practices vary widely. For example, citation rates in information science are routinely higher than in library science but are often combined, as we’ll see soon. Google Scholar does some limited rankings — if a journal is in the top 20 for any subject ranking, its rank will be listed. Otherwise, directly comparison H5-index metrics is the only way to contextualize these numbers.
The list of top 20 journals in a subject category can be found from the Google Scholar Metrics main page by clicking on “View All” under “Top Publications.” This will bring up the “Categories” drop-down menu. Library & Information Science is a subcategory under Social Sciences.
Another resource is SCImago Journal Rank, which publishes journal rankings based on Scopus indexing. These metrics are freely available, and all indexed journals are also ranked within one or more subjects. From the journal ranking homepage, search for a specific journal or pull up a subject — as with Google Scholar, Library and Information Sciences is a subcategory of Social Sciences. The primary metric is SJR, or SCImago Journal Rank, which uses a complicated formula to weight citation counts for the previous three years, which, like the H5-index, can be directly compared with other similar journals. Finding subject rankings is a bit tricky — individual journal profiles will give you the SJR, and then you can browse through the appropriate category (like Library and Information Sciences) to find the journal. SJR also ranks journals by their quartile (or 25th percentile) within their subject field — Q1 means the journal ranks in the top 25% of the field; Q2 is in the 26-50% percentage; and so on. SCImago also has a fascinating “Viz Tools” section, including Compare Journals, which is an easy way to graphically compare SJR between multiple journals over time, although, like Google Scholar’s graph, the result must be screenshotted or recreated to share — there is no download option.
A visualization tool from SCImago comparing the metrics of three LISLibrary and Information ScienceAn interdisciplinary field that examines how physical and digital information is organized, accessed, collected, managed, disseminated and used, particularly in library settings. journals.
Finally, one more amazing tool, for those who need more bespoke metrics is Publish or Perish (PoP). This tool calculates various metrics based on a variety of data sources, including Google Scholar, OpenAlex, CrossRef, or even a citation software export. For example, if you cannot locate a journal-level metric, you can calculate an H5-index using this tool, as well as H-index variations for researcher-level metrics. If a Google Scholar profile isn’t an option, PoP can also recreate the metrics included in those profiles.
A note about journal-level metrics: two other commonly used journal-level metrics in subscription databases are Impact Factor, available in Clarivate’s Journal Citation Reports based on Web of Science indexing, and CiteScore, available in Scopus. However, many journals will include one or both metrics on their homepage, so the metric can often be found, though the ranking is generally helpful for contextualizing both metrics.
Note that the average length of time it takes for 10 of the top librarianship journal articles to reach half of their total citations is more than 7 years. That means if your article is less than 7 years old, the citation rate is likely still increasing — cold comfort if you’re being evaluated sooner than that, but it is a useful statistic to point to. In other words, there’s reason to believe that your citation count will continue to increase for articles greater than 7 years old.
Topic 2 References
Borchardt, Rachel, Symphony, Bruce, Click, Amanda, and Charlotte Roh. “Are We Walking the Talk? A Snapshot of How Academic LIS Journals Are (or Aren’t) Enacting Disciplinary Values.” In the Library with the Lead Pipe, January 2022. https://www.inthelibrarywiththeleadpipe.org/2022/walking-the-talk/.
Huang, Mu-hsuan, Wang-Ching Shaw, and Chi-Shiou Lin. “One Category, Two Communities: Subfield Differences in ‘Information Science and Library Science’ in Journal Citation Reports.” Scientometrics 119 (2): 1059–79. doi.org/10.1007/s11192-019-03074-3.
Citation-Based Metrics and Scholarly Impact
Citation counts and author-level citation-based metrics
Let’s return to that basic building block of research impactResearch impact A way to describe and measure the ways in which research causes some sort of positive impact on a community. Research impact can be measured through quantitative or qualitative data. measurement, the citation count. As mentioned previously, several databases track comprehensive citation counts, although many other platforms track for a more limited range of resources (such as subject-specific databases). Two of these sources require a subscription to access: Web of Science, and Scopus. We will focus on two other sources that can be freely accessed: Google Scholar and Dimensions (though for Dimensions, a subscription is needed for some advanced analysis features).
Before we do that, however, we need to discuss the appropriateness of citation-based metrics for academic librarians. A 2022 study by Borchardt, Bruce, Click, and Roh found uneven indexing of academic librarianship journals, meaning that journal-level metrics relying on database indexing will always be fairly incomplete when it comes to reflecting the academic librarianship literature. The two subscription-based resources, Web of Science and Scopus, each index roughly half of the academic literature journals identified in the study. On the other hand, Google Scholar has more complete indexing but also has the most basic journal-level metric. Journal-level metrics measure the impact, reach, or prestige of a journal. They are designed to measure the aggregate impact of a publication as a whole. This is one reason why journal-level metrics are not generally recommended as an accurate reflection of academic librarian research, though as we’ll discuss later, institutional expectations do not always reflect this reality.
Turning now to citations, we start with Google Scholar, a major tool that is widely used, particularly for social science and humanities researchers. Its coverage is the most comprehensive but also contains the widest range of peer-reviewed and other sources, including journal articles, conference proceedings, books and book chapters, theses and dissertations, preprints, bibliographies, and more. As you might expect, this means that Google Scholar’s citation counts are routinely higher than in other resources. In librarianship, this difference is often significant due to the lack of standardized indexing for library research outputs. While you can look up individual outputs in Google Scholar, creating a Google Scholar profile is an efficient way to both gather relevant scholarly impact metrics and lead others (including evaluators) to the full body of your research.
Dimensions, a Digital Science database, primarily indexes journal articles, although it also contains more limited indexing of books and book chapters, proceedings, preprints, and datasets. As with Google Scholar, individual publications can be searched for to find citation counts, and many researchers also have their own research profile. These profiles have a similar graph showing both publications and citation counts over time and can be downloaded as a PNG, JPG, PDF, or as a CSV or XLS file to make a customized chart or graph. The profile also contains some advanced data, such as Fields of Research and Concepts. However, as noted earlier, many librarians will find their research profile either missing or heavily incomplete compared to Google Scholar. Finally, it’s worth noting that Dimensions does not include any author-level metrics like the H-index or i10-index, though they can be calculated relatively easily as needed.
Look up a scholar you admire on Google Scholar and look at their profile. Then, if you haven’t already, create your own Google Scholar profile.
Once created, you can collect citation counts for individual research published and some author-level metrics. You should be able to see the sum of citations across all outputs and a graph that shows the distribution of counts by year — this graph can be a great way to show whether your citations are “on the rise,” indicating that your scholarly impact is increasing. Other metrics are also included, but these are not commonly used in librarianship, including the H-index, a metric designed to measure the overall impact and productivity of researchers with large written portfolios, particularly in STEM fields. We’ll discuss the H-index again later. The i10-index, the other author-level metric Google Scholar provides, is also rarely used — it is simply the number of publications with 10 or more citations.
Journal-level metrics
In the world of scholarly metrics, article-level metrics are often presented alongside journal-level metrics, at least for many disciplines. Journal metrics are designed to measure the prestige or relative impact of a journal within a subject or field. We’ll discuss two tools for finding these metrics.
Google Scholar contains a separate database, Google Scholar Metrics, which can be found in Google Scholar’s menu of options in the top-left corner. By clicking on the magnifying glass at the top, you can search for individual publications — if the journal has been indexed, it will give you two journal-level metrics: H5-index and H5-median.
The H5-index is the same formula as discussed for the author-level H-index metric but applied to a journal within a 5-year period. So, an H5-index number of 8 means that the journal has published 8 articles in the past 5 years that have each received 8 or more citations. The H5-median calculates the median (or middle) number of citations among those (in this case, 8) articles.
However, the main usefulness of any journal-level metric is in its comparison. So, H5-index metrics can be directly compared, as long as they’re for journals similar in scope. Most journal-level metrics should not be compared across disciplines or even subdisciplines because citation practices vary widely. For example, citation rates in information science are routinely higher than in library science but are often combined, as we’ll see soon. Google Scholar does some limited rankings — if a journal is in the top 20 for any subject ranking, its rank will be listed. Otherwise, directly comparison H5-index metrics is the only way to contextualize these numbers.
The list of top 20 journals in a subject category can be found from the Google Scholar Metrics main page by clicking on “View All” under “Top Publications.” This will bring up the “Categories” drop-down menu. Library & Information Science is a subcategory under Social Sciences.
Another resource is SCImago Journal Rank, which publishes journal rankings based on Scopus indexing. These metrics are freely available, and all indexed journals are also ranked within one or more subjects. From the journal ranking homepage, search for a specific journal or pull up a subject — as with Google Scholar, Library and Information Sciences is a subcategory of Social Sciences. The primary metric is SJR, or SCImago Journal Rank, which uses a complicated formula to weight citation counts for the previous three years, which, like the H5-index, can be directly compared with other similar journals. Finding subject rankings is a bit tricky — individual journal profiles will give you the SJR, and then you can browse through the appropriate category (like Library and Information Sciences) to find the journal. SJR also ranks journals by their quartile (or 25th percentile) within their subject field — Q1 means the journal ranks in the top 25% of the field; Q2 is in the 26-50% percentage; and so on. SCImago also has a fascinating “Viz Tools” section, including Compare Journals, which is an easy way to graphically compare SJR between multiple journals over time, although, like Google Scholar’s graph, the result must be screenshotted or recreated to share — there is no download option.
Finally, one more amazing tool, for those who need more bespoke metrics is Publish or Perish (PoP). This tool calculates various metrics based on a variety of data sources, including Google Scholar, OpenAlex, CrossRef, or even a citation software export. For example, if you cannot locate a journal-level metric, you can calculate an H5-index using this tool, as well as H-index variations for researcher-level metrics. If a Google Scholar profile isn’t an option, PoP can also recreate the metrics included in those profiles.
A note about journal-level metrics: two other commonly used journal-level metrics in subscription databases are Impact Factor, available in Clarivate’s Journal Citation Reports based on Web of Science indexing, and CiteScore, available in Scopus. However, many journals will include one or both metrics on their homepage, so the metric can often be found, though the ranking is generally helpful for contextualizing both metrics.
Note that the average length of time it takes for 10 of the top librarianship journal articles to reach half of their total citations is more than 7 years. That means if your article is less than 7 years old, the citation rate is likely still increasing — cold comfort if you’re being evaluated sooner than that, but it is a useful statistic to point to. In other words, there’s reason to believe that your citation count will continue to increase for articles greater than 7 years old.
Topic 2 References
Borchardt, Rachel, Symphony, Bruce, Click, Amanda, and Charlotte Roh. “Are We Walking the Talk? A Snapshot of How Academic LIS Journals Are (or Aren’t) Enacting Disciplinary Values.” In the Library with the Lead Pipe, January 2022. https://www.inthelibrarywiththeleadpipe.org/2022/walking-the-talk/.
Huang, Mu-hsuan, Wang-Ching Shaw, and Chi-Shiou Lin. “One Category, Two Communities: Subfield Differences in ‘Information Science and Library Science’ in Journal Citation Reports.” Scientometrics 119 (2): 1059–79. doi.org/10.1007/s11192-019-03074-3.