Matching Questions to Quantitative MethodologyMethodologyThe theoretical framework that informs how a researcher approaches their work and what methods are used to collect data.
Most research questions can be answered in a variety of ways. The exact details of the question may need tweaking to fit a certain method you find particularly appealing. However, most of the time there are several methodological approaches that could fit a question, depending on researcher priorities. A great library-centered roundup of short discussions of various research methodologies can be found in the Evidence Based Library and Information Practice journal‘s special issue on EBL 101.
Quantitative methodologies are the most frequent group of methods, so quantitative designs are a good place to start considering your research design. Quantitative methodology includes techniques that depend on numerical measurements and those that count things or responses. Counting or measuring are fundamental to quantitative methodology. Quantitative data collection can include usage statistics, surveys, rubric scores, numeric observations like headcounts, and more. Quantitative analysis can include complex statistics like testing for significant differences or measuring strength of association through correlation. Quantitative analysis can also include less statistically elaborate methods, such as tables of response counts, cross-tabulation through pivot charts, and graphs. Bar charts are particularly useful and flexible ways to analyze quantitative data.
If convincing a science-focused audience is a priority, then a quantitative methodology is particularly effective. In addition, quantitative methodology brings several other unique strengths:
Participant pools are generally larger, which can allow both big group insights and smaller subgroup analyses.
A caveat to this strength is that it can be costly to get access to large participant pools or to buy access to datasets.
There are ways to refine your sample so that the analyses can be generalized to represent even larger populations.
Quantitative analysis is faster than qualitative.
Documenting the impact of an intervention, such as the effect of a class, is often more precise with quantitative approaches.
It is often easier to build on or adapt existing approaches, such as using other researchers’ instruments and surveys and following through with the same analyses they published.
If those strengths sound appealing, then it is worth considering the measurement issue. Measurement is a particular challenge of fitting a research question to quantitative methodology. While the math may be considered the hardest part, often a good summary table and optionally some bar graphs will be sufficient for presenting quantitative data. Furthermore, there are many discussions available to help you choose which test to use based on the types of variables involved in your data. Therefore, before worrying about how (or whether!) to apply a statistical test, it is important to think about how to count or otherwise measure the concepts in quantitative study.
Measurement is the essential challenge of quantitative methodology
When thinking about whether a quantitative approach fits a research question, we must identify what comprises our question. We need to know the population of people or things being studied, what items we are trying to measure about that population, and if there are relationships among those items.
If you’ve taken any of the previous lessons, you’ve seen some types of data that have been discussed include surveys, local library statistics like circulation and patron interactions, large, curated datasets, text scraping, social media, citations, content analysis, and more. In some cases, the right data to measure a study may be obvious, or a piece of data may even inspire the study. But in many cases, the next step after deciding to design a survey or do text analysis is to wonder “how?”
It is challenging to identify what we want to measure. For example, a question like “What was the impact of changing our service desk layout?” requires more definition about impact and what that means and how it can be measured. Determining what we mean by impact may require going back to the earlier worksheets or reflection exercises in previous courses to understand impact on whom or to whom we need to express impacts. For example, there is a difference between measuring impacts that express the value of a service, versus measuring impact in terms of effectiveness for a majority of patrons, versus measuring impacts on specific minority patron subgroups. This question feels narrow, but it would still require us to define the most challenging measurement: impact. This is typical of a focused question that needs definition (or technically “operationalization”) of the terms. Short, vague nouns like impact, effect, inclusiveness, effectiveness, value, reactions, or perceptions often need the most added details and cannot stand alone without further definition.
Matching qualitative methods to a research question depends on whether it is possible to define exactly how to measure every concept in the research question. Consider this image of how you might brainstorm possible ways to measure the parts of this example research question:y concept in the research question. Consider this image of how you might brainstorm out possible ways to measure the parts of this example research question:
RQ: What was the impact of changing our service desk layout?
Figure 4, created by Nina Exner for LibParlor Online Learning, 2023.
A measurement may be invented by the authors based on their research question and intent. As long as the measurement approach is clearly addressed, researchers can define their concepts in a way that makes sense for their research question and context. The transparent definition of measurements is key.
Nevertheless, it is considered more valid if the measurement can be based on a documentable process related to work that has been tested or at least applied by other researchers. An existing scale or “validated” (that’s a good keyword to use in searching) inventory is a great option if one appears in the literature. However, that ideal is not always possible. Other ways to find support for measurements in the literature include borrowing surveys from related studies, finding existing rubrics that measure the concept, or identifying articles that research the subject in a broader sense (such as service impact in general) and adapting them to the specific library context.
If measurement can be nailed down specifically in a countable or numerically measurable way, the remainder of a quantitative study is less complicated. Some ways to analyze quantitative data are:
Looking at how big a measure is
Calculating the average
Examining or ranking the sizes of measurements by subgroups
Examining the change in a measure, with before and after sizes, and optionally testing for significant difference
Testing the strength of relationship between two measures by a correlation coefficient
Examining subgroups of measures by cross-tabulating counts or averaging a measure by other subsets
Keep in mind that a useful way to use the literature is by examining how other researchers have written about similar topics. Looking at articles or conference presentations measuring a similar concept and seeing write-ups that seem usable can be inspiring.
Quantitative analysis
Many of us may be intimidated by the idea of statistics because they involve complex math. However, it isn’t necessary to understand all the math behind statistics in order to use them. Ideally, we should understand the mathematical concepts because that helps us understand the limitations of a certain statistic. Yet, our first research study does not have to be the one where we learn all of the math behind statistics.
In fact, a great deal can be done with making graphs and tables of counts or averages. It is not necessary to go beyond counting and bar graphs to publish a good study! Many quantitative questions can be answered with graphs, percentages, counts, and averages. Questions that are well-suited for these include ones involving what the most or least of something is or what types of libraries or patrons use more or less than others. More specific examples may include:
What patron types use more print books than e-books?
How does traffic change on average over a year? If we break it out by patron types, do different types of patrons come during times that are otherwise quiet?
In academic libraries, who uses reference services most during academic breaks?
In public libraries, who uses children’s services most when summer reading programs are over?
Do librarians of large or small library attend more professional development webinars?
Frequencies and averages can be used for all those questions. Consider Caitlin Bakker’s intro to statistics for librarians column that touched on these topics for more. These questions can be answered using Excel, Libreoffice Calc, Google Sheets, or similar spreadsheets. Using spreadsheets can be intimidating to people who don’t use them often; however, it is likely that someone in your library or system is using them regularly. This could be a good time to reach out to Even for more complex statistics, you rarely need to understand the math of the analysis. Most statistics are done using a statistical program (SPSS, PSPP, JMP, SAS, Stata, R, etc.). Websites can be helpful when choosing which statistic is suitable for significance testing. There are resources to help you decide what types of variables you are analyzing (like Caitlin Bakker’s brief but very helpful overview of variable types). Once you know what types of variables you are using, take a look at “Choosing the correct statistical test in SAS, STATA, SPSS and R” site which helps both with choosing which statistical test suits your variables and giving you a walkthrough of that test in various software.
If you do not have a paid package like SAS, Stata, or SPSS, you can use R for free. There are also free web statistics calculators available, such as VassarStats or Statistics Kingdom. Finally, the open-source program PSPP is an excellent alternative to SPSS. Search the web for advice on using the stats test’s name plus “PSPP” as a keyword for help files.
Some of the statistical concepts you will be introduced to are complicated and require extra reading, especially if these concepts are new to you. In the next section, we’ll discuss new concepts in qualitative analysis.
Topic 1 References
Bakker, C. “An IntroductionIntroductionThe start of a research article providing background information and an overview of the research presented in the article. to Statistics for Librarians (Part Two): Frequency Distributions and Measures of Central Tendency.” Hypothesis: Research Journal for Health Information Professionals, 35, 1 (2023), Article 1. doi.org/10.18060/27162.
Bakker, C. J. “An Introduction to Statistics for Librarians (Part One): Types of Data.” Hypothesis: Research Journal for Health Information Professionals, 34, 1 (2022), Article 1. doi.org/10.18060/26428.
Bedi, Shailoo, and Jenaya Webb. “Through the Students’ Lens: Photographic Methods for Research in Library Spaces.” Evidence Based Library and Information Practice 12, no. 2 (2017): 15–35. doi.org/10.18438/B8FH33.
Matching Questions to Quantitative MethodologyMethodology The theoretical framework that informs how a researcher approaches their work and what methods are used to collect data.
Most research questions can be answered in a variety of ways. The exact details of the question may need tweaking to fit a certain method you find particularly appealing. However, most of the time there are several methodological approaches that could fit a question, depending on researcher priorities. A great library-centered roundup of short discussions of various research methodologies can be found in the Evidence Based Library and Information Practice journal‘s special issue on EBL 101.
Quantitative methodologies are the most frequent group of methods, so quantitative designs are a good place to start considering your research design. Quantitative methodology includes techniques that depend on numerical measurements and those that count things or responses. Counting or measuring are fundamental to quantitative methodology. Quantitative data collection can include usage statistics, surveys, rubric scores, numeric observations like headcounts, and more. Quantitative analysis can include complex statistics like testing for significant differences or measuring strength of association through correlation. Quantitative analysis can also include less statistically elaborate methods, such as tables of response counts, cross-tabulation through pivot charts, and graphs. Bar charts are particularly useful and flexible ways to analyze quantitative data.
If convincing a science-focused audience is a priority, then a quantitative methodology is particularly effective. In addition, quantitative methodology brings several other unique strengths:
If those strengths sound appealing, then it is worth considering the measurement issue. Measurement is a particular challenge of fitting a research question to quantitative methodology. While the math may be considered the hardest part, often a good summary table and optionally some bar graphs will be sufficient for presenting quantitative data. Furthermore, there are many discussions available to help you choose which test to use based on the types of variables involved in your data. Therefore, before worrying about how (or whether!) to apply a statistical test, it is important to think about how to count or otherwise measure the concepts in quantitative study.
Measurement is the essential challenge of quantitative methodology
When thinking about whether a quantitative approach fits a research question, we must identify what comprises our question. We need to know the population of people or things being studied, what items we are trying to measure about that population, and if there are relationships among those items.
If you’ve taken any of the previous lessons, you’ve seen some types of data that have been discussed include surveys, local library statistics like circulation and patron interactions, large, curated datasets, text scraping, social media, citations, content analysis, and more. In some cases, the right data to measure a study may be obvious, or a piece of data may even inspire the study. But in many cases, the next step after deciding to design a survey or do text analysis is to wonder “how?”
It is challenging to identify what we want to measure. For example, a question like “What was the impact of changing our service desk layout?” requires more definition about impact and what that means and how it can be measured. Determining what we mean by impact may require going back to the earlier worksheets or reflection exercises in previous courses to understand impact on whom or to whom we need to express impacts. For example, there is a difference between measuring impacts that express the value of a service, versus measuring impact in terms of effectiveness for a majority of patrons, versus measuring impacts on specific minority patron subgroups. This question feels narrow, but it would still require us to define the most challenging measurement: impact. This is typical of a focused question that needs definition (or technically “operationalization”) of the terms. Short, vague nouns like impact, effect, inclusiveness, effectiveness, value, reactions, or perceptions often need the most added details and cannot stand alone without further definition.
Matching qualitative methods to a research question depends on whether it is possible to define exactly how to measure every concept in the research question. Consider this image of how you might brainstorm possible ways to measure the parts of this example research question:y concept in the research question. Consider this image of how you might brainstorm out possible ways to measure the parts of this example research question:
A measurement may be invented by the authors based on their research question and intent. As long as the measurement approach is clearly addressed, researchers can define their concepts in a way that makes sense for their research question and context. The transparent definition of measurements is key.
Nevertheless, it is considered more valid if the measurement can be based on a documentable process related to work that has been tested or at least applied by other researchers. An existing scale or “validated” (that’s a good keyword to use in searching) inventory is a great option if one appears in the literature. However, that ideal is not always possible. Other ways to find support for measurements in the literature include borrowing surveys from related studies, finding existing rubrics that measure the concept, or identifying articles that research the subject in a broader sense (such as service impact in general) and adapting them to the specific library context.
If measurement can be nailed down specifically in a countable or numerically measurable way, the remainder of a quantitative study is less complicated. Some ways to analyze quantitative data are:
Keep in mind that a useful way to use the literature is by examining how other researchers have written about similar topics. Looking at articles or conference presentations measuring a similar concept and seeing write-ups that seem usable can be inspiring.
Quantitative analysis
Many of us may be intimidated by the idea of statistics because they involve complex math. However, it isn’t necessary to understand all the math behind statistics in order to use them. Ideally, we should understand the mathematical concepts because that helps us understand the limitations of a certain statistic. Yet, our first research study does not have to be the one where we learn all of the math behind statistics.
In fact, a great deal can be done with making graphs and tables of counts or averages. It is not necessary to go beyond counting and bar graphs to publish a good study! Many quantitative questions can be answered with graphs, percentages, counts, and averages. Questions that are well-suited for these include ones involving what the most or least of something is or what types of libraries or patrons use more or less than others. More specific examples may include:
Frequencies and averages can be used for all those questions. Consider Caitlin Bakker’s intro to statistics for librarians column that touched on these topics for more. These questions can be answered using Excel, Libreoffice Calc, Google Sheets, or similar spreadsheets. Using spreadsheets can be intimidating to people who don’t use them often; however, it is likely that someone in your library or system is using them regularly. This could be a good time to reach out to Even for more complex statistics, you rarely need to understand the math of the analysis. Most statistics are done using a statistical program (SPSS, PSPP, JMP, SAS, Stata, R, etc.). Websites can be helpful when choosing which statistic is suitable for significance testing. There are resources to help you decide what types of variables you are analyzing (like Caitlin Bakker’s brief but very helpful overview of variable types). Once you know what types of variables you are using, take a look at “Choosing the correct statistical test in SAS, STATA, SPSS and R” site which helps both with choosing which statistical test suits your variables and giving you a walkthrough of that test in various software.
If you do not have a paid package like SAS, Stata, or SPSS, you can use R for free. There are also free web statistics calculators available, such as VassarStats or Statistics Kingdom. Finally, the open-source program PSPP is an excellent alternative to SPSS. Search the web for advice on using the stats test’s name plus “PSPP” as a keyword for help files.
Some of the statistical concepts you will be introduced to are complicated and require extra reading, especially if these concepts are new to you. In the next section, we’ll discuss new concepts in qualitative analysis.
Topic 1 References
Bakker, C. “An IntroductionIntroduction The start of a research article providing background information and an overview of the research presented in the article. to Statistics for Librarians (Part Two): Frequency Distributions and Measures of Central Tendency.” Hypothesis: Research Journal for Health Information Professionals, 35, 1 (2023), Article 1. doi.org/10.18060/27162.
Bakker, C. J. “An Introduction to Statistics for Librarians (Part One): Types of Data.” Hypothesis: Research Journal for Health Information Professionals, 34, 1 (2022), Article 1. doi.org/10.18060/26428.
Bedi, Shailoo, and Jenaya Webb. “Through the Students’ Lens: Photographic Methods for Research in Library Spaces.” Evidence Based Library and Information Practice 12, no. 2 (2017): 15–35. doi.org/10.18438/B8FH33.
Choosing the Correct Statistical Test in SAS, Stata, SPSS and R. (n.d.). Accessed July 31, 2023,https://stats.oarc.ucla.edu/other/mult-pkg/whatstat/.
Wilson, V. EBL 101 Special Issue. Evidence Based Library and Information Practice, 11, 1 (2016): page number. https://journals.library.ualberta.ca/eblip/index.php/EBLIP/issue/view/1571.