Quantitative Reasoning:
An Overview

Marcia Davidson and Gary McKinney
Western Washington University

WHAT IS QUANTITATIVE REASONING?

In 1997, the Washington State Legislature directed the Higher Education Coordinating Board (HEC Board) to implement a budget-based accountability system. From this directive, four assessment initiatives were developed. Two of these - writing and critical thinking - are familiar concepts to most educators. Two others - information technology literacy and quantitative reasoning - may be relatively new concepts, or at least relatively new terminologies. The May, 2000, Dialogue (Issue No. 6) presented an overview of information technology literacy. In this issue, an overview of quantitative reasoning will be presented.

Often, quantitative reasoning (QR) is assumed to be synonymous with mathematics, and, indeed, the two are inextricably linked. Yet there are differences, one of which is that while mathematics is primarily a discipline, QR is a skill, one with practical applications. A mathematician might take joy in abstraction, but the well-educated citizen can apply QR skills to daily contexts: for instance, understanding the power of compound interest or the uses and abuses of percentages; using fundamental statistical analysis to gauge the accuracy of a statistical study; or applying the principles of logic and rhetoric to real world arguments.

Moreover, while mathematics is often exclusive, frequently with a language of its own, QR is inclusive, its language plain and everyday. In our information-rich - some might say information-overloaded - society, QR skills are especially important. We may no longer need to perform quantitative calculations by hand, but we do need to interpret them and judge their accuracy. Few people are trained to work with complex mathematical concepts, but all educated citizens should be able to understand mathematics well enough to develop informed opinions about quantitative concepts.

To illustrate the point, here are some test questions taken from a freshman Quantitative Reasoning Study Packet at Wellesley College. Answering them requires quantitative skills that most educators would agree all educated citizens should possess.1

[top]

WHY IS QUANTITATIVE REASONING IMPORTANT TO ASSESS?

QR is a state-mandated accountability measure

While arguably not the most important reason to address QR as a component of a complete education, it is one of four state-mandated student learning outcomes. (As mentioned, writing, critical thinking, and information technological literacy are the others.) Western Washington University is leading the state effort in developing an assessment of student learning in quantitative reasoning. We will be developing a plan for assessing QR on our campus and will provide a progress report to the Higher Education Coordinating Board later this year.

QR is a student learning outcome

[top]

WHAT SHOULD A BASIC COMPETENCY IN QR INCLUDE?

According to the Mathematical Association of America (MAA)3 , the following quantitative literacy (or QR) requirements should be established for all students who receive a bachelor's degree:

• Interpret mathematical models such as formulas, graphs, table, and schematics, and draw inferences from them.
  
• Represent mathematical information symbolically, visually, numerically, and verbally.
  
• Use arithmetical, algebraic, geometric, and statistical methods to solve problems.
  
• Estimate and check answers to mathematical problems in order to determine reasonableness, identify alternatives, and select optimal results.
  
• Recognize that mathematical and statistical methods have limits.
  

Another example of what QR competency might look like is found in Wellesley College's "Quantitative Reasoning Requirement". At Wellesley, all freshmen are required to take a QR placement test. If they don't meet minimum standards, they must enroll in QR 140, a course that brings them up to competency. Once they have completed QR 140 (of if they have already passed the QR placement test) students must, at some point in their academic career, take a QR overlay course, designed to "engage students in the analysis and interpretation of data in a scientific or social context."4 The overlay course is intended to provide students with a basic understanding of important ways that numerical data are used in problem solving. Overlay courses are offered in the humanities, sciences, and social sciences. They have five basic goals (note that these goals echo the Quantitative Literacy requirements established by the MAA).

• LITERACY: Topics and depth of coverage enough so that students have the knowledge they need to function in real-life situations involving quantitative data.
  
• AUTHENTICITY: Students use authentic numerical data whenever possible. The experience should arise naturally from the context of the course.
  
• APPLICABILITY: Examples should be adequate to convince students that the methods of analysis they learn are of general applicability and usefulness.
  
• UNDERSTANDING: It is important that student learning go beyond rote application. They should be able to recognize when they can apply what they have learned in the future.
  
• PRACTICALITY: Breadth and depth of topics should be consistent with reasonable expectations of students when data analysis is only part of a course requirement.

[top]

WHAT ABOUT A MINIMUM QR COMPETENCE AT COLLEGE ENTRANCE?

Just as we already use test scores to establish the abilities of in-coming students - for example, from the SAT and Math Placement Test - we should be taking a measure of students' QR abilities. As mentioned earlier, the connection between mathematics and QR is close, yet also different - in all likelihood, different enough to warrant taking stock. An additional test, however, does not necessarily mean subjecting students to more tests; a slight remolding of the existing testing framework would also work. Students anticipating they will major in science or mathematics could continue to take the Math Placement Test (MPT); students anticipating they will major in non-science or mathematics could take a Quantitative Reasoning Test (QRT).

Alternatively, all students, other than those excepted under existing guidelines, could take a hybrid test that included elements of the existing MPT, plus additional QR-related questions. Regardless of the direction decided upon, new test questions - or possibly an entirely new test - would need to be developed.5 This logistical concern would need to be factored into any changes to the current system. Most importantly, any revamped, or entirely brand-new tests would need to reflect the anticipated curricular changes.

[top]

IMPACT OF A QR TEST ON CURRENT SYSTEM

Logistically, the idea of two tests - MPT and QRT - would create a fork: one leading to mathematics-intensive courses and/or majors, the other leading to less mathematics-intensive courses and majors. In circumstances where a student took the QRT, then later decided to major in science or mathematics, the departments affected could either accept the QRT results, or require the student to take the MPT. Even if a second test were required, the amount of testing for such students would have, practically speaking, a minimum impact of current testing logistics.

The idea of an integrated mathematics/QR test would not lead so obviously to a fork and has the rhetorical advantage of creating the sense of a more integrated curriculum, a sense that the University values equally science and humanities, mathematics and quantitative reasoning.

Whichever test were developed and implemented, the next decision to make would be what to do with the scores - which would depend entirely on the changes in curricular policy that are adopted. If no curricular policy change is anticipated, then of course the whole question of even having a test is moot.

If curricular changes are anticipated, then the issue of testing transfers arises, and has a very different impact. Currently, transfers take the MPT only if they are going to enroll in a mathematics course at Western. (Although transfers who have taken calculus are not tested.) Depending on how QR is eventually woven into Western's academic fabric, this practice might have to change. If, for instance, courses with a strong QR component are developed and/or recognized, or if certain courses get infused with a stronger QR component, and these become required for graduation, transfers not anticipating taking a mathematics course at Western might be required to take the QRT for the same reason they take the MPT: to find out if they are academically prepared for certain courses. If, however, QR is concurrently woven into the curricular fabric in the community colleges, then assessing a transfer's QR competence might be addressed any number of ways, with the QRT being only the most obvious. There could also be transcript reviews; articulation agreements could be reviewed and revised, etc.

[top]

WHAT DO WE CURRENTLY KNOW ABOUT THE QUANTITATIVE REASONING SKILLS OF WWU STUDENTS?

Currently, there is no quantitative reasoning general education requirement (GER) at WWU. Students are offered a variety of methods to satisfy their GER requirement in mathematics, and the Math Placement Test (MPT) is administered to most students to assist in placing them in appropriate courses, but there is no general quantitative reasoning test or course available. Thus, we have no data on the QR skills of our students.

[top]

MEETING THE CHALLENGE: SOME MODELS FOR ASSESSING QUANTITATIVE REASONING SKILLS

As has been mentioned, some kind of QR education has been considered as an alternative GUR to the current math requirement. In October, 2000, a group of faculty from the six public baccalaureate institutions met at a colloquy in Leavenworth, Washington, to discuss how to define and assess quantitative reasoning in higher education. Based on those discussions, but subjectively reinterpreted by the authors of this paper, three models for the assessment of QR skills will be presented below. In the first, there is a rubric, but no QR pre-test; in the second, a QR pre-test is the model's centerpiece; and in the third, aspects of the first two are combined into a hybrid.

MODEL 1: This first model is based on a rubric that examines the QR content that already exists in the curriculum of major disciplines and/or departments. Each discipline and/or department would need to be actively involved in developing rubrics that worked within their area. As an example, faculty in the social sciences who were attending the colloquy described seven statistics-based research components that could be considered fairly generic in their area, and that would need to be included in any QR rubric applied to the social sciences. (It was also noted that not all social science courses contain these research components; nonetheless, the guiding principle of this model is that disciplines and/or departments are the most appropriate base for QR assessment.) This model does not use a pre-test of QR abilities or skills. It rather takes an inventory of QR skills and/or abilities that already exist within the curriculum, makes a vigorous public acknowledgement of the existence and importance of QR, and encourages the emphasis of QR in higher education. It may or may not stipulate that a student must take one of the courses identified as having a strong QR component in order to satisfy graduation requirements. The major pitfall of this model is it may not meet the legislative mandate for QR assessment.

MODEL 2: This model borrows heavily from the Wellesley model described earlier in this paper. The heart of this model is a pre-test of QR skills, administered to all in-coming students, either as an addition or an alternative to the MPT (math placement test). Students unable to meet the minimum standards of QR competency, as based on the pre-test, would enroll in a basic QR course. Those meeting the standards would be required to enroll in an "overlay" course in which QR concepts would be identified and evaluated in courses in the major disciplines. Students required to take the basic QR course would also be required to enroll in an "overlay" course as a stipulation to graduation. As in MODEL 1, a discipline and/or department QR rubric would be vital in identifying appropriate "overlay" courses for students to take. When talking about course logistics and the best use of resources, the idea of using already existing courses as part of a QR requirement would be far superior to creating a slew of new ones.

MODEL 3: This model would not require a pre-test in QR, but would require all students to take a basic QR course in addition, or as a supplement to the University's current mathematics stipulations. This model may or may not include the additional requirement of a QR "overlay" course, although a discipline and/or department rubric would still be a sensible, probably necessary plan of action, if for no other reason than to create a broader sense of QR's current existence within the curriculum, as well as to respond to legislative concerns.

[top]

NEXT STEPS

If any of the above models, or parts of them, were to be adopted into Western's higher education goals, how will we know if those goals have been reached? Regardless of the domain - QR, writing, ITL, critical thinking - this problem is central to all student learning outcomes assessment. Are exit tests the solution? This strategy has its appeals - straight-forwardness, the promise of quantifiable results, a clean finality - but is also fraught with problems. For example, who would pay for the costs? Do we pass them along to students? What about the logistics of having 2500 people a year sit down for such testing? Do we impact course/class work by setting aside a week of testing days? What about the messy work of assessing writing? No one yet has come up with a way of feeding a computer a sample of student writing and have it spit out a meaningful rating. And what, pray tell, happens if students don't meet the standards? Do we force them to repeat courses? Do we hold them back? Yet of all the legitimate concerns over exit testing, what educators fear most is that exit tests lead inexorably to instructors teaching to the test rather than their hearts, thereby stifling academic creativity, flexibility, and innovation.

Certainly a more practical student learning outcomes assessment tool are satisfaction surveys. Our current ones could be modified to include questions about QR, Information Literacy, etc. Yet such surveys never seem quite enough. They're a "soft" measure, subjective and not as representative of our student population as most researchers would like them to be. While the surveys themselves could be more finely honed, and return rates could be increased, satisfaction surveys would only ever be one part of a fuller student learning outcomes assessment effort.

Probably QR assessment will need to tie in to the current student learning outcomes assessment technique: allowing a well-designed, thoughtfully considered curriculum to take care of the end results by itself. Yet this approach, too - as steeped in tradition and as well-intended as it may be - often feels "soft", especially by today's higher academic standards, driven by the need for objective quantifiable data.

Something relatively new that might help this student learning outcomes conundrum are the performance standard rubrics that have been developed, or that are being developed for the domains established by the legislature. (To date, rubrics have been developed for writing and critical thinking, while rubrics for information technology literacy and quantitative reasoning are in process.) What's useful about these rubrics is how they delineate the expected performance abilities of students at various levels along their academic careers. That these rubrics have been so diligently thought out and produced is a very good thing - some might even argue that it's a process long overdue. Content standards have been debated and produced seemingly since the inception of education; they are, at least currently, the bases for most curricular decisions. On the other hand, clearly articulated performance standards have been either sorely lacking or all but invisible to those outside academia.

So having the rubrics is good, but what is going to happen to them once they are honed and ready for use? Will they be used as part of an exit test? Oops. There's that again. But if not an exit test, what about the idea of applying the standards of each domain's rubric to the curriculum? What if from day-one to day-last of a student's academic career each course contained some of, maybe even sometimes all of the domain skills, with those skills then assessed according to the standards proposed by the rubrics?

Indeed, by way of promoting a point of discussion, we are going to suggest in this Dialogue the idea of student learning outcomes assessment serving as a matrix - and here we are defining matrix as that which gives form to a thing. We are thinking specifically of calling it an institutional matrix.

How an institutional matrix might work is this: a) here are some domains of knowledge - writing, QR, ITL, critical thinking - that we want to send our students into the world with as an educated citizenry; and b) here, as laid out under each domain, are the rubrics establishing the performance standards for those domains of knowledge. With the domains and rubrics in place, c) each instructor incorporates into his or her courses as many of these domains as is appropriate and reasonable; and d) utilizes the rubrics (also as appropriate and reasonable) to assess student learning outcomes.

It might be helpful to think of this institutional matrix as an overlay to most of the existing pedagogical techniques already practiced and courses already taught. It's worth is as a curricular clarifier, an intensifier, a focuser, the tool with which murky areas can be made clear again, not just for those of us in academia, but for those outside it - legislatures, taxpayers, the parents of our students. For those folks especially clarity is of utmost importance, yet also what many of them feel academia lacks.

Our current system of courses, grades, capstone experiences, et al., is not necessarily bad or out-dated, but it is sorely lacking an overlay of contemporary clarity. It may be that in the course of applying such an institutional matrix there will be some fundamental rethinking of our goals as higher educators; maybe there will need to be some discussion about the balance between life-long learning skills versus content knowledge in the various departments and majors, but such discussions - and maybe even the reevaluations that might come from them - will not hurt us, but rather might renew the energies of all those involved in higher education.

[top]

SUMMARY

QR and math are inextricably linked, but while math is primarily a discipline, QR is primarily a skill - one with practical applications. Moreover, while mathematics is often exclusive, frequently with a language of its own, QR is inclusive, its language plain and everyday. Few people are trained to work with complex mathematical concepts, but most can understand mathematics well enough to develop informed opinions about quantitative concepts.

The expectation that college graduates will demonstrate competence in quantitative reasoning is broadly supported on both the state and national levels; however, the assessment of QR skills in undergraduates is complicated. For one thing, QR crosses domains. For another, important assessment questions have yet to be addressed: what QR skills should students enter with? what should their QR skills look like when they graduate?

We have suggested three different assessment models that might achieve the goal of determining whether our graduates meet acceptable standards of QR competence. Moreover, we have suggested the idea of an institutional matrix focused in the general education program that would support, sustain, and encourage the development of student learning outcomes from each of the four state initiatives - quantitative reasoning, information technology literacy, writing, and critical thinking - which form the core of a new and exciting assessment arena for higher education in our state. Such a broad scale approach to student learning outcomes has enormous potential to impact how faculty think about student learning in all courses they teach.

[top]

* * *

NOTES:
1. Answers: 1. 1/4 lb/per party-goer; 2. Value now is $18,000; 3. More than doubled, almost tripled.
2. Simpson, C. (November, 1999) Quantitative Reasoning (QR) Progress Report (page 2). Office of Institutional Research and Resource Planning. Western Washington University.
3. QR for College Graduates: A complement to the Standards and the MAA's Subcommittee on Quantitative Literacy Requirements (Committee on the Undergraduate Program in Mathematics).
4. Guidelines for the Quantitative Reasoning Overlay Requirement, page 4.
5. To see more examples of QR questions and/or tests, see contacts at the bottom of page 6 of this report.

[Top]

published by
Office of Institutional Assessment and Testing
Dr. Joseph E. Trimble, Director; Gary R. McKinney, General Editor
technical assistance by Center for Instructional Innovation
Dr. Kris Bulcroft, Director; Web Design by Karen Casto

For copies of Dialogue, OIAT technical reports, Focus Research Summaries, or InfoFacts, please contact Gary McKinney, Western Washington University, MS: 9010, Bellingham, WA 98225. Telephone: (360) 650-3409. FAX: (360) 650-6893. E-mail: mailto://garyr@cc.wwu.edu. TTY: (800) 833-6388. Join in discussions of Dialogue issues on the web at: http://www.ac.wwu.edu/~dialogue.

Dialogue Home | Institutional Assessment Home |
Center for Instructional Innovation Home | Western Home