physics problem solving rubrics

• Twin Cities
• Other Locations
• College of Science and Engineering
• CSE Directory
• Give to CSE
• Student Dashboard

Problem-Solving Rubric

Rubric categories.

We are developing an easy-to-use problem-solving assessment rubric to assess students' problem-solving skills based on their written problem solutions. The rubric has been tested for validity, reliability, and utility and that work is described in the doctoral dissertation of Jennifer Docktor (click here) . The rubric has five categories:

Useful Description: representing information from the problem statement symbolically, visually, and/or in writing

Physics Approach: selecting relevant physics concepts and principles to apply to the problem

Specific Application of Physics: applying physics concepts and principles to the specific conditions in the problem

Mathematical Procedures: applying math rules and procedures in the context of physics

Logical Progression: the overall solution process is clear, focused toward a goal, and logically connected (consistent)

Latest version of the rubric

Rubric version 4.4 (9/16/2008) in Word or PDF

Rubric training

The rubric can be used either for research purposes or for general assessment purposes. For research purposes, a high inter-rater reliability is important. We have developed training materials to help novices learn to use the rubric for either purpose.

General purpose rubric training materials PDF

These materials help people learn how to apply the problem-solving rubric to score students' written problem solutions.

Research purpose rubric training materials PDF Sample students' solutions

This is a more rigorous training process to achieve a high inter-rater reliability. Some repetition of the training process may be necessary. Sample student solutions used in the training and instructions are included.

• Education Materials
• Graduate Program
• Photo Gallery

Related Sites

• per central
• curriculum instruction
• Educational Psychology
• University of Minnesota
• Physics Department Homepage
• Onestop@Umn

Page maintained by Evan Frodermann [[email protected]]

Twin Cities Campus:

• Maps & Directions
• Parking & Transportation
• Contact U of M
• Directories
• © 2012 Regents of the University of Minnesota. All rights reserved.
• The University of Minnesota is an equal opportunity educator and employer. Privacy
• Last modified on October 15, 2012

Physical Review Physics Education Research

• Collections
• Editorial Team
• Open Access

Assessing student written problem solutions: A problem-solving rubric with application to introductory physics

Jennifer l. docktor, jay dornfeld, evan frodermann, kenneth heller, leonardo hsu, koblar alan jackson, andrew mason, qing x. ryan, and jie yang, phys. rev. phys. educ. res. 12 , 010130 – published 11 may 2016.

• Citing Articles (55)
• INTRODUCTION
• ASSESSMENT INSTRUMENT DESIGN
• RUBRIC TESTING: VALIDITY AND RELIABILITY
• UTILITY: APPLICATIONS OF THE RUBRIC
• ACKNOWLEDGMENTS

Problem solving is a complex process valuable in everyday life and crucial for learning in the STEM fields. To support the development of problem-solving skills it is important for researchers and curriculum developers to have practical tools that can measure the difference between novice and expert problem-solving performance in authentic classroom work. It is also useful if such tools can be employed by instructors to guide their pedagogy. We describe the design, development, and testing of a simple rubric to assess written solutions to problems given in undergraduate introductory physics courses. In particular, we present evidence for the validity, reliability, and utility of the instrument. The rubric identifies five general problem-solving processes and defines the criteria to attain a score in each: organizing problem information into a Useful Description, selecting appropriate principles (Physics Approach), applying those principles to the specific conditions in the problem (Specific Application of Physics), using Mathematical Procedures appropriately, and displaying evidence of an organized reasoning pattern (Logical Progression).

• Received 26 May 2015

DOI: https://doi.org/10.1103/PhysRevPhysEducRes.12.010130

This article is available under the terms of the Creative Commons Attribution 3.0 License . Further distribution of this work must maintain attribution to the author(s) and the published article’s title, journal citation, and DOI.

Published by the American Physical Society

Physics Subject Headings (PhySH)

• Research Areas

Authors & Affiliations

• 1 School of Physics and Astronomy, University of Minnesota–Twin Cities, Minneapolis, Minnesota 55455, USA
• 2 Department of Physics, University of Wisconsin–La Crosse, La Crosse, Wisconsin 54601, USA
• 3 Robbinsdale Armstrong High School, Plymouth, Minnesota 55441, USA
• 4 Department of Postsecondary Teaching and Learning, University of Minnesota–Twin Cities, Minneapolis, Minnesota 55455, USA
• 5 Physics Department, Central Michigan University, Mount Pleasant, Michigan 48859, USA
• 6 Department of Physics and Astronomy, University of Central Arkansas, Conway, Arkansas 72035, USA
• * Corresponding author. [email protected]

Article Text

Vol. 12, Iss. 1 — January - June 2016

Authorization Required

Other options.

• Buy Article »
• Find an Institution with the Article »

Download & Share

Problem corresponding to student solutions in Figs.  2 and 3 . It can be solved in at least two different ways. One uses Newton’s second law to relate the force on a CO molecule to its acceleration, and then kinematics to relate this acceleration to the final speed. A second method uses conservation of energy to relate the work done on a CO molecule by the electric field to its kinetic energy (and thus its speed) when it exits the box. The correct answer is 1160     N / C .

Example of applying the MAPS rubric to a student solution.

Example of applying the MAPS rubric to a second student solution.

First (main) problem-solving task used in student interviews.

Scatter plot of total rubric score vs TA grade for all eight midterm problems ( N = 918 ). The correlation ( ρ ) between the two scores is 0.82 ( p < 0.0001 ). Points are shifted by a small random number so as not to mask clusters of scores.

Rubric scores from (a) 160 student solutions and (b) 159 solutions written by an instructor or taken from a textbook solution manual. This rubric was identical to the final rubric but used a maximum score of 4.

Graph of score agreement (weighted kappa) for two raters as a function of number of solutions scored for eight problems. The solutions were scored sequentially in time. The ninth data point is a rescoring of the first problem initially scored as the first data point.

Sign up to receive regular email alerts from Physical Review Physics Education Research

Reuse & Permissions

It is not necessary to obtain permission to reuse this article or its components as it is available under the terms of the Creative Commons Attribution 3.0 License . This license permits unrestricted use, distribution, and reproduction in any medium, provided attribution to the author(s) and the published article's title, journal citation, and DOI are maintained. Please note that some figures may have been included with permission from other third parties. It is your responsibility to obtain the proper permission from the rights holder directly for these figures.

• Forgot your username/password?
• Create an account

Article Lookup

Paste a citation or doi, enter a citation.

• Basics for GSIs
• Advancing Your Skills

Examples of Rubric Creation

Creating a rubric takes time and requires thought and experimentation. Here you can see the steps used to create two kinds of rubric: one for problems in a physics exam for a small, upper-division physics course, and another for an essay assignment in a large, lower-division sociology course.

Physics Problems

In STEM disciplines (science, technology, engineering, and mathematics), assignments tend to be analytical and problem-based. Holistic rubrics can be an efficient, consistent, and fair way to grade a problem set. An analytical rubric often gives a more clear picture of what a student should direct their future learning efforts on. Since holistic rubrics try to label overall understanding, they can lead to more regrade requests when compared to analytical rubric with more explicit criteria. When starting to grade a problem, it is important to think about the relevant conceptual ingredients in the solution. Then look at a sample of student work to get a feel for student mistakes. Decide what rubric you will use (e.g., holistic or analytic, and how many points). Apply the holistic rubric by marking comments and sorting the students’ assignments into stacks (e.g., five stacks if using a five-point scale). Finally, check the stacks for consistency and mark the scores. The following is a sample homework problem from a UC Berkeley Physics Department undergraduate course in mechanics.

Homework Problem

Learning objective.

Solve for position and speed along a projectile’s trajectory.

Desired Traits: Conceptual Elements Needed for the Solution

• Decompose motion into vertical and horizontal axes.
• Identify that the maximum height occurs when the vertical velocity is 0.
• Apply kinematics equation with g as the acceleration to solve for the time and height.
• Evaluate the numerical expression.

A note on analytic rubrics: If you decide you feel more comfortable grading with an analytic rubric, you can assign a point value to each concept. The drawback to this method is that it can sometimes unfairly penalize a student who has a good understanding of the problem but makes a lot of minor errors. Because the analytic method tends to have many more parts, the method can take quite a bit more time to apply. In the end, your analytic rubric should give results that agree with the common-sense assessment of how well the student understood the problem. This sense is well captured by the holistic method.

Holistic Rubric

A holistic rubric, closely based on a rubric by Bruce Birkett and Andrew Elby:

[a] This policy especially makes sense on exam problems, for which students are under time pressure and are more likely to make harmless algebraic mistakes. It would also be reasonable to have stricter standards for homework problems.

Analytic Rubric

The following is an analytic rubric that takes the desired traits of the solution and assigns point values to each of the components. Note that the relative point values should reflect the importance in the overall problem. For example, the steps of the problem solving should be worth more than the final numerical value of the solution. This rubric also provides clarity for where students are lacking in their current understanding of the problem.

Try to avoid penalizing multiple times for the same mistake by choosing your evaluation criteria to be related to distinct learning outcomes. In designing your rubric, you can decide how finely to evaluate each component. Having more possible point values on your rubric can give more detailed feedback on a student’s performance, though it typically takes more time for the grader to assess.

Of course, problems can, and often do, feature the use of multiple learning outcomes in tandem. When a mistake could be assigned to multiple criteria, it is advisable to check that the overall problem grade is reasonable with the student’s mastery of the problem. Not having to decide how particular mistakes should be deducted from the analytic rubric is one advantage of the holistic rubric. When designing problems, it can be very beneficial for students not to have problems with several subparts that rely on prior answers. These tend to disproportionately skew the grades of students who miss an ingredient early on. When possible, consider making independent problems for testing different learning outcomes.

Sociology Research Paper

An introductory-level, large-lecture course is a difficult setting for managing a student research assignment. With the assistance of an instructional support team that included a GSI teaching consultant and a UC Berkeley librarian [b] , sociology lecturer Mary Kelsey developed the following assignment:

This was a lengthy and complex assignment worth a substantial portion of the course grade. Since the class was very large, the instructor wanted to minimize the effort it would take her GSIs to grade the papers in a manner consistent with the assignment’s learning objectives. For these reasons Dr. Kelsey and the instructional team gave a lot of forethought to crafting a detailed grading rubric.

Desired Traits

• Use and interpretation of data
• Reflection on personal experiences
• Application of course readings and materials
• Organization, writing, and mechanics

For this assignment, the instructional team decided to grade each trait individually because there seemed to be too many independent variables to grade holistically. They could have used a five-point scale, a three-point scale, or a descriptive analytic scale. The choice depended on the complexity of the assignment and the kind of information they wanted to convey to students about their work.

Below are three of the analytic rubrics they considered for the Argument trait and a holistic rubric for all the traits together. Lastly you will find the entire analytic rubric, for all five desired traits, that was finally used for the assignment. Which would you choose, and why?

Five-Point Scale

Three-point scale, simplified three-point scale, numbers replaced with descriptive terms.

For some assignments, you may choose to use a holistic rubric, or one scale for the whole assignment. This type of rubric is particularly useful when the variables you want to assess just cannot be usefully separated. We chose not to use a holistic rubric for this assignment because we wanted to be able to grade each trait separately, but we’ve completed a holistic version here for comparative purposes.

Final Analytic Rubric

This is the rubric the instructor finally decided to use. It rates five major traits, each on a five-point scale. This allowed for fine but clear distinctions in evaluating the students’ final papers.

[b] These materials were developed during UC Berkeley’s 2005–2006 Mellon Library/Faculty Fellowship for Undergraduate Research program. M embers of the instructional team who worked with Lecturer Kelsey in developing the grading rubric included Susan H askell-Khan, a GSI Center teaching consultant and doctoral candidate in history, and Sarah McDaniel, a teaching librarian with the Doe/Moffitt Libraries.

University of Minnesota

Digital conservancy.

•   University Digital Conservancy Home
• University of Minnesota Twin Cities
• Dissertations and Theses
• Dissertations

View/ Download file

Persistent link to this item, appears in collections, description, suggested citation, udc services.

• About the UDC
• How to Deposit
• Policies and Terms of Use

Related Services

• University Archives
• U of M Web Archive
• UMedia Archive
• Copyright Services
• Digital Library Services
• News & Events
• Staff Directory
• Subject Librarians
• Vision, Mission, & Goals

• help_outline help

iRubric: Physics Problem Solving rubric

Main content

Links to this project.

Introduction

This instrument requires students to clearly describe in full sentences how they would solve four well-defined physics problems without going through the entire problem solving process, i.e., without explicitly solving the problem. The problem solving process of experts roughly consists of four steps: conceptual analysis, strategic analysis, quantitative analysis and meta-analysis (e.g., Selçuk & Çalýskan, 2008). This means that expert problem solvers tend to first analyse a problem qualitatively, based on the fundamental physics concepts and principles involved (e.g. conservation of energy, equilibrium of forces; Chi et al., 1981; Larkin et al., 1980). Then a strategy is selected which encompasses the concepts to solve the problem (conceptual knowledge), a justification for why these concepts can be applied (conditional knowledge), and a procedure how the concept is applied in the specific situation (procedural knowledge; Leonard et al., 1996). This instrument specifically focusses on students' conceptual and strategic analyses when engaging in physics problem solving.

General Description

At the beginning of the test, students are provided an introduction clarifying what is expected of them. This introduction includes a description of the instrument's structure, an exemplary item, and an exemplary answer to the item. Then, students receive the actual test consisting of four well-defined physics problems from basically two branches of classical physics (mechanics and electrostatics). These problems focus on (but are not completely limited to) physics concepts such as conservation of energy and momentum, gravitational and electrostatic forces, and harmonic oscillations.

Coding of Students' Open-Ended Answers

A theory-based coding rubric was used for scoring students’ responses for each problem. This rubric is based on the work of Docktor et al. (2016) who developed a similar coding rubric for students’ written solutions for physics problems. Our coding rubric comprises the categories concept , context , execution , and detail ; some of which can also be found under different names in the rubric by Docktor et al. (2016). Concept indicates whether students provide all relevant concepts necessary for solving the given problem. Maximum points were given if the response included all central concepts to the respective problem. Context informs whether important assumptions and conditions for applying the concepts were considered. Maximum points were given if important assumptions and boundary conditions were identified (e.g., neglecting friction, no rotation etc.). Execution provides information about the comprehensiveness of the response including the extent to which the reasoning is consistent. Maximum points were given if the sentences of the response were understandable and logically connected. Finally, detail reports to what extent students interspersed their responses with an elaboration of the concepts such as through mathematical formulas or qualitative relationships between quantities. Consequently, an identification of all relevant variables and their qualitative or quantitative relationship were necessary for maximum points. The four aforementioned categories were graded from zero to two points, while concept and context could also be graded negatively (one minus point) if certain aspects of these categories were definitely wrong (e.g., equating the gravitational force with the potential energy). On closer examination, these categories partially relate to the underlying strategy definition by Leonard et al. (1996): concept is trivial, context includes the justification for the concepts and execution is linked to the applied procedures. One can also notice that the categories are not independent of each other. As a consequence, we aggregated the points of all four categories over all four problems to an overall score measuring physics problem solving ability. Thus, a maximum of 32 points (i.e., eight points per problem) would indicate that a student has an extremely elaborate physics problem solving ability.

Sample Item and Coding Example

The first of the four items reads:

A very small mass slides along a track with a vertical loop (see Figure 1). The mass starts from a height above the highest point of the loop. Assume the motion to be frictionless.

Determine the minimum starting height above the lowest point of the loop necessary for the mass to run through the loop without falling down.

Describe clearly and in full sentences how you would solve this problem and what physics ideas you would use.

A solution for this mechanics problem given by a student who participated in our study reads:

At the uppermost point of the vertical loop, the radial force must compensate the weight of the mass so that the mass does not fall. Thus, the velocity can be determined as the square root of the product of gravitational acceleration and the radius of the loop. Then I use the law of conservation of energy, neglecting the friction of the mass. The initial potential energy must equal the sum of potential and kinetic energy at the highest point of the loop. Using this the required starting height can then be calculated.

Six out of eight possible points were given for this answer. A maximum of two points was given for the category concept since all central concepts (i.e., conservation of energy and the weight acting as centripetal force) were included. Only one of two possible points was given in the category context. Even though the student emphasises the assumption of non-existing friction when considering the conversation of (mechanical) energy, it is not reflected that the loop must be modelled as a circular path when relating the radial force with the radius of the loop. Even so, the student’s reasoning in this answer is comprehensible and consistent leading to a maximum of two points for the category execution. The answer contains some details, mostly given through a description of quantitative relations. However, the introduced mechanical energies could have been represented more precisely using formulas. Therefore one of two possible points was given in the category detail.

Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science , 5(2), 121–152. https://doi.org/10.1207/s15516709cog0502_2

Docktor, J. L., Dornfeld, J., Frodermann, E., Heller, K., Hsu, L., Jackson, K. A., Mason, A., Ryan, Q. X., & Yang, J. (2016). Assessing student written problem solutions: A problem-solving rubric with application to introductory physics. Physical Review Physics Education Research , 12(1). https://doi.org/10.1103/PhysRevPhysEducRes.12.010130

Larkin, J., McDermott, J., Simon, D. P., & Simon, H. A. (1980). Expert and novice performance in solving physics problems. Science , 208(4450), 1335–1342. https://doi.org/10.1126/science.208.4450.1335

Leonard, W. J., Dufresne, R. J., & Mestre, J. P. (1996). Using qualitative problem‐solving strategies to highlight the role of conceptual knowledge in solving problems. American Journal of Physics , 64(12), 1495–1503. https://doi.org/10.1119/1.18409

Selçuk, G. S., & Çalýskan, S. (2008). The effects of problem solving instruction on physics achievement, problem solving performance and strategy use. Latin-American Journal of Physics Education , 2(3), 151–166.

Add new wiki page

Wiki syntax help.

The wiki uses the Markdown syntax. For more information and examples, go to our Guides.

Page permissions have changed

Your browser should refresh shortly…

Renaming wiki...

Wiki page deleted

Press Confirm to return to the project wiki home page.

Connected to the collaborative wiki

This page is currently connected to the collaborative wiki. All edits made will be visible to contributors with write permission in real time. Changes will be stored but not published until you click the "Save" button.

Connecting to the collaborative wiki

This page is currently attempting to connect to the collaborative wiki. You may continue to make edits. Changes will not be saved until you press the "Save" button.

Collaborative wiki is unavailable

The collaborative wiki is currently unavailable. You may continue to make edits. Changes will not be saved until you press the "Save" button.

Browser unsupported

Your browser does not support collaborative editing. You may continue to make edits. Changes will not be saved until you press the "Save" button.

Start managing your projects on the OSF today.

Free and easy to use, the Open Science Framework supports the entire research lifecycle: planning, execution, reporting, archiving, and discovery.

Copyright © 2011-2024 Center for Open Science | Terms of Use | Privacy Policy | Status | API TOP Guidelines | Reproducibility Project: Psychology | Reproducibility Project: Cancer Biology