In this episode, listen to Drs. Kate Owens from the Math department, Ashley Pagnotta from Physics, and Jenn Wilhelm from Psychology discuss their alternative grading strategies. In this part, Ashley and Kate give insight on their use of competency-based/standards-based grading. In Part II, Jenn discusses her use of "ungrading."
Resources mentioned:
In this episode, listen to Drs. Kate Owens from the Math department, Ashley Pagnotta from Physics, and Jenn Wilhelm from Psychology discuss their alternative grading strategies. In this part, Ashley and Kate give insight on their use of competency-based/standards-based grading. In Part II, Jenn discusses her use of "ungrading."
Resources mentioned:
(narrator) Hi, and welcome to Impactful Teaching at the College of Charleston. A teaching and learning podcast, where we discover innovative strategies and practices to engage learners, celebrate the successes of others making an impact on today's students academic achievements and inspire each other to learn and grow in our own teaching practices.
(Mendi - Host) Hello and welcome to episode five. I'm your host, Mendi Benigni, an instructional technologists at the College of Charleston. And I'm here with three amazing professors at COFC to discuss alternative grading strategies. Before we get into it, I'd like to give each of you a second to introduce yourselves. Why don't we start with Kate.
(Dr. Owens) Hi, my name is Kate Owens. I'm the Associate Chair of the math department at COFC. And I've been teaching here since 2011, so 11 years now.
(Dr. Wilhelm) Hi, I'm Jenn Wilhelm. I am an Associate Professor and Director of experiential learning for the Department of Psychology. I also have faculty affiliations with the programs of neuroscience, women and gender studies and medical humanities. And I came in 2012 to the College of Charleston.
(Dr. Pagnotta) Hi, my name is Ashley Pagnotta. I am apparently the only person in this group who is just an assistant professor and doesn't have any other roles in my department or affiliations with other departments, but I'm an Assistant Professor in the Department of Physics and Astronomy. I started here in 2017, so this is my sixth year at the college.
(Mendi) I want to thank you guys for being with us today. I'd like to begin talking with Kate and Ashley who are using competency-based grading. So Kate, can you tell us a little bit about what that is? What does competency-based grading mean to you?
(Dr. Owens) Yeah. I think there are two terms that are used for this style of grading. One of them is competency-based grading, one of them is standards-based grading and they're used interchangeably. So I've been using standards-based grading in several of my math courses here at the College of Charleston for the last, gosh, six or seven years. So there are three key features about standards-based grading that sort of embody the system as a whole. The first one is that students are assessed based on how they perform against a clear list of learning targets, specific skills or problems, or objectives that they're supposed to be learning through the course. So instead of looking at how a student does on Quiz 7, maybe we're looking at how students demonstrate understanding of Newton's Laws of Motion. The second thing is that students are assessed based on their level of understanding instead of using a partial credit points-based, percentage-based system. So I'm looking to see if my students have shown a depth of understanding or if you're just starting out with understanding or don't have any understanding yet, as opposed to showing that a student got maybe eight points out of 11 points on a particular math problem. The third part of my standards-based system is that it's the eventual understanding that matters. So I allow my students to have multiple attempts. If they don't yet understand something at the start of a course, they're invited to retry that. Show me further understanding at some point later. So eventual understanding of the thing that I'm looking for and it's not just a one-and-done deal.
(Mendi) Why did you decide to change from your standard method of grading from the standard method of grading, I should say.
(Dr. Owens) Yeah, I realized at some point that there were students that were doing very well in my classes, were making Bs or B+, who had never fully solved any problem all the way correctly. And that just sort of shocked me and I had no idea how that was possible, but I realized that they were kinda like doing most of the things mostly okay, but they had never really done anything really well. So that was a really big problem. Secondly, I would have students who would know some of the course material extremely well and then not understand a full chapter, and they would still pass the class. And to me there's this huge gap in knowledge that they would need later on for other courses that they weren't getting from me in my course. So my, my thinking about this is that I really wanted my students' grades to reflect their demonstrated understanding of the course content. That's what I really wanted their grade to show. So I didn't want it to show anything about their performance or whether or not they're coming in late or if they needed extra help in office hours. Like I really just wanted their grade to reflect how much of the class they really understood. And the second thing was I wanted the grades to be motivational as opposed to just a brick wall in front of them. So with my standards-based system, if students show that they don't yet understand something that gives them the place where they can come back and retry that and work with me. And because I'm only interested in their eventual understanding of that, it's not a closed door in their face. So I can help those students raise their level of understanding for any of those gaps in their knowledge.
(Mendi) I like that because often we end up with summative assessments only and we give feedback, but then we don't give them an opportunity to use that feedback to improve or learn more. So I really like that.
(Dr. Owens) There's been some research that has shown that if you really want students to take their feedback that you've written, you shouldn't put a grade on that assignment at all. That basically as soon as you put a grade at the top of a paper, that the students just see that and they don't read your comments and they don't reflect on how I could do this better in the future or here's something that I missed, like they just see the number. That's the other problem with grades is that we think that we're giving students some kind of like insight into how they're doing. But I think the research is really showing that they just see the A, B, C and there's no further insight beyond that number. They're done. The grade is there and they're done.
(Mendi) Ashley, I see you nodding a lot.
(Dr. Pagnotta) I was evangelized about this by Kate. She was an evangelist to me about standards-based grading. And so, I started doing it actually right at the beginning of the pandemic. The first semester, I had to redo my physics 101, which is our intro non-calculus-based physics class. It's primarily taken by pre-health majors and I had to redo the class significantly anyways, because it was gonna be taught hybrid, where some of the students would be in the classroom, some of the students would be on Zoom simultaneously. And so, I figured I may as well go all in and redo the grading system as well. And I do a little bit of a modification of standards-based grading that is often referred to as standards-based testing. So, the vast majority of the points that students can earn for their final grades come from their performance, demonstrating proficiency on the standards. But I do also have a homework component that is a small percentage and I have a final project that they work on. Although that is not true standards-based grading, but there are some aspects to it that are. One of the things I did was read the book Grading for Equity, which I highly recommend. A lot of the things that Kate said and a lot of the things that the author of that book talks about really made a lot of sense to me, especially for physics, where it is the eventual demonstration of knowledge that matters. The students in my class are taking it because they need to know how to do all of these things for future classes or the MCAT, or whatever. And I realized that I don't care when they learn how to decompose vectors and use those to model 2D kinematics. I care that they learn it eventually by the end of the class. And so having that reassessment opportunity and having the standards grades, having their quizzes graded question by question based on whatever standard that question addresses really helps. I think both me and the student identify the areas that they are weak in and the areas that they need to rework some extra example problems, that kind of thing. I think it also really evens the playing field levels, levels the playing field because a lot of my students come in never having taken any physics before in their life. And then I have some students who come in who took two years of physics, including AP physics in high school. They come in with a really wide variety of backgrounds. There's a steep learning curve right at first. And I really like how this doesn't punish people who don't get it quite in the beginning because physics is hard and it's weird and different from anything they've ever taken before.
(Mendi) That is true, it’s a different way of thinking about stuff. It really is.
(Dr. Pagnotta) And so I really liked being able to say, “hey, I get that you've never seen vectors before in your life. And they're different and weird and confusing and you didn't get it the first time. But hey, maybe a week or two later, after you've worked with them some more, you understand them and can show me that you know how to use them now.” And great, that's awesome.
(narrator)
You're listening to Impactful Teaching. We’ll be back in just a few minutes to bring you the rest of today's episode. If you miss any of today's episode, you can hear it again on our website at tlt.cofc.edu. That’s tlt.cofc.edu.
(commercial) Attention faculty, are you looking for a free online conference with dozens of live sessions dedicated to best practices in teaching and learning? Do you want to discuss new pedagogical ideas and concerns with faculty from universities around the world? If so, then you'll want to attend the Seventh Annual Teaching Learning and Technology Conference, or TLTCon, May 16 and 17th, 2023, hosted by the College of Charleston’s Teaching and Learning Team. TLT Con 2023 will feature special keynote, Jay McTighe, best-known for co-authoring the backward design framework in the award-winning and bestselling Understanding by Design series. McTighe is a veteran educator, speaker, and accomplished author. Add TLTCon 2023, May 16th and 17th to your calendar today. You won't want to miss it!
(narrator) Once again, here's impactful teaching.
(Dr. Owens) There are two ways for me that standards-based grading changes my day to day interactions with students. So, one way is that I would have students in the past who would come to my office hours who maybe got a C- on a test, who really thought that they knew the material much better than they did. They'd asked me, “What can I do to do better?”, and I don't know. I mean, all the information I had about this student was they got to 72 and you might advise them would be get a better grade on the test. I don't, I don't have anything specific for you unless we go back to look at your tests to see what happened, what were the problems that you've missed? Now, because my grade book has the standards in it, I can look at that student and see, oh, you seem to be very strong with solving equations, but when it comes to graphing functions like that's your weakness and let's work on that together. Now, I've got specific action items that we can do that are related to the course content and not just broad study habits that would be applicable to every student. The other thing is that it gives me feedback on how I'm doing as the leader of this learning community that we have. Because now I can see that, oh wow, all of my students are really struggling with that idea. We need to all go back as a group and revisit that and have a conversation about what was missing that day so that I can kinda see where we are as a whole group and where we could improve.
(Mendi) Ashley.
(Dr. Pagnotta) Yeah, we just did that in my physics class on Friday and Monday because I had graded a quiz and there was one standard on drawing free body diagrams. And I don't know what happened that day, but they did not do well on those questions on the quiz. There were three questions on the quiz where they had to draw a free body diagram. And one student out of 45 drew all three correctly. But almost all of them had misconceptions creeping in on, on their free body diagrams. And so, I got those and I said, okay, we need to talk more about this because it's just going to come back in a physics class and it's similar for Kate in a math class, a lot of the stuff that you've learned comes back later. And if you don't get it the first time around, you're still going to be confused if you don't do some extra work on that. And so that was the thing where it wasn't just like a couple of students need to do some extra work on that. We all needed to do some extra work on that. So that's, that is also a really helpful thing, that it's not just feedback for them. It's also feedback for me.
(Mendi) How are you guys implementing this? Is it a test on OAKS where it's pulling from a pool of questions that they can take as many times as they want. Or what does this look like in your classes? And is it a lot more work for you guys?
(Dr. Owens) Okay. So those are two different questions Mendi and I'll say that having done this for a number of years, when I describe this to my faculty colleagues, they all say, wow, that sounds like a lot of work. So that's the top question. That's always the top question. I'm going to answer that one first. I think that overall, my students are solving many more problems over the course of this semester that they're giving to me to evaluate. On the other hand, because I'm not giving any partial credit, like I'm looking at either you showed me you know how to do this or you haven't yet showed me how to do this. I can grade those questions much more quickly.
(Mendi) So you get credit by going through the entire problem. Like you said earlier, they can go through part of it, but they never completed a whole one. So now they are completing an entire process.
(Dr. Owens) Yes, it is completely correct or it is not. Like those are the two bins and I have a medium bin for kinda “working on getting there”. So, what would take so long in my old school grading was that I'd have some lengthy word problem and it was worth 12 points and I'd find somebody's solution and I would have to debate, wait, is this worth nine out of 12 or ten out of 12? Five papers ago somebody made the same mistake. What did I give them? I wanna make sure I'm being fair to all students. And those kinds of questions just took up so much of my time. Now, I think I'm grading many more problems per student, but it takes me 2 seconds or 3 seconds and it's just a yes, no. I can do it much more quickly. So overall, the amount of time that I'm spending with a red pen in my hand is not significantly different than what it was before. As far as what this really looks like. I don't think that it looks any different from what I was doing before. So I'm giving in-class quizzes, I'm giving in-class exams and giving a final exam. All of those types of assignments are still happening. It's just what happens once the student hands it to me. So instead of assigning a number value to every problem than adding up those numbers and then averaging together quiz one and quiz eight to get a quiz average. And then waiting that against the final exam and doing all these calculations. I'm looking at each problem and seeing what was the idea behind this problem. And my OAKS grade book has a column for that specific idea. So perhaps it's graphing parabolas. I've got a column in OAKS about graphing parabolas. And anytime a student solves any question that involves graphing parabolas, I'll update that in OAKS to see if they've shown me they understand what they're doing or if they've not yet gotten that idea yet. The assignments themselves look the same. It's just sort of way that I'm using my Gradebook is quite different.
(Mendi) Ashley, how about you? Is it similar?
(Dr. Pagnotta) So yeah, this is similar for me because I'm still kind of experimenting and figuring out the best way to track things. It ends up actually being, what I call the bookkeeping, is, I'm not going to say a nightmare because it's not a nightmare, but it is a lot harder to keep track of individual question grades. Basically, I'm keeping track of question grades instead of quiz grades. The bookkeeping is a challenge for me just time-wise, but also for me at least, even though it is annoying, the bookkeeping is annoying. That's really a better way to describe it than a nightmare. It's annoying, but it's worth it to me for the all of the benefits.
(Dr. Wilhelm) So Ashley, do you mind if I jump in? This is Jenn. I tried specifications grading and standards-based grading several times in my classes and fully, completely failed at it. I loved it for all of the reasons that you all have described and read several books about it. After taking some time to really reflect on what went wrong in my classes, I think I used too many standards and too many specification, like too many standards. I think for each chapter I had something like 50 to 60 standards. We covered 15 chapters that semester. So there were hundreds of standards that I was keeping track of. How you were taught? If you're trying to talk to somebody about how to set this up in their classroom and how did you come up with the standards that you use? And how many standards about should we be looking at in making a good standard base class?
(Mendi) First of all, for the audience, I wish you could see Kate holding her head and shaking her head No, like, Oh my gosh. And Ashley's eyes about popped out of her head when a Jenn mentioned 50 standards in one chapter.
(Dr. Pagnotta) Actually, can I start Kate, because I'm not as good at this as you are. So I will start and then Kate will tell you the real way to do it. So, we cover about 13 or 14 chapters in this semester, and I have about two or three standards per Chapter. In total, I have this semester, I have 35 standards. The way that I got my standards that I started with is I was very lucky that there are some other folks who have used standards-based grading in not identical, but similar classes. And so, I was able to basically crib their standards. I'm going to turn it over to Kate because she does it the right way so she can tell you that.
(Dr. Owens) I'd like to just say that I think this conversation really illustrates what we want our students to be doing, right? We want our students to come into our classroom and try things out and then maybe not be that successful and then not like give up and walk away from the thing, but to retry it and just try and do a little bit better next time. So, I think that the style of grading is exactly what we are doing in our own courses, right? You try it, you modify it a little bit, Ashley is still working a lot. It just experimenting with different things like that is, that is what we all should be doing as humans, right? We're just trying to get a little bit better than last time. About the number of standards. Yes, I did scream a little bit. I was muted, but I screamed a little bit about Jenn’s number. Okay, so how many standards is the right number? I don't know exactly. I think that one way to start out is to find somebody else's syllabus who's teaching a similar course and then just borrow their standards, try them on, see if they work for you like Ashley did. Another thing that I've done is math textbooks have one section, and usually it's aligned with a one-hour lecture or one-hour class. And then just trying to look at that one section and think like, what is the idea here? Why is this a section on its own? What do I want the students to be able to do after this particular class? So, if you look at that about one section, maybe per day plus time spent for assessments or Q and A's or review days or whatever, you'd probably come up with around 30 or 35 as Ashley did. Just for the number of classes that we have in a semester. For me, I settled on less is more because I have a pretty high bar for what I think is a successful solution. I'm going with fewer things, but a higher bar for success. So right now in my linear algebra course, I imagine that we'll have 24 or 25 standards for the course of the semester. So it's definitely fewer than one per day. I've tried doing maybe one standard every couple of days or maybe two standards a week and that seems to be a good spot. They’re small enough that I can tell you how one's different from another one, but there's not like thousands of them, right? We don't want to be overly atomic and run into Jenn’s issue of like my students can't track for 100 things. So, I would say 15-25 is probably a sweet spot for my courses.
(Mendi) And so, then I would also ask what your all's opinion is on this. I think that most people would agree that what we teach and what they absolutely mission-critical have to know are often two different things. That doesn't mean we shouldn't teach it. And it doesn't mean we shouldn't expose them to it. But what do they have to come out of that class really knowing to be successful in the field or in future classes within that same topic. So maybe what would you think about even looking at those competencies as being more targeted in that area too.
(Dr. Owens) Yeah, I think that that's a good point, Mendi, one of the things that's helped me and trying to figure out what the skeleton of my class should look like is, I've re-thought about what the point of these courses are in terms of big questions, like what are the five or six big questions that we're going to look at over the course of the semester. And my standards are the answers to those questions. I fully realize that an irrational world my students are going to forget this day of class. They're not going to remember this technique maybe in a couple of years. But I do hope that they remember those big questions, if not the specific answers.
(Mendi) Those are great tips. So, what are your students think about this? How are they dealing with the new grading scheme? What do my students think about this style of grading?
(Dr. Owens) One of the things I do on our first day of class when we start talking about the style of grading and how it's going to impact their daily life. As I show them a picture, it's a graph of their happiness over time. At the start of the semester, they are very, very excited about this idea because they see that they're gonna get multiple tries to do well. And that failing one quiz doesn’t really impact their grade because they can improve later. Then, sometime in the middle of the semester, usually there's a big drop, usually right around now, it's October right now. And usually there's a huge slide and their happiness because this is the time when they realize I was very serious that their grade is going to correlate to their demonstrated understanding, and they're going to have to learn that hard thing that they've been avoiding learning. And that their grade really will show how much they've actually learned in the class. They can't get by on just partial credit. And this is the time where they've had a dawning realization and they are very unhappy that they can't just coast the rest of the semester. It's really going to come back to bite them. And I know this, and I tell them this, and I tell them, Please let me know when this happens to you because I would like to be there to support you and I know that this will happen and that's okay. We don't have to like it every day. Then by the end of the semester, usually they're very happy again because they see their growth over time. And that thing that they were really struggling with the whole semester, they finally figured out how to do it. And they have this sense of pride of like actually having done it as opposed to just sweep it under the rug.
(Mendi) Thank you. We're going to stop here for this episode due to time, but don't worry, there'll be a part two. In part two, we're going to hear from Dr. Jenn Wilhelm on her implementation of “ungrading”. We'll hear again from Ashley and Kate and their advice to faculty who may want to implement alternative grading, so be sure to tune in.
(narrator) Thank you for listening to impactful teaching brought to you by the teaching and learning team at the College of Charleston. Until next time.