Test systems and learning

At the University of California Santa Cruz where I’m currently an exchange student I’ve encountered one aspect of the organization of university life that has been standing out to me as being remarkably far from my conception of what higher-level education entails: The way the grading and test system for some undergraduate classes is designed in an extremely rigid, inflexible and disciplinary fashion. Being confronted with this, I think it’s interesting and important to try to think about which attitudes towards learning are encouraged and which understandings of knowledge are performed in these systems. Which ways of being a student does the system enable and encourage me to pursue?

This is the way one upper division (which means the students have usually been to university/college for between two to four years) class was designed:

The final letter grade which is the one that will figure on the student’s transcript is generally translated from fairly meticulous 100-grade scale on which you during the course accumulated points. An array of different activities gave us these points. Each week we had 4-8 reading questions typically asking us to describe a certain concept from a text and compare it to something else. Each question had to be roughly a paragraph. All answers for these questions had to be uploaded on a webpage before a certain time in order to elicit points. It didn’t matter that much, we were told, whether the answers were actually good or correct as long as they were there and somehow tried to engage with the question. In the middle as well as by the end of the term we had two-pillar exams consisting of an assigned take home-essay and an in-class quiz. The midterm essay had two questions to choose from. An example of the form of one of them is:

“What is the relationship between concept 1 and concept 2? Discuss this through Author A, B and C (including citations), and give 2 examples each from Author D and E (with citations) to support your argument.”

The answer to this question had to be in three double-spaced pages. We were provided the rubric which the teaching assistant were to use to grade our papers, detailing how every inclusion of a requested article gave us a couple of points while a coherent argument gave a couple of extra points too. The final exam, in five pages, was a veritable puzzle and too complicated to explain here, but included three different quite broad ranging questions that all had to be answered and six lists of different texts from the course with us having to use one text from each list once, with all lists except one being tied to specific questions. The in-class quizzes had the form of a list of sentences with a blank spot in them in which we had to insert a concept from a list from the top of the page and write a small justification beneath it.

The design of a course can be thought of as an apparatus that works to create and/or sustain a certain conception of what knowledge and learning is. The particular class in which I experienced the test system described above was in addition also characterized by a fairly large reading amount, which at least for me (admittedly not a fast reader, but from talking to other students my impression was that many of those experienced the same) meant that it wasn’t possible to actually thoroughly read all pages of all the assigned texts, at least not without deprioritizing other classes simultaneously.

Here are some thoughts on what the design of the course in my experience did:

The actual practice of reading changed. The reading questions pointed out in advance what one should look for in the text, which meant that interpretation to some extend got uniformed across the class. Overall, Everyone reads after the same. My own experience was also that, due to the busyness I tended in a higher and higher degree to read with a purpose of getting a very superficial understanding of a given text in order to be able to answer the reading questions as fast as possible. This also strengthened a tendency to understand new texts through concepts and patterns that I already was acquainted with before starting to read the text (usually actually even before starting the course), thus reading for sameness rather than difference, the identic rather than the non-identic. I noticed that I started to disregard the types of analyses that were more ambiguous, complex or multifaceted, something I otherwise tend think of as important to pursue, in favor of the articles that to a higher degree allowed themselves to be reduced to what was entailed in an abstract or a conclusion.

The in-class multiple choice tests gives the impression that social scientific concepts are unambiguous and that they to a large extend can have a fixed meaning outside of a larger textual context. The fact that these quizzes count as an exam suggests that knowledge is primarily about memory, that being able to remember the (very simplified version of) meanings of concepts is an end in itself rather than doing something with the concepts. The reading questions and the prompts for the essays did in their formulation sometimes allow an approach that was a little bit more analytical explorative, but the extremely small amount of space and especially time for each of those meant that it in practice was very unlikely to get very far with them, and if you did end up spending too much time thinking about one of the questions and actually did an effort to answer it in an interesting way you’d have lost the time needed to answer the other ones and would end up getting lesser points in the end.

Thus in contrast to classes with exams based primarily on small research-like projects or more individually defined essays the test-design of this class had practically no mechanism to reward creativity or originality. This counters curiosity and creates a sense that more discoverous, groping reading/investigation is an ineffective way to spend ones time as a student. It also gives very little room to develop an individual flow of motivation and working style. I guess it that it as a contrast to neo-liberal demands of constantly being entrepreneurial selves could be argued to be somehow liberating to be allowed to feel less personally engaged. It can be stressful always to be creative and make decisions; this course had such a narrow scope for a creativity that it doesn’t matter anyway. One could be allowed to feel a bit of comfort in the oldschool experience of allowing oneself to be distant and despising of an inhumane and alienating system and a bit of joy in the hacking-like challenge of trying to formally meet the demands of the system with as little effort as possible. But then on the other hand it’d be hard to seriously argue that being able to accommodate so clearly defined goals could ever be a viable way of actually practicing social science. Thus the kind of skill and the kind of work the test-system encourages doesn’t point very far beyond being able to do well in this particular course. We are approaching something almost tautological: The primary goal of learning in the course becomes to be able to get a good grade in the course.

Flattr this!

3 thoughts on “Test systems and learning

  1. First off, I should admit that I’m the above-discussed professor, and that the class Bue is talking about in this post is my Biology of Everyday Life course (syllabus available upon request). It’s a midsized lecture course — about 70 students — and there’s one teaching assistant who does most of the week to week grading, but I help with the exams. Most of the class periods feature me lecturing for 90 minutes, with space for interjections and questions from students. It’s a class that I’ve taught three times, and each time it changes ever so slightly.

    It’s worth thinking about some of the assumptions that Bue is making in the initial post. Are creativity, flexibility and self-direction something that we should actually be encouraging a room of 70 students to pursue, or is that something better left for a smaller classroom? Should we promote interpretive flexibility, or ensure that students actually know what an author is trying to say? Is the purpose of a lecture-based class to enable students to do social science or just to give them a modicum of knowledge about a particular topic? Given Bue’s and my own description of the class, it’s probably pretty easy to infer my answers to these questions.

    In a smaller class it’s easier to go the other way — to encourage creativity and practice (and that’s how our department’s curriculum is structured, with upper division seminars that focus on individual research and writing projects). But when you have 70 students and one or two people in charge of grading all their assignments — with the burden of ensuring that there’s parity and transparency between how students are graded — it’s pretty impossible to appropriately factor in the weight of ‘creative’ work, which is entirely subjective.

    And, so, the assignments in BoED are based on two assumptions: for half of a student’s grade, it’s just a matter of being diligent (the reading questions that Bue describes, as well as showing up to class). The other half of a student’s grade is based on their actual knowledge of the texts & lectures (the midterms and finals that Bue discusses). Should there be interpretive flexibility in a term like ‘affect-moulding’ (Norbert Elias)? Or ‘biopolitics’ (Michel Foucault)? I would suggest that for undergraduate students the answer is a resounding ‘no.’ Once they have the basics down and are doing their own research, then ‘maybe.’ But first students need to understand basic concepts. You have to learn how to boil water before you can make pasta, after all.

    After more than a decade of teaching at the undergraduate level, I’ve become very aware that in large lecture courses the vast majority of students prefer pure anonymity — they want to sit back and listen to lecture, without any fear of being called on or having to demonstrate their knowledge of the texts. A handful of students each term decide to speak up and would actually like to be creative in their work. And, for them, I always offer another option — which Bue doesn’t discuss — in the form of a research paper on a topic related to class. Very few students complete these papers, which I’m come to believe is based on the needs I outline in the syllabus for them to produce content in a regular way (with parts of the paper due every other week). This leads me to believe that what most students want to do when they ask for ‘creative’ and self-directed outlets is just to not take tests and often turn in half-baked papers at the end of the term. And who gets anything out of that?

    1. First: Thanks a lot, Matthew, for taking your time to comment on my post.

      Regarding the option of doing the research paper in BoED, I guess it could be argued that in fairness towards the class I should have mentioned it. The reason I didn’t, apart from generally trying to make blogposts that are a little bit shorter and a little bit less tortuous, was that in my experience there weren’t very many students who in practice experienced it as an actual option, as it was still a demand if doing that that you had to answer the reading questions, which meant that making the research paper would obviously entail even more work on the top of an amount of reading assignments already appearing overwhelming.

      I can to some extend see the point about the administrative aspect of grading, that with, I imagine, limited resources in terms of available work-hours, a very explicitly defined system of grading is a way of efficiently doing the job while ensuring a basic fairness and avoiding the sense of arbitrariness that grading of more openly defined exams can sometimes, granted, bring about. Yet regardless of it being practically necessary or not, I still think it’s a really sad trade-off because it puts the system logic of grading way to much in the foreground way to much of the time, sometimes actually at the expense of genuine interest in the subject matter of the course, as I’m trying to demonstrate in the above. Grades are probably a necessary evil in education (though I’ve heard that UCSC actually used to do without them), but in my opinion it’s really unfortunate if the study culture gets to revolve around them too much. The more pervasive a grading-system is throughout a course the more that kind of culture is promoted.

      Some part of the discussion could also relate to a question about what the goal of an education in social science, in this case anthropology, is. I think it can be questioned how much it’s worth just to be able memorize some concepts – to a large extend abstracting them from their rootedness in more complex and situated analyses. What use is it to know a broad definition of biopolitics if you don’t have any training in using it in an argument? If you, after having graduated go out and work and socialize somewhere outside of the academy, nobody (or at least very few) really cares if you know about Foucault and Elias, and if you continue with an academic carrier, nobody cares if you’re not able to say anything new and fresh with or about them. The concepts of social theory doesn’t have a utility as facts in the way some knowledge within the sciences might (the contestability of those is another discussion), and thus another suggestion for what social science could/should be about is method in a broad sense, (which actually is a lot more in line with Foucault who, as far I remember, have actually dismissed his work being looked upon as ‘theory’): A way of reasoning, arguing, thinking, working, practicing.

      And even though the idea about division of labor between the classes seems reasonable, I’m inclined to think it’s a waste to have classes where you only have to know concepts without (or at least almost without) doing anything else with them, as I don’t think that promotes a very fruitful mode of learning.
      You’ll feel a lot more inclined to learn to get water boiling if it’s part of the process of making delicious pasta rather than just having hot water that you aren’t using for anything and that eventually will go cold again after the gas is turned off.

Leave a Reply