Congruent self- and supervisor assessment

Recently, I had a long office hour with three or four students to discuss my written responses to their graded term papers. After the office hour my colleague Katharina, whom I share the office with, said that she had noticed a pattern in how students react to these responses: No matter how critical I am of their work and no matter the grade the students receive, they almost always look up after reading the response and say “I totally understand your criticism, you are describing exactly what I knew I had difficulties with while writing.”

This expression of congruity between the students’ own assessment of their work and my professional response is indeed a reaction I get often. Students recognize themselves in my response, they feel seen not only in the successful parts of their work but also in their struggles with the form of academic writing.

This is important because when students experience their supervisor’s critique as constructive and as engaging with issues they themselves have already identified, it lays the groundwork for a successful revision and improvement of their work.

There is a method to congruity

This congruity is not a coincidence, it is an effect of the form of response I choose and the mindset (Haltung) from which the form originates.

I have written about my revision-oriented supervision and about the way I give appreciative responses to student writing before; in short: I write up a response for every student paper in which I first paraphrase the paper’s argument, then appreciate the especially successful aspects, then give an expert critique of higher-order concerns, to finish with suggestions for an (optional) revision.

Katharina’s observation confirmed to me the importance of the first two steps in my response process: The paraphrase (introduced by the phrase “[Name], in your paper you [do this and that] …”) and the appreciation (introduced by the phrase “What I like most about your paper is …”), which precede the expert critique that we usually think of when we talk about supervisor feedback.

Paraphrase and appreciation

I often find it difficult to provide a descriptive non-evaluative paraphrase of a paper, because what is on my mind immediately after reading a paper are either its most successful aspects or its gravest problems. But forcing myself to disregard those for a moment enables me to look at the paper on its own terms. While grading is never objective, I can still put myself in a non-evaluative reader stance that acknowledges what the paper’s project is, how its line of argument proceeds, what it actually does.

When I write this descriptive paraphrase first and only then proceed to appreciative feedback on where the project succeeds in fulfilling its own aspiration, I create a mental framework for myself in which I don’t respond to the paper in terms of its deficiencies (deviations from my professional academic criteria of excellence), but in terms of the frame of reference of the writer themselves.

‘Issues’ vs. ‘deficiencies’

This in turn enables me to see where the most important issues are with the paper, which is not the same as its deficiencies:

Deficiencies are isolated failures to conform to very specific technical or rhetorical conventions of writing. Pointing out deficiencies never makes writers feel seen in what they tried to accomplish, but it makes them see themselves as deficient, which is not constructive.

Issues are complex clusters of mismatches between intent and execution, they are problems on the level of the design and the content of the paper’s project. These issues are not often easy to identify or to express, but having acknowledged and appreciated the paper on its own terms makes it so much easier to identify them.

This is, first, because the descriptive paraphrase takes seriously what the paper’s stated intention is and how it goes about fulfilling it: in my experience, many of the initially identified deficiencies immediately become lower-order concerns or even completely irrelevant if put into this perspective. ¹

Second, expressing appreciation of successful aspects of the paper, i.e. where and how the stated intention is indeed matched by the execution, serves to create contrastive priorities so that it becomes clearer how the paper’s problems relate to the paper’s merits (and I say ‘relate to’ rather than ‘weigh against’ because I think as a scholar, not as a grader).

Critiquing a scholarly project

From this point onward, after having understood how the paper is supposed to work and where it does work well, it is easier to phrase a proper professional critique to express how exactly the paper does not work in one or two important ways.

My critiques often focus on the following aspects:

• on argumentative logic (where conclusions don’t match the evidence),

• on conceptual or epistemological confusion (where terminology or entire theoretical approaches are used inappropriately),

• on analytical rigour (where the scholarly toolbox was used inefficiently),

• on interpretive depth (where the textual evidence was used inefficiently),

• or on the use of scholarly literature (where the academic conversation was not joined in effective or appropriate ways).

These aspects are obviously complex, and it often takes me a bit of concentration to be clear in my mind why I think something is an issue and then to express it in language that effectively communicates this to my student – writing responses takes effort. ²

But the effort pays off when my responses actually address the paper as a scholarly project and are not just lists of the students’ successes and failures in an exam. This is why students feel congruity between their own assessment of their work and mine:

I don’t use their work as a foil to practice my grading criteria on, but I acknowledge their work in its own right. This makes it easy for students to feel taken seriously, to accept even fundamental criticisms openly, and to relate themselves to their own work not just as examinees but as writers.

Example: response and revision

This is an example for a response to an ambitious student project which had several interrelated issues that ultimately led to a fail grade for the draft. But the revision for the paper was extremely successful, and I wrote another shorter response for the reworked version, which is appended below. (I edited both texts for brevity.)

The student also expressed congruity between my assessment and her self-perception and was able to rework her paper extremely well. There are more examples for responses in my article on appreciative responses to student writing.

Response to graduate module paper draft, “Queer in Popular Media: The Representation of Queer in One Piece: How progressive is the anime adaptation?” (25 pages)

[Name], in your paper you argue for an understanding of the anime series One Piece as ‘progressive’ on the grounds of its inclusion of queer characters and its subversion of stereotypical representation. You begin by introducing terminology of gender theory and the cinematographical lens of mise-en-scene, the latter of which you then use as the structural model to analyse the appearance and performance of several prominent characters from the series. You proceed to look at One Piece in the context of scholarly discussions of queer representation in popular media and in the context of the fandom-internal reception of the show as queer. You eventually present an overview of the positive and the problematic aspects of the show, while concluding that the show is ultimately progressive.

What I like about your paper is the ambitious scope of the project and the political drive behind it: To look at a range of different characters and the fan discussion to evaluate an entire anime series in terms of its representation of queerness is quite a challenge. I also like (in principle; see below for the caveat) the detail in which you describe the characters in order to collect as many features of their queerness as you can.

However, I see a number of grave problems with your paper, on the theoretical level (gender and queerness), the methodological level (mise-en-scene), and the argumentative level (evaluating the series’ ‘progressiveness’). Let me tackle them bit by bit:

Theory. The theoretical approach you did choose doesn’t work for two reasons: First, your presentation of de Lauretis’ notion of gender is a vague summary of some of her important points, which in the second half (page 7) consists almost entirely of direct quotes that remain unexplained and unexplored. I get the impression that you didn’t understand de Lauretis and just dropped some quotes in your text. This impression is supported by your reference of de Lauretis on page 25 (“gender does not exist”), which is such a misrepresentation that it is ‘not even wrong’. Second, even if you had understood de Lauretis, I don’t think that a discussion of gender as socially constructed and culturally produced is useful for your argument. This notion of gender is so fundamental to all queer theory that it is hardly worth mentioning.

What is useful for your argument, though, is queer theory proper, which you don’t introduce at all. In David Halperin’s words, queer is “a positionality vis-à-vis the normal” and the normal in your context would be hegemonic binary notions of gender and heteronormative notions of sexuality. In your analysis you address exactly this ‘positionality’ of the characters, but since you don’t have any real theoretical framework, all the details you collect to describe the characters are ‘hit and miss’ – sometimes really interesting but not theoretically embedded, and very often simply irrelevant to your project.

This lack in clarity and precision with regard to your approach also generates problems on the level of expression and thought, for example you use the terms ‘identity’ and ‘natural’ in a way that does not reflect the complex deconstruction of both concepts in queer theory (e.g. page 16 or page 24). […] These are details, but not ‘just details’: They signal to me as your reader that you have only a vague understanding of the theoretical underpinnings of the most important term of your paper: ‘queer’.

Methodology. The cinematographical concept of mise-en-scene also, in my opinion, does not work for your project. The schematic analysis of ‘costume’, ‘colors’, and ‘action and performance’ almost forces you to collect a lot of details that seem to have very little to do with what you ultimately want to argue, namely that the representation of gender and sexuality of the show really is queer on a deeper level than just on the stereotypical surface level. But what you describe mainly is exactly this surface level (that’s what mise-en-scene is, after all: what you see). Moreover, this surface level representation is extremely stereotypical, which you have to concede again and again in your chapters 4 and 5. I see your argument becoming weaker and weaker toward the end, and that is a direct consequence of your choice of method.

Argumentation. Your argument is that the series is ‘progressive’ in its representation of queerness. You never quite explain what ‘progressiveness’ means, I think you use it pretty much synonymously to ‘queer’ – but then again you don’t explain ‘queerness’ either. So I feel your argument is on shaky ground from the beginning, because you don’t really make clear what the controversy is about in the first place: what is the problem, what is the criticism, why wouldn’t a show about many queer characters be queer? This whole problematic becomes a bit clearer only late in your paper at the beginning of chapter 4, where you cite snippets from Peele’s text on queer popular culture and where you paraphrase critics of One Piece. Peele’s criteria for ‘good’ queer representation make sense to me and they serve to contextualize your argument: Queer representation, in his understanding, is successful when, beneath the stereotypical surface, queer characters matter within the fictional universe. Thus, queer representation which depicts queer characters as superficially stereotypical is ‘not really queer’ or even hurtful to the queer community – something that the show has been criticized for. Your main point to refute the criticism seems to be that “Bon Curry and Ivankov are essential for the progress of the story” (page 24), which is something I can now understand. But: You never really address this importance for the story in your analysis (on the contrary: you mostly show me how superficially stereotypical the characters are), which is why your statement is an unsupported claim – as such, it doesn’t strengthen your argument.

I think all these problems can be solved, but it will require an extensive rewrite of the paper. Think in terms of the problem that you are addressing: what are the oppositional sides and why does it matter in the first place? Then pick your side and choose your theory and method so that you can make the best argument for this side. Take good care to represent the other side as fairly and adequately as possible, and refute their arguments one by one.

There are other problems with your text but in comparison to the large-scale issues above, they are secondary:

a) Your way of quoting is often un-scholarly: you use drop quotes without elaboration, you misrepresent quoted text [where you do elaborate on it] (like de Lauretis, see above), and you unduely eliminate the context and the specificity of quoted text (especially for Peele, who makes specific points about the series Friends – to use his words, you need to make clear how you think they can be generalized).

b) [Selection of scholarly sources].

c) [Language issues].

Assessing your paper draft was really difficult. If this had been a final version, I would have failed it […]: beyond the cumulative problems outlined above, the deciding factor was the misrepresentation of de Lauretis’ scholarship. […] There are many ways to approach a revision of your paper now, which we can discuss in the consultation.

Thanks for your hard work!

[Signature, date]

Response to the revision (under a new and more specific title), “Queer in Popular Media: The Representation of Homosexuality in One Piece” (25 pages)

[Name], I’ve read your revision and I think it’s very successful […] – I’m impressed by the way you seem to have synthesized the critique and used it to build up your rewrite! What works really well now is how you explain the initial problematic of representing queer characters, the possible pitfalls and scholarly positions of argument and how you then take up your own position and build your own argument around it.

Your theoretical introduction is focussed and very much connected to your later interpretation and conclusion. The additional layer that you have generated by introducing the Japanese discourse on homosexuality and contrasting it to Western views and stereotypes not only makes your argument so much more interesting and complex, it also fits well into the general outlook of queer theory: it questions normative points of view and exposes that without taking into account a specific cultural context, even a ‘standard’ queer theoretical analysis will have blindspots.

In the final chapters 4 and 5 you phrase your argument strongly and confidently in favour of the series, while also acknowledging its representational shortcomings – these parts are very professional in rhetorical style!

The weakest part, for me, is your chapter 3 – it is far too detailed and contains too much summary. While I like how you use specific scenes to characterize [Bon Curry] and Ivankov and how you generate a sense of their narrative arcs, it’s just too much; this part could have profited from being cut down to maybe half its length. Maybe it would have been easier to decide what is relevant information and what is not, if you had interwoven the analysis with the interpretation in chapter 4. As is, chapter 3 reads like a long list of plot points, with a descriptive rhetoric in short sentences which are often not really analytical (in the sense that I often don’t understand their purpose for the argument).

[…]

There is still a noticeable number of language mistakes in your text. But your citations are now well-embedded and you treat your sources much better than in the draft paper, providing context for their analyses and making clear why they are important for your paper.

I will grade this final version 1,3 [which corresponds more or less to an A–] – the problems in editing annoy me as a reader, but ultimately they don’t weigh very heavily against a strong and critical argumentative approach that does full justice to the complexity of a queer theoretical analysis of popular culture.

Thanks for your hard work!

[Signature, date]