Praxis: A Writing Center Journal • Vol. 15, No. 2 (2018)

PEER OBSERVATION, REFLECTION, AND EVALUATION PRACTICES IN THE WRITING CENTER: A GENRE PEDAGOGY APPROACH

Daniel Lawson
Central Michigan University
lawso3d@cmich.edu

Abstract
In this essay, the author shares the results of a local assessment conducted on his center’s peer observation, reflection, and evaluation practices for graduate assistants (GAs), focusing especially on the form used to facilitate these practices. The author interviewed the participants and analyzed completed “Writing Center Graduate Assistant Observation” forms. The interviews focused on three major areas: 1) what they perceived the purpose of the observation and reflection exercise to be, 2) how they felt and what they learned about observing their peers and being observed, and 3) how they felt the form affected the observation and reflection. In brief, the author argues that melding evaluation and consultant self-reflection is fraught because the rhetorical situation of each requires markedly different social action. Two critical lenses guide this examination: reflective practice and genre pedagogy. Ultimately, the author cautions those who use observation and reflection in their assessments to consider carefully the documents and genres surrounding those assessments because these genres may (intentionally or not) draw on antecedent genres that are inappropriate for the social action they intend to facilitate. Perhaps more troubling, some of these genres may implicitly draw on and/or perpetuate ideologies that are fundamentally at odds with reflective practice.

In a description of peer observation and evaluation practices at Fordham University’s writing center, Jane Van Slembrouk has “seen that genuinely productive assessment can occur between equals and that observing a peer is inevitably a reciprocal process, prompting meditation on one’s own values and practices” (Van Slembrouk). In fact, many college writing centers have also adopted this egalitarian approach to assessment and evaluation practices, embedding peer-observation practices both in tutor training courses and in continuing professional-development efforts. That is, tutors are encouraged to observe one another’s sessions, take notes, and share their observations with their peers and/or directors. In so doing, they are also potentially evaluating one another in the process, as well as helping the director triangulate a sense of what is happening in sessions. Interestingly, while there is a great deal of conceptual inquiry and training lore on the relationship between observation and reflection, there is surprisingly little empirical work on the topic. More, outside of Van Slembrouk’s valuable work, little has been done to directly examine the relationship between observation, reflection, and evaluation.

Compositionists have discussed the problematic nature of observation in the context of supervisory teacher-training classroom visits, and many of those issues apply to the practice of observation in the writing center. In particular, these scholars note the Foucauldian nature of these sorts of observation. That is to say, these observations can often merely serve to reify categorical difference—between supervisor and supervisee, teacher and student, teaching subject and learning object, etcetera—which makes, observation more a matter of disciplining subjects and enacting power than nurturing independent practitioners. Despite these problems, Denise Comer argues “that reflective, reciprocal supervisory class visits are a unique, powerful, and positive mechanism for fostering and generating pedagogic and programmatic growth” (519). To address the problematic nature of supervisory visits, she encourages practitioners to make those visits “reciprocal rather than unidirectional—that is, formative for both parties—and to try to make more visible the many ways that WPAs are learning through these visits” (526, emphasis hers). And in theory, peer observation and reflective practices in the writing center do just this.

In addition to being discussed and modeled in tutor-training texts like The Bedford Guide for Writing Tutors and The Longman Guide to Peer Tutoring, the practice of combining reflection and observation has been remarked on (and also productively problematized) in articles by authors such as Michael Mattison and R. Mark Hall. The basis for most peer-observation practices is reflection, and as Hall notes, “reflective writing has long been a cornerstone of writing center tutor education” (82). If, however, as Hall and Mattison each assert, that observation and reflection can often still play a role (intentionally or not) in fostering a form of panopticism wherein tutors subject themselves to regulatory scrutiny, how might actually imbricating reflection and observation in assessment practices further complicate these efforts? Can peer observation meet the needs of both reflective practice and evaluation? What role can peer observation play in facilitating reflective practice? To address these questions, I share the results of a local assessment I conducted on my center’s peer observation, reflection, and evaluation practices for graduate assistants (GAs). I especially focus on the form used to facilitate these practices.

As a relatively new writing center director a few years ago, I inherited a system of evaluation that incorporated observation and reflection. I wanted to see how the GAs felt about these practices and if the observation and evaluation components of the exercise fostered or hindered reflective practice—both in observing and learning from others and from being observed. In brief, I argue that melding evaluation and consultant self-reflection can be fraught beyond the reasons Comer shares in that the rhetorical situation of each requires (and prompts) markedly different social action. Two critical lenses guide my examination: reflective practice and genre pedagogy. Ultimately, I caution those who wish to take this approach to consider carefully the documents and genres surrounding it because these genres may (intentionally or not) draw on antecedent genres that are inappropriate for the social action they intend to facilitate. Perhaps more troubling, some of these genres may implicitly draw on and/or perpetuate ideologies that are fundamentally at odds with reflective practice.

I feel it necessary here to address distinctions between assessment—or, as Muriel Harris has characterized it, local research—and research in a more general sense (Harris). As Rebecca Day Babcock and Terese Thonus observe, both “should be based on empirical data [. . .] [and] involve inquiry” but whereas assessment implies judgment and may “seek immediate application to a local context,” research does not (4, emphasis theirs). Where the two endeavors converge, however, is when assessment projects extend “to a more global inquiry, inviting others to participate in a comprehensive research project across local sites” (Babcock and Thonus 5). I address this distinction here because this study was based in local conditions and the practice it examines was intended to assess those conditions. However, the questions I raise based on this study extend—and, I hope, problematize—considerations of a rather common writing center practice. In short, I share the example of this case study to begin theory-building and to help frame future empirical examinations of writing center practices that combine observation, reflection, and evaluation.

Reflection and Peer Observation

As my research questions regard reflective practice, I define it here by relying on Kathleen Blake Yancey’s formulation, which is specific to the writing center. Yancey in turn draws on Donald Schön, a philosopher and professor of urban planning, who developed the concept of reflective practice to explain how professionals improvise during practice but then improve upon that practice through reflection. Yancey thus borrows from Schön when she defines reflective practice as:

Recording practice,

Reviewing it,

Understanding it, and

Then learning from and applying it elsewhere. (“Seeing Practice” 190, formatting hers)

Yancey contrasts reflection-in-action with constructive reflection. As Yancey describes it, reflection-in-action occurs during the actual session and involves “implementing a plan based on [an] emerging hypothesis” (“Seeing Practice” 191). Whereas reflection-in-action takes place during the session, constructive reflection is “the process by which a single tutoring event and/or several tutoring events are reviewed and understood as a part of practice theorized” (“Seeing Practice” 191, emphasis hers). Put another way, constructive reflection occurs when a practitioner examines an instance of practice, draws generalizations about it, transfers that knowledge to other contexts, and once again reflects on its application in those contexts.

Yancey’s formulation of reflective practice is useful for those interested in the role that peer observation plays in fostering reflection for two reasons. First, Yancey’s definition of reflection indicates the points at which observation can intervene: in recording it, the observer offers a different perspective; in reviewing it, the observer can supplement observations and confirm or problematize the practitioner’s perceptions; in understanding it, the observer can triangulate the practitioner’s nascent theorizing with community norms and expectations; in learning and applying it, the observer can work with the practitioner to develop a plan for implementing the developing theory(ies). Conversely, the observer can draw on the session to reconsider their own practice. Second (and as a necessary consequence of the first point), it indicates that of the two conceptions of reflection Yancey offers, observation is most suitable for constructive reflection because it helps a practitioner make sense of a session post hoc. Accordingly, reflective observation should follow the parameters of constructive reflection: helping the practitioner theorize practice based on the events of the session. Thus, if an observation (and the discursive forms around it) meets these criteria and facilitates this sort of social action, it can be assumed that it has enabled reflective practice.

Genre Pedagogy

Implicit in the notion of a discursive form enabling social action is genre pedagogy—that is, the idea that genres facilitate social action and thus have to be taught as such. Consequently, in addition to reflective practice, genre pedagogy informs my analysis in this study. I draw here on definitions and approaches to genre from genre theorists such as Charles Bazerman, John Swales, and Carolyn Miller, who all emphasize the social function of genre. As Miller explains, “a rhetorically sound definition of genre must be centered not on the substance or form of the discourse but on the action it is used to accomplish” (151). Genres are purposeful and goal-directed and, as Bazerman observes, genres are shaped by context but also inflect contexts as well (316-317). Genre pedagogy itself is an approach to teaching writing that emphasizes the role genre plays in structuring discourse and facilitating social action. As composition and genre pedagogy scholar Amy Devitt explains,

With genres understood as actions rather than forms, and as rhetorically meaningful rather than just conventional, writing teachers can use genre-based pedagogies to do much more than teach students the conventions of a few genres. If genres are rhetorical actions, then genre pedagogies can help students to learn to act rhetorically; and if genres are based in situations, then genre pedagogies can use genres to help students perceive, understand, and even change situations. (“Genre Pedagogies” 146)

In addition to working with students on classroom and academic genres, writing center consultants also use and draw upon several genres for writing center work. If, as Hall asserts (and I quoted earlier), “reflective writing has long been a cornerstone of writing center tutor education,” and if, as I argue below, reflective writing and writing center sessions are genres that make rhetoric visible through typified rhetorical actions, then perhaps writing center practitioners need to be more cognizant of the genres operating around that work (82). Consequently, consultants should be trained according to best practices in genre pedagogy. This is particularly important because, as Devitt as demonstrated in her studies on genre, “writers use the genres they know when faced with a genre they do not know. These genres are not in fact transferable; they do not meet the needs of the situation fully” (“Transferability” 222). In particular, as I explain in the Discussions section, the genres surrounding the acts of observation, reflection, and evaluation can over-determine the social action by drawing on antecedent genres that aren’t appropriate for the situation. In particular, I found that our GA observation form directed the social action in unanticipated ways.

Context and Site Description

The case study itself took place in a writing center at a large Midwestern state university. The writing center at this university is large, spanning three sites on campus and maintaining a high-traffic asynchronous online program. Depending on the semester, the staff consists of approximately forty to fifty undergraduate and graduate hourly consultants and a cohort ranging from eight to ten graduate assistants (GAs). All members of the staff are required to take a writing center practicum that introduces them to writing center pedagogy and prepares them for observing, being observed, and reflective writing. As opposed to hourly consultants, GAs receive a fixed stipend and tuition waiver. In addition to facilitating writing center sessions, they are responsible for additional administrative and leadership duties. More relevantly, in the early part of the 2010s, the university’s GAs unionized. The new GA union contract mandated formal annual evaluations. Program directors and chairs had a fair amount of leeway in designing these evaluations. The writing center director at the time wanted the evaluation process to reflect the values and philosophy of the center. Consequently, she collaborated with the GAs to develop a formal observation form and process, which they conceived as less a way of summatively evaluating the consultants than as a way to provide opportunities for reflection and integration into the center’s community of practice. That said, at that time, both the director and the GAs wanted a numerical piece to the reflection as a way to track individual progress. What’s more, the director incorporated these observations and the associated form with the required annual evaluation.

The observation process involved several steps. In brief, near the end of each semester, the director made the peer observation form available to the writing center GAs (Appendix A). Every semester, each GA was required to have a session they conducted and that was observed by a fellow GA. GAs chose who observed them. The observer would watch the session and complete the form. The front of the two-page form consisted of fourteen rows divided into three columns. Each row was organized around of an observable cue or dimension of a writing center session such as “projected enthusiasm,” and “helped writer actively engage in the learning process.” The first column enumerated these cues and provided a “Yes/No” checklist of several parts under each. For example, under “learning process,” were checks for “asked relevant and probing questions,” “helped writer understand terminology,” and “helped writer understand fundamental principles, concepts, and theories.” For each dimension there was a box in the row for the observer to provide examples and notes, and in the far right (and final) column, there was a box for the GA conducting the session to self-rate their performance in each dimension (see Figure 1 in Appendix B).

After observing the session, the observer would complete the form, including a blank section on the back labeled “Observer: Overall Comments.” These comments could be descriptive or evaluative depending on the session and observer (see “Results”). The observer would give the form to the GA, who would then self-assess along each axis, rating themselves on each between 1 through 4 (4 being the best). The GA would then complete a blank section on the back labeled “Consultant Comments and Reflection,” responding to the session and to the observer’s overall comments. The GA would then give the form to the director, who would briefly comment on the session based on both the observer’s and the GA’s notes and then connect it to her comments about the GA’s performance throughout the semester. For example, the director might mention that “this session mirrors much of what I have seen in Deanna’s practice, particularly in her probing questions and excellent use of interpretive paraphrase,” before moving on to discuss the GA’s performance in other areas, both in sessions and in regards to other responsibilities.

GAs who facilitated online sessions could elect to have an “online” observation in place of a face-to-face session. In fact, most of the participants in this study—all but two—were observed online. These sessions are asynchronous and occur through comments offered by the GA in the MS Word documents the writers submit to the service. In these instances, the observer merely downloaded a copy of the document the GA worked with the comments visible. In short, the observer reviewed the MS Word comments and their relationship to the text, filled out the observation form as if observing the session, and accounted for the different delivery medium.

The Study

For the assessment of our observation and evaluation practices, I secured approval from our Institutional Review Board for Research Involving Human Subjects. I interviewed the participants via email and analyzed each of the completed “Writing Center Graduate Assistant Observation” forms. All of the GAs working in the spring semester of 2014 were invited to participate in the research. Seven participants accepted. In the interviews, I asked participants questions in three major areas: 1) what they perceived the purpose of the observation and reflection exercise to be, 2) how they felt and what they learned about observing their peers and being observed, and 3) how they felt the form affected the observation and reflection. I grouped the answers to each of these question clusters to examine the themes that emerged. I also examined the reflections produced in the observation forms to see if they mapped onto the rhetorical moves Yancey used to define reflection; in short, I performed a rhetorical analysis to map out the generic moves made in the responses, the genre being reflective writing. Below I summarize the results of the interviews by question cluster, and I summarize my analysis of the observation form. I have assigned the participants pseudonyms.

Results

What They Felt Was the Purpose of Observation
In the interviews, all but one of the GAs identified the purpose of the observations as reflective insofar as it enabled them to learn from watching other GAs and to receive feedback from their peers. Most emphasized the ability to improve with phrases such as “When we are observed, we become aware of our practices and the places we can change or improve,” and “It is a way to realize what we have been doing well and what we can change or improve to make our sessions more effective.” That said, they also acknowledged the evaluation function, as well. Most mentioned eliciting “feedback from the director” as one of the purposes, and others alluded to the administrative functions it served. For example, one remarked,

“So, I believed that observing writing center sessions can help both the observer and the observed grow professionally. In addition to this, it is my general assumption that supervisors need to know how their staff members are doing at work. Peer observation can be a democratic and participatory means of evaluating the service in the writing center.”

The GA who omitted any reference to reflection or professional growth said that she “saw it as a procedural element that simply has to be conducted every year to be placed in the GA files.” In short, though GAs identified the purpose as reflective, they were also aware of the evaluative function of the exercise. The participants made this clearer in their descriptions about their feelings about the process.

How GAs Felt about the Process and What They Learned
Because many of the GAs chose to be observed in asynchronous online sessions, the dynamic of the observation was somewhat different for them. Given the small sample size, it is impossible to attribute this difference to the medium, but the GAs reported differently: the two who had a face-to-face session observed by a peer reported feeling anxious or nervous about being observed whereas those who had asynchronous online sessions did not. However, the two GAs who were observed face-to-face reported that they did not feel the observation affected their session negatively, though each did feel “pressured” or “intimidated.” Of the two who were observed during face-to-face sessions. For example, one remarked,

“Honestly, being observed in any situation is a nerve-wracking process. As an anxious person in general, having someone oversee and check off every move I made and every word I said felt a little too evaluative at times. I don’t think it affected the session negatively, but it definitely added some unwanted pressure on my behalf.”

The other mentioned that being observed “made me feel a bit nervous. However, I reminded myself that I did not need to feel intimidated.”

The GAs who conducted online sessions did not report feeling anxious, though again, the medium perhaps played a role. For instance, one GA stated,

“I don’t remember feeling nervous or any specific emotion. Because I was observed online, the consultant was sitting at another computer away from me observing an online session I had already completed while I continued working on other responsibilities. I would have been distracted at the time.”

The other GAs who had their online sessions observed echoed this sentiment. For example, another GA reported that “Peer observation did not affect my performance because I was not actively observed while I completed an online submission” and still another mentioned that “I felt comfortable about being observed, as I had already received feedback from multiple people on my online sessions when I was being trained.” Interestingly, the medium potentially affected the observations in other ways; one GA reported that he was “able to use one of my onlines for the review. I was somewhat careful about the one I chose, as some of them are harder to work with than others.” In short, the medium seems to have alleviated potential anxiety about being observed for a few reasons in particular. First, because the online sessions are asynchronous, GAs chose which session would be “observed” after the fact. This, in turn, enabled those GAs to be tactical about the session they chose, privileging sessions that mapped more closely onto the criteria laid out in the form. Finally, receiving feedback from their peers on their online sessions was part of the training for gaining online approval, thus those GAs were quite accustomed to an “observer” in that milieu.

Regarding what they felt they learned in either observation or by being observed, the participants either focused on surface-level and discrete aspects of their performance or spoke in general terms about learning their value. For example, GAs remarked that,

“One thing I learned overall was not to use ‘we’ in my comments when I was really indicating that the student could take action in a specific way in his or her writing.”

“[S]he gave some excellent feedback on how to better phrase a few of my comments, as well as noting that I hadn’t really given much positive feedback in the online that she reviewed. This was something that was extremely helpful for me.”

“[T]he session observation did allow me to see how I conduct a session from an outsider’s perspective, noting details that I may have to work on, (like asking if the student had been there before or how his or her last session went).”

“I learned that my voice was too low for the observers to hear.”

“I learned what phrases I use often.”

Other GAs reported that what they learned was that they had been following the parameters of the observation form and thus felt reaffirmed in their practice:

“I was surprised at how the eleven categories in the form can cover almost everything we practice in our sessions, both face to face as well as online. It felt good to know that I covered almost all of them.”

“I learned that I had improved in my online interactions since the same consultant had reviewed me during my ENG 510 online assignment. In my online I had learned to appear friendly and compliment students during sessions. I also learned that I was following the same expectations I had for consultants who were training.”

In short, the participants reported that they found concrete instances of their practice they could improve upon, and they reaffirmed that their practice overall matched the expectations of the community as relayed via the observation form.

In terms of what they learned from observing others, three reported having learned something they felt to be useful, whether it was adjusting their volume, “dealing with needy students effectively,” or as one reported,

“I did, however, observe a session where I felt the consultant did a really effective job at helping the student with a paper: connecting examples to personal life situations, asking open-ended questions, and being genuinely friendly. The student even said ‘Wow, this was really helpful. I will definitely be coming back.’ I tried to model my sessions after this consultant after observing her.”

Perhaps more interesting than the three who remarked on specific things they learned from their observations was the fact that the other four did not articulate any specific practices they observed that they might emulate or avoid.

How They Felt the Observation Form Affected the Observation and Reflection
All but two of the GAs indicated that they felt the form was useful in that it gave them some indication of what to look for and privilege in their observations (incidentally, one of those GAs had a face-to-face session). However, as I will explain, all of them were ambivalent about the value of rating themselves in the numerical portion. All of them pointed to the constitutive function the form played in structuring their experience—that is, each indicated that the form played a significant role in how they perceived and responded to one another’s performance in the sessions. Even in answers to questions that were not specifically about the form, participants alluded to its constitutive function in their observations and reflection. For example, one GA went immediately to the form in responding to a question about what they felt was the purpose of the observation:

“The questions asked on the form are pretty extensive and specific: did the consultant listen, provide helpful feedback, and engage the writer in the writing process? All the components listed are important to the writing process, and as a consultant, if one of those things did not get covered in the session, it would be helpful to know to make a future session more productive. Also, the “consultant’s comments and reflections” section of the form allowed for personal consideration of the process and session, which leads me to believe that this is a highly self-reflective practice.”

Despite their experience and training with reflective journals, the GAs reported having little sense of how to conduct the observation beyond what was specified in the form. The GAs also responded that the form strongly influenced the sorts of feedback they offered their peers: “The form affected my observation by telling me what to look for in my fellow consultant’s session and, thus, guiding how that consultant would be observed and evaluated.” Most used the form as a checklist to make sure they had “covered everything,” although some had pointed out that the Form did not necessarily fit every situation in which some of the categories did not apply.

Each of the GAs found the numerical self-assessment portion to be either unhelpful and or limiting. They either reported not understanding the purpose of the scoring or stated that it felt evaluative or arbitrary. For example:

“I tend to be pretty self-critical in my own reflections and evaluations, so I may have rated myself lower than another person would have in the ‘Self-Assess Rating’ section. I’m not sure if the numeral rating system helped me in the process, though. It felt evaluative.”

“The form negatively impacted my observation because the work conducted at the writing center cannot be limited to numerical values. I felt pressured to give myself all ‘4’s’ because this document will be reviewed by my instructor.”

“The numerical piece did not help my understanding: I felt confident in giving myself high scores in each area, but I also felt this could be perceived as overconfidence or cockiness. While I understood the purpose of writing a reflection paragraph on what the other consultant noticed in my session, I didn't really understand the purpose of assigning myself a score in each area.”

In short, the form seemed to act as a facilitating agent—a rhetorical actant that played a role in structuring the discourse—in the observation in two ways. First, it seemed to dictate what GAs felt they were supposed to do in the session. Second, it inflected the observation by implying an evaluative function for the observer. Technically, each GA was only assessing themselves, and then only in the sense that it would enable them to reflect on that session as part of a larger practice theorized. All but one GA, however, indicated the evaluative “feel” of the numbers, referencing either the observer or the inevitable review by the Director.

Summary of Form Results
On the form itself, most of the writing done by the observers was to provide examples of the GA making the sorts of moves indicated on the checklist. In the “Observer: Overall Comments” section, observers offered feedback that tended to be brief, ranging from two to four sentences. Each began with general praise statements such as “Betty was very friendly and informative in her comments and met all of the criteria above” or “Sara offers clear, specific, constructive strategies for revision to writers.” The comments then invariably turned toward a specific area the GA did well. Only a few ventured suggestions, and of these, they tended toward what might be considered lower-order concerns in tutoring such as “The only suggestion I have is to use less ‘we’ to avoid implying that the student and GAs are writing the paper together” and “I would suggest the consultant speak louder and be more confidently.” In this regard, the observer comments often resembled the genre Summer Smith describes as the “teacher end comment”—that genre teachers draw on typically appearing at the end of a student’s paper to provide that student feedback. I will more fully examine the implications of this resemblance below.

These observer comments thus informed the “Consultant Comment and Reflection” section. These reflections were longer than the observer comments, ranging from four to seven sentences, and these sentences tended to be longer than those in the observer comments were. Though some patterns emerged among the responses here, they were less formulaic than the observer comments. Half began with an evaluation and affirmation of the observer comments (which in turn affirmed their own practice): “Sara astutely pointed out some areas for me which I hadn’t considered during the session […],” “Joanna’s comments were beneficial in that they acknowledge my effectiveness as an online consultant […]” “By reading Mary’s comments, I have been reassured that my online language to students has been appropriate and friendly.” The other half began with an evaluation of themselves; two spoke to their strength as GAs in general (and did not comment directly on the session in that first sentence) whereas one began with an evaluation of what she could have improved.

All of the reflections functioned as a response to the observer. Most contain an element of the defense genre in that they either speak exclusively to what they felt they did well in the session or in general, or they explained some of the context that negated some of the feedback from their peer observer. For example, “I didn’t ask about the previous visit, nor did I do an overview or explain reading strategies. I think this was/is because I have worked with this student over the course of the semester and knew what she understood about the session’s terminology.” Another responds to a peer observer after praising the observer’s feedback, saying, “Fortunately, because I have worked closely with Joanna in my position, I am already aware of these strategies I need to uphold in my online sessions […]” Still, each of these two GAs then noted how they would apply this feedback to future sessions, mapping generically to Yancey’s formulation of constructive reflection as “[u]nderstanding [practice], and [t]hen learning from and applying it elsewhere” (“Seeing Practice” 190).

Of the consultant reflections, two did not meet Yancey’s criteria for constructive reflection. Given the small sample size, this is troubling because facilitating reflective practice is precisely the point of that section of the form. One spoke only in positive terms about her practice in general and did not reference any practices specific to the session. She ended her reflection by saying that because of her observer’s comments and her experience she had “rated [her]self as very effective in all the applicable areas.” The other GA also failed to address the specific session, thus not addressing Yancey’s “review” and “understand practice” criteria. Moreover, this GA’s reflection contained similar moves to the previous in that it resembled a defense of overall practice and ability.

Discussion

Genre’s Constitutive Function

If the guiding question for the project was, “did peer observation facilitate meaningful evaluation and self-reflection?” the answer would seem to be “perhaps not as intended, and the observation form (as well as the implicit ideologies informing it) may have played an overly large part in that regard.” In short, the mingling of purposes in the form—to evaluate and to prompt reflection—seemed to muddy the potential for GAs to genuinely reflect on their practice. Overall, the GAs found the process instructive, but often found the form prohibitive, and the form clearly played an active role in structuring the sorts of discourse around reflection. Of the themes that emerged from the data, to me the most striking was the role that genre played in facilitating and directing GAs’ attempts to both assess and reflect on their practice.

In brief, several genres/social actions intersect in the act of these peer observations. First, the GA’s performance in the session itself is a secondary genre intended to facilitate an intervention in and possible revision of the writer’s work. In turn, it draws on several primary speech genres such as those described by Summer Smith in her work on the genre of teacher end comments. These speech genres include judging genres, reader-response genres, and coaching genres (252-261). Second, written constructive reflection is a genre meant to facilitate a self-assessment (as described by Smith and Yancey) and a possible reconsideration of practice (as described by Yancey and Schön). Third, the observation form is a written genre intended to mediate a discussion between GA and observer that in turn refines the reflection begun in the form. Fourth and finally, the director’s evaluation is meant to convey the director’s expectations for and assessment of the GA’s overall performance in the center. Of course, many of these genres blur into one another, such as the GA’s self-assessment and the director’s evaluation. In other words, these genres often resemble one another or hearken to purposes that confuse the rhetors operating within them.

As such, because each of these genres often themselves draw on and/or resemble other genres, they further serve to complicate the participants’ response(s) to their rhetorical situation(s). For instance, the observation form draws on several secondary written genres—the checklist and notes draw on and resemble both ethnographic field notes as well as criteria-based rubrics, and the “Observer Overall Comments” section clearly resembles the genre of the end comment. In this case, the GAs operated within genres they did not feel they knew. Given Devitt’s observation about genre repertoires and writers drawing on inappropriate genres in unfamiliar rhetorical situations, it seems that the GAs may have drawn on antecedent genres that resembled the form and the situation they found themselves in, namely that of the criterion-based rubrics and the teacher end comment.

Consequently, the observation comments written by the observers mostly resembled the very rigid form of the end comment that Smith describes and are thus susceptible to Smith’s critique of that genre: “The stability of the genre—the very feature that makes end comments recognizable and, perhaps easier to write—may also reduce the educational effectiveness of the comment” (266). Put another way, because the GAs drew on this genre, it over-determined the sort of discourse produced at the expense of enabling them to learn anything meaningful from the exercise. When the checklist format of the observation format precedes it, it implicitly conveys a judgmental function for the overall comments. Given that this judging function follows a format that looks very much like a criterion-based analytical rubric, those comments must address each of the features that appear in the checklist. Accordingly, this conveys a pedagogical approach to the session that implicitly argues that a session may be evaluated by its correspondence to its textual form—its generic/formal features—rather than by the contexts, shared perceptions, and settings that inform it. Further, because the reflection followed the evaluative-sounding observer comments and the rubric-like checklist, these often drew on what would appear to be a defense genre: a rationale for the decisions made in response to the observer and in anticipation of the director evaluating that session.

Genre, Reflection, and Observation

The observation checklist and “comments and reflection” portion of the form the GAs completed mapped generically onto the criteria Yancey enumerates for reflection—that is, GAs basically recorded, reviewed, articulated a nascent understanding, and ostensibly learned something from the process that they would apply to future sessions. However, although they had produced discourse in these reflections that conforms to the generic features of the reflective genre, these reflections did not necessarily foster the sorts of social action that reflective genres are intended to facilitate. In particular, the form did not seem to facilitate constructive reflection if that term is defined as understanding single sessions (or instances of practice) as part of a larger practice theorized. Nor did the form invite the sorts of critical rhetorical moves necessary for substantive revision of that practitioner’s guiding assumptions. In short, melding evaluation and reflective observation was fraught because they were at cross-purposes: to critically reassess the basis of one’s practice can be substantively at odds with justifying one’s competence to one’s supervisor or community of practice. The observation form and process invited observers to look at the consulting session as a genre in a prescriptive rather than descriptive sense, evaluating whether or not it met certain generic criteria. And though the form is ostensibly descriptive in some sense in that it asks if a particular generic parameter occurred in a session, it is very much prescriptive in its implicit genre pedagogy, listing features typical to the genre of peer consulting and prompting identification and classification according to those features. Moreover, by asking GAs to rate themselves according to these features, it reifies these features as essential elements of a session. In this regard, the form resembles less a means of promoting reflection and more a criteria-based analytical rubric.

Perhaps more troubling, in rating themselves according to the parameters, GAs subject themselves to a different form of panopticism then they might otherwise through observation. Although Mattison has described the panoptic dimensions of peer observation and reflection, the form itself invites a particular form of self-regulation that is at once confessional and normalizing. That is, it does not merely make the GAs the monitors of their own behavior: the numerical component also asks them to normalize themselves against an imagined numerical standard. What’s more, as expressed by the GAs, there is no knowledge on their parts regarding what actually constitutes a “normal” score. If they rate themselves too highly, they may be seen as overconfident or unwilling to reflect; too low and they are incriminating their performance.

Consequently, many of the GAs reported “playing it safe.” Rather than incriminating themselves or each other, they chose sessions that conformed to the dimensions on the form. They offered the token kinds of advice and reflection asked for in the generic conventions without actually engaging in a genuine reconsideration of their prevailing theories about practice. In fact, a few of them reported pursuing reflective practice outside the bounds of the peer-observation process, remarking that they had given each other feedback “off the record.” That is, they wanted to help each other navigate what they felt to be the community’s expectations, but they also wanted to make sure they avoided getting their peers “in trouble.” Still, most reported that they felt they learned something from the process, though it was perhaps more surface-level than the process was intended to mediate. In short, their ability to act as reflective practitioners outpaced their ability to navigate the evaluative constraints informing the peer observation form.

Conclusion: Adapting Genre Pedagogy to Reflection and Observation

Because of this study, I have changed several things about the peer observation and evaluation practices at our center, and I would encourage other directors to consider them when crafting their own. First, I now work with our GAs to approach writing center sessions—and observation reports—as genres. That is, I encourage them to look beyond typified rhetorical features of these genres to look more closely at the social action these genres engender. As Devitt explains, “Genre awareness pedagogy treats genres as meaningful social actions, with formal features as the visible traces of shared perceptions. Analyzing the contexts and features of a new genre provides an inroad to understanding all genres” (“Genre Pedagogies” 152). In other words, rather than prescribing the sorts of generic features that often appear in a given session, I encourage our GAs to observe and understand the contexts and perceptions that lead to those features; observers can watch a session, identify the contexts and situations, note patterns in the features, and analyze what those patterns indicate about the situations (Devitt, Reiff, & Bawarshi 93-94). Such an approach, as Devitt asserts, “teaches metacognitive reflection and explicitly discourages formulaic writing” (“Genre Pedagogies” 153). These are exactly the sorts of skills necessary for constructive reflection.

Second, I have revised the journal assignment in the practicum course to train consultants more explicitly for the kind of reflective writing that meets Yancey’s criteria for constructive reflection and for the form they complete as part of the process. Whereas it used to be a more general reflective journal, it is now a slightly more structured “Session Summary and Analysis Memo.” In it, each move is made explicit by subsections: “Record Practice,” “Review Practice,” and “Understand and Learn from Practice.” Each subsection defines the rhetorical task through questions that prompt the sorts of thinking necessary for constructive reflection. My hope is that in addition to helping the GAs become more focused reflective practitioners in general, it will provide them a frame and an antecedent genre to draw on when completing the revised observation form.

Third, I have also changed the forms that facilitate the observation process (see Figure 2 in Appendix B). Because of the heuristic nature of genre analysis, questions—rather than categories—now form the basis of our peer observation prompts. Like Van Slembrouk, I feel that questions will facilitate “more open-ended ‘observation’” than the checklist that had previously been used (Van Slembrouk). This revised form has been incorporated into the curriculum and structure of the required practicum to help prepare GAs and consultants alike to be observed (see Figure 2 in Appendix B).

Fourth, I—along with some of the GAs who help prepare other consultants for online sessions—have begun to use a coding system from another project for consultant training. This system asks consultants to identify the sorts of comments they offer in online sessions (by type, mode, and focus). We have begun to extend this coding to other endeavors in hopes that it will help them to regard their work more objectively and through different frames. This, in turn, might help consultants to better “review” practice, picking out patterns in the sorts of response. It may also have the benefit of enabling them to consider how the types, foci, and modes of response they use facilitate the session in different ways independently of reifying the features of the genre of peer consulting.

Fifth and finally, we now separate peer observation from direct evaluation practices. Although not in the purview of this study, the director’s evaluation was clearly the source of much of the anxiety the consultants have about the process. While evaluation is still mandated by the Graduate Student Union, we have tried to separate the two somewhat: observation is no longer the basis of the evaluation but it is required for the evaluation to take place. That is, although the GAs are still required to have a peer observe them and reflect on the session, that session is no longer what is evaluated. Rather, the director speaks to his experience seeing other sessions the GA has facilitated as well as that GA’s performance in other writing center tasks over the course of a semester. Still, the observation and reflective process play a role in evaluation in that they can provide the director a concrete example of situations the director has perceived, thus giving director and GA a shared ground for discussing the GA’s praxis. The director can also use the form to speak to the GA’s fluency in reflection, helping them to become more critical practitioners. To return to Comer’s discussion about the value of supervisory visits in the context of WPA work, I would make a parallel claim regarding writing center work:

“What I am suggesting is that supervisory class visits are a uniquely powerful mechanism for faculty and programmatic growth and that supervisors and faculty will be able to tap into the fuller potential of these visits if we reshape them as more broadly evaluative and formative, as reciprocal, explicit learning moments for faculty members, supervisors, and the program.” (Comer 533)

I would say that peer observation in the writing center—informed by genre pedagogy—does just this.

In sum, I agree with Van Slembrouk’s claim that observation can play an integral role in reflection and in evaluation. My hope is that these observations can be more than an opportunity for GAs to learn something by watching other GAs and by receiving feedback from their peers: that they also become more aware of the work that genre does in shaping their practice, in shaping the way they see others’ practice, and in shaping their reflections on that practice. I would encourage writing center directors to keep the findings of this study in mind when considering the incorporation of peer observation into evaluation practices, as well as to clearly define the parameters and expectations of each. Further, a more active genre-pedagogy approach can alert consultants to the rhetorical work not only of the genres they review with writers, but the genres that circulate around them in writing center practice itself.

Finally, I would advise other directors to consider which questions the practice of combining peer observation with reflection might answer and if those questions are appropriate for the sorts of evaluation or assessments they are conducting. I came to realize that far more than instructing me on the quality of the actual sessions, the peer-observation process instead alerted me to how my GAs were reflecting on those sessions and struggling to navigate a number of genres surrounding their work. Accordingly, other directors might use methods similar to what I have described here to examine how their practitioners articulate their growing knowledge of best practices and how they navigate the “meta-genres” around their work in order to foster a more critical level of reflection.

Works Cited

Babcock, Rebecca Day, and Terese Thonus. Researching the Writing Center: Towards an Evidence-Based Practice. Peter Lang, 2012.

Bazerman, Charles. Shaping Written Knowledge: The Genre and Activity of the Experimental Article in Science. University of Wisconsin Press, 1988.

Comer, Denise K. “Bending the Gaze: Transparency, Reciprocity, and Supervisory Classroom Visits.” Pedagogy, vol. 11, no. 3, 2011, pp. 517-537.

Devitt, Amy. “Transferability and Genres.” Locations of Composition. Eds. Christopher J. Keller and Christian R. Weisser. State University of New York Press, 2007. 215-27.

--- . “Genre Pedagogies.” A Guide to Composition Pedagogies. Ed. Gary Tate et al University Press, 2014. 146-162.

Devitt, Amy, Mary Jo Reiff, and Anis Bawarshi. Scenes of Writing: Strategies for Composing with Genres. Longman, 2004.

Gillespie, Paula, and Neal Lerner. The Longman Guide to Peer Tutoring. Longman, 2008.

Hall, R. Mark. “Theory In/To Practice: Using Dialogic Reflection to Develop a Writing Center Community of Practice.” Writing Center Journal, vol. 31, no. 1, 2011, pp. 82-105.

Harris, Muriel. “Diverse Research Methodologies at Work for Diverse Audiences: Shaping the Writing Center to the Institution.” The Writing Program Administrator as Researcher: Inquiry in Action and Reflection. Ed. Shirley K. Rose and Irwin Weiser. Heinemann, 1999. 1-17.

Mattison, Michael. “Someone to Watch over Me: Reflection and Authority in the Writing Center.” Writing Center Journal, vol. 27, no. 1, 2007, pp. 29-51.

Miller, Carolyn. “Genre as Social Action” Quarterly Journal of Speech vol. 70, 1984, pp. 151-167.

Ryan, Leigh, and Lisa Zimmerelli. The Bedford Guide for Writing Tutors. Bedford/St. Martins, 2016.

Smith, Jane Bowman, and Kathleen Blake Yancey, eds. Self-Assessment and Development in Writing: A Collaborative Inquiry. Hampton Press, 2000.

Smith, Summer. “The Genre of the End Comment: Conventions in Teacher Responses to Student Writing.” College Composition and Communication, vol. 48, no. 2, 1997, pp. 249-268.

Thonus, Terese. “What are the Differences? Tutor Interactions with First- and Second-Language Writers.” Journal of Second Language Writing, vol. 13, no. 3, 2004, pp. 227-242.

Van Slembrouk, Jane. “Watch and Learn: Peer Evaluation and Tutoring Pedagogy.” Praxis: A Writing Center Journal, vol. 8, no. 1, 2010.

Yancey, Kathleen. Reflection in the Writing Classroom. Utah State University Press, 1998.

---. “Seeing Practice Through Their Eyes: Reflection as Teacher.” Writing Center Research: Extending the Conversation. Eds. Paula Gillespie, Alice Gillam, Lady Falls Brown, and Byron Stay. Lawrence Erlbaum Associates, 2002. 189-201.