Praxis: A Writing Center Journal • Vol. 17, No. 2 (2020)
Grammarly vs. Face-to-face Tutoring at the Writing Center:ESL Student Writers’ Perceptions
Jing Zhang
Indiana University of Pennsylvania
xftx@iup.edu
Havva Zorluel Ozer
Indiana University of Pennsylvania
lvwx@iup.edu
Raneem Bayazeed
Indiana University of Pennsylvania
wrqw@iup.edu
Abstract
This study investigated how English as a Second Language (ESL) writers perceive their use of Grammarly, an online grammar checker, in relation to face-to-face tutoring at the writing center. Forty-three (N= 43) international ESL writers studying at universities in the United States participated in an anonymous online survey. Mixed methods were employed to examine participants’ perceptions of Grammarly and face-to-face tutoring at the writing center respectively as well as their perceptions of Grammarly in relation to face-to-face tutoring. Results rendered from descriptive analysis of the data revealed: 1) participants perceived both services with advantages and limitations; 2) participants used Grammarly more frequently than visiting the writing center, while they used face-to-face tutoring for a wider variety of purposes compared to Grammarly; 3) participants reported a both/and approach toward these two writing resources and used them to meet different needs in different contexts. Implications were offered for ESL writers, instructors, writing center tutors, and Grammarly program developers.
“Even though technology can be more available, humans provide a wider range of support and can alter ineffective approaches through training and self-improvement. Thus, “reaching” students (regardless of the reasons or their writing concerns) is still a human activity.” (98)
— Jenelle M. Dembsey
English as a Second Language (ESL) students who attend universities in the United States often need help with English writing, like their native English-speaking peers. Among the various resources and support that American universities offer to help ESL students manage college-level writing, writing centers cultivate ESL writers to become better writers through one-on-one tutoring services where they discuss their works with trained tutors for revision directions and readerly feedback (North). Besides visiting writing centers for help, ESL writers also use grammar checkers to receive feedback and suggestions on their writing, such as Grammarly. As a popular automated online grammar checker that offers writing support, Grammarly has been juxtaposed with the face-to-face (F2F) tutoring service at the writing center in a grammar-checker-versus-human-tutors discussion, as is reflected by the quote above. As a form of technology, how has Grammarly “reached” ESL student writers in relation to writing center tutors? How do ESL students navigate and utilize grammar checkers and tutors as writing support? This study aims to seek answers to these questions by investigating how ESL writers perceive and compare Grammarly and F2F tutoring at the writing center. We provide implications for ESL writers, writing center tutors, instructors, and Grammarly program developers with regards to supporting ESL writers effectively with grammar checkers and F2F tutoring services.
Online grammar checkers have been in use for a long time and attracted scholarly attention (Bigert et al.; Burston; Chen; O’Regan et al.; Potter and Fuller; Radi; Vernon). As one of the most popular online grammar checkers, Grammarly announces that “over 15 million people use Grammarly to improve their writing” (Grammarly). According to Jill Duffy, the success of Grammarly to have reached a large market results from its quality as “more than a grammar checker. It looks for repetitive words, jargon, homonyms, and hackneyed phrases, as well as words that nonnative speakers commonly misuse” (Duffy). Furthermore, Grammarly takes into consideration the genre of writing to make appropriate suggestions accordingly (Duffy): after one imports a piece of writing into Grammarly, Grammarly first prompts the writer to set goals, which informs Grammarly of the writing context and helps it to propose customized suggestions (see Figure 1 in Appendix A). In addition, Grammarly tracks users’ writing progress and provides users with weekly reports, informing them of issues to be resolved (TD Magazine).
Specifically, there are two versions of Grammarly: the free version and the premium version. In the free version, Grammarly checks writing for 150 types of grammar errors, while in the premium version, it checks writing for over 400 types of grammatical issues, makes suggestions for word choice and citation, and offers services for plagiarism detection (Nova). Figure 2 in Appendix A displays the interface of Grammarly.
Existing research on Grammarly has provided insights spanning across a variety of topics related to student writers’ use of the program. However, with such a popular and arguably powerful automated grammar checker, which is used prevalently by student writers, we are yet to gain more in-depth and nuanced understandings of this digital tool. We can achieve that particularly by juxtaposing it with writing resources such as F2F tutoring services. We examine how Grammarly can potentially change, influence, and shape students’ writing processes, their use of writing resources, and their reactions to feedback, especially in relation to the writing center, where feedback is often generated and delivered through F2F conversations. Particularly, we are interested in how the services provided by Grammarly are perceived by ESL writers in relation to the F2F tutorials offered by the writing center due to two reasons: first, as a group that has been historically associated with a strong need for surface-level writing support (Matsuda and Cox), ESL writers’ perception of Grammarly is worth studying. Second, because extant research predominantly examined Grammarly from the perspectives of researchers and teachers (Dembsey; Nova), empirical inquiry into students’ perspectives is conducive to a richer understanding of this tool. By adding a piece of ESL students’ perceptions of Grammarly to the “puzzle,” our study contributes takeaways for writing center tutors and instructors to better support their ESL student writers. As such, this study aims at answering the following research questions:
What are ESL writers’ perceptions of Grammarly?
What are ESL writers’ perceptions of F2F tutoring at the writing center?
How do ESL writers perceive Grammarly in relation to F2F tutoring at the writing center?
Literature Review
With Grammarly’s popularity among millions of users, a relatively sparse body of research has been devoted to the use and impact of this grammar checker. Research on Grammarly covers: user experiences of Grammarly (Cavaler and Dianati; Jayavalan and Razali; Karyuatry et al.; Nova; Qassemzadeh and Soleimani; Schraudner), review of Grammarly (Daniels and Lesli; Perelman), and comparison of Grammarly to the writing center (Dembsey). Extant literature has reflected mixed attitudes toward Grammarly, highlighting both its advantages and limitations.
User Experiences of Grammarly
One trajectory of research on Grammarly has focused on user experiences. Favorable user experiences with Grammarly were reported by Michelle Cavaleri and Saib Dianati, who studied the software’s perceived usefulness by eighteen college students (both ESL and English as a first language (L1) speaking students), through an online survey at two Australian colleges. Results revealed that the majority (94.4%) of the students found Grammarly to be easy to use, and the majority (83.3%) of students perceived Grammarly as useful, for reasons such as receiving detailed and helpful feedback, gaining a better understanding of grammatical rules, and gaining confidence in writing. Thus, Cavaleri and Dianati concluded that Grammarly can benefit both students and teachers in academic writing contexts.
In particular, Grammarly has been used in English as a Foreign Language (EFL) contexts as a tool to aid learners in their learning of English and English writing. Studies on the use of Grammarly in EFL contexts have focused on the advantages of this grammar checker. For example, by comparing the impact of feedback provided by Grammarly and teachers on Iranian EFL students’ learning of passive structures, Abolfazl Qassemzadeh and Hassan Soleimani reported that in terms of long-term memory, the feedback generated by Grammarly had a stronger impact than teachers’ feedback on the retaining of passive structures by EFL learners. Furthermore, other research zoomed in on the grammar elements in English writing by investigating how Grammarly helped EFL students reduce grammar errors and improve their writing quality. For instance, conducting research in the Malaysian and Indonesian contexts respectively, Kalpana Jayavalan and Abu Bakar Razali and Karyuatry et al. highlighted the effectiveness of Grammarly in helping EFL students identify and correct grammatical errors and thus improve their narrative/descriptive writing.
Additional research on the use of Grammarly in EFL contexts was conducted by Muhamad Nova, who reported both strengths and weaknesses of Grammarly through a narrative inquiry of three Indonesian graduate students. Results revealed that Grammarly had advantages such as offering quick and useful feedback, being highly accessible to students, and providing free services; meanwhile, weaknesses of Grammarly included providing inaccurate feedback and its inability to check the context and content of writing. Similarly, Michael Schraudner revealed students’ mixed attitudes toward Grammarly. By running 135 book summaries written by 17 Asia University Business Hospitality students in Grammarly, Schraudner indicated that Grammarly was able to identify sentence-level errors and create individually tailored reports for the entire class, saving time and effort for instructors; on the other hand, it sometimes flagged non-mistakes and did not have the function of peer comparison.
Based on the literature above, two things stand out: first, an important line of research focuses on ESL/EFL writers’ use and perceptions of Grammarly, especially those in EFL contexts; second, extant scholarship tends to investigate the effectiveness of Grammarly from a pedagogical standpoint without considering whether or not and how Grammarly should be used, especially ignoring students’ perspectives. Thus, our study aims to contribute to research on ESL writers’ use of Grammarly by focusing on international ESL students at universities in the United States, an English-speaking context, and by opening up our investigation to students’ self-reported use and experience of Grammarly broadly in academic settings.
Review of Grammarly: Among Other Grammar Checkers
Another trajectory of research has examined the efficacy of Grammarly through technology reviews situating Grammarly among other grammar checkers. For instance, Paul Daniels and Davey Leslie evaluated three online spelling and grammar tools, including Microsoft Word (MW), Grammarly, and Ginger, to determine to what extent these tools might help second language (L2) students with writing. The results showed that Grammarly was able to identify the missing spaces and spelling mistakes and provide several alternatives for misspelled words; however, when Grammarly identified fragments and offered advice on the verb form, it did not always provide suggested corrections and its explanations of error were complicated.
Similarly, with the question “Are grammar checkers reliable?”, Les Perelman submitted a number of essays to different online grammar checkers including WriteCheck, WhiteSmoke, MS Word, ETS’s e-Rater 2.0 in Criterion, Grammarly (free version), Ginger, Virtual Writing Tutor, and Language Tool. Based on the results revealing grammar checkers’ failure in catching errors, Perelman argued that computer grammar checkers were not reliable because depending on the algorithmic tagging of words, they provided misleading information and flagged certain correct constructions as errors. Like the other grammar checkers examined by Perelman, Grammarlywas perceived unreliable.
Grammarly vs. the Writing Center: Yet to Explore
With Grammarly and writing centers both serving ESL writers as available writing resources, Dembsey initiated a comparison between the services provided by Grammarly with those by the writing center. Specifically, Dembsey presented a thorough and rigorous analysis comparing the feedback provided by Grammarly on three freshmen’s essays with the feedback provided by ten asynchronous writing center consultants. Results demonstrated three patterns:
Grammarly provided twice more feedback than the online consultants due to repeated comments;
Despite the differences in amount, the writing center consultants covered more issues in their feedback than Grammarly. Feedback provided by Grammarly focused on the same issues regardless of specific writer concerns, while the feedback given by online writing center consultants differed across the essays;
With respect to accuracy, neither Grammarly nor writing center consultants provided 100% accurate feedback because both groups used incorrect terms and explanations in their comments.
In general, despite Grammarly’s availability, Dembsey held a more favorable attitude toward writing center consultants because of their advantages of being trainable, flexible, and interactive over the disadvantages of Grammarly of being rigid and driven by algorithms.
Meanwhile, although he did not specifically focus on the relationship between Grammarly and the writing center, Perelman called on researchers to investigate how feedback from online grammar checkers compares to the individualized approach of writing center consultants. This call was echoed by Dembsey, who suggested conducting further research on how students view Grammarly in relation to the writing center. To respond to Perelman’s and Dembsey’s call, to contribute knowledge of students’ perceptions of using Grammarly, and to juxtapose their perceptions of Grammarly with F2F tutoring at the writing center, our study aims to explore ESL writers’ experiences and perceptions of using Grammarly in relation to consulting tutors F2F at the writing center.
Furthermore, whereas current research has offered useful insights regarding the role of Grammarly in the field of EFL instruction and L2 writing, an important issue calls for scholarly attention and further research: studies such as Jayavalan and Razali and Karyuatry et al., both seem to equate the reduction of grammar errors with the improvement of writing. Does a smaller number of grammatical errors in writing necessarily mean a better learning outcome for L2 writers and thus the effectiveness of the tool/resource? With the important writing center philosophy of cultivating better writers rather than better writing (North), we are more interested in how ESL students perceive their writerly development with the use of Grammarly; thus, we intend to move beyond the discussion of the number of grammatical errors by conducting a mixed-method study to elicit both quantitative and qualitative responses from ESL Grammarly users.
Methods
To reveal ESL writers’ perceptions of Grammarly and the F2F tutoring service at the writing center and to examine how they compare their use of Grammarly in relation to F2F tutoring, a mixed-method research design was implemented to collect quantitative and qualitative data in this study.
Participants
Forty-three (N = 43) international ESL students attending universities in the United States participated in the current study. Because the survey did not mandate participants to answer all the questions, the total number of participants may not equal the number of respondents in the demographics section. Our analysis showed that most of the participants (N = 21) were Chinese native speakers. The remainder consisted of native speakers of Arabic (N = 10), Korean (N = 3), Bengalese (N= 1), Hindi (N=1), Thai (N=1), Ukranian (N = 1), and Turkish (N=1). Twenty-eight participants identified as female, and 15 participants identified as male. The participants’ ages ranged from 19 to 46. Fourteen participants reported that they were undergraduates, and 29 participants reported that they were graduate students, with 10 being masters students and 19 being doctoral students. The participants were enrolled in a wide range of majors including biology, computer science, accounting, management, finance, marketing, fashion, speech-language pathology, student affairs in higher education, educational psychology, curriculum and instructions, literature and criticism, and composition and applied linguistics.
Among the 43 participants, 34 reported that they were users of Grammarly. Of the 34 Grammarly users, 29 were using a free version, while five were using a premium version. Fifteen participants reported using Grammarly for one to 12 months, nine using it for two years, six using it for less than a month, three using it for more than two years, and one using it only for once. Of the 43 participants, 36 participants reported that they were writing center visitors. Among the 43 participants, 27 reported that they were using both Grammarly and the writing center services.
Instrument
After receiving IRB approval, we distributed an anonymous online survey (Qualtrics) through our social networks to international ESL students who were above the age of 18 and were studying at American universities. We used two social network platforms to reach out to our potential participants (WhatsApp and WeChat). We sent an invitation message and the link to our survey to our personal contacts on these two social network platforms, i.e., EFL students who were studying at a U.S. higher education institution. Meanwhile, with a snowball sampling method, we asked them to forward our invitation message and survey link to their personal contacts of other EFL students studying at U.S. universities; similarly, we shared our invitation message and survey link in group chats that included EFL students as group members, inviting them to take the survey and encouraging them to spread the message for us. The survey included four sections. First, participants who responded “yes” to “Have you ever used Grammarly?” could proceed to Section 1, which required participants to respond to nine statements on a 5-point Likert-scale on their perceptions about Grammarly; participants also responded to an open-ended question and a set of multiple choice questions about Grammarly. Following Section 1, participants were asked “Have you ever visited a writing center?” Those who responded “yes” could proceed to Section 2, which required participants to respond to nine statements on a 5-point Likert-scale and answer an open-ended question and a set of multiple-choice questions about using the F2F tutoring service at the writing center. Participants who reported having used both Grammarly and the writing center could proceed to Section 3, where participants were asked to compare their experiences of using Grammarly and F2F tutorials at the writing center through three 5-point Likert-scale statements and an open-ended question. The last section of the survey included demographics questions.
Analytical Method
Statistical and qualitative analyses were conducted to answer the research questions in this study. First, the Statistical Package for Social Sciences (SPSS) program was used for descriptive statistical analysis for the scale items in the survey. Descriptive data analyses were conducted to examine participants’ perceptions of Grammarly and F2F tutoring at the writing center respectively as well as their perceptions of Grammarly in relation to F2F tutoring. After descriptive statistical analysis were completed and finding that data were non-normally distributed, a Mann Whitney U test was conducted to examine the differences between participants’ perceived satisfaction, recommendation, and future use of Grammarly and those of F2F tutoring. Secondly, a one-pass, exploratory thematic analysis was conducted to analyze participants’ responses to the three open-ended questions regarding reasons why they liked and disliked Grammarly and F2F tutoring at the writing center and how they compared these two writing resources. First, after exporting survey results from Qualtrics, a Word document was generated to store all the participants’ responses to the three open-ended questions; the Word document was then imported to NVivo, a qualitative data analysis software, for thematic coding. The coder, i.e., one of the authors, read this qualitative data set carefully and iteratively, with the three open-ended questions in mind. During this process, the coder first conducted an exploratory thematic coding (Saldana) and assigned a list of codes and categories to capture the salient features of the responses. Then, a pattern coding (Saldana) was performed to generate themes that place the codes and categories in relation to one another and to describe the patterns in the data set. Before finalizing the themes, the data set and codes were reexamined again for any possible formulation of new themes. The codes, categories, and themes are presented in the following sections.
Descriptive Data: ESL Students’ Perceptions of Grammarly
Statistical Analysis: Perceptions of Grammarly
Table 1 in Appendix B presents the means, medians, and standard deviations of the rated items related to Grammarly. Table 1 illustrates that item 1 had the highest mean (M = 4.12), indicating that participants somewhat agreed that Grammarly is easy to use for ESL writers. Similarly, participants tended to somewhat agree that Grammarly makes helpful suggestions to improve ESL writers’ writing (M = 3.82) and that Grammarly gives detailed feedback to help ESL writers improve their writing skills (M = 3.74). Table 1 also shows that items 8 and 9 had the lowest means (M = 3.24), indicating that participants remained neutral with regards to Grammarlyhelping them to improve their writing skills and to get better grades on their assignments.
Thematic Analysis: Likes and Dislikes about Grammarly
To understand why participants liked and disliked Grammarly, their responses to the open-ended question (“Why do you like and dislike Grammarly?”) were coded through an exploratory thematic analysis and a summary of the findings are provided in Table 2 (Appendix B).
For participants’ reported reasons for liking Grammarly (31 references), the following reasons stood out: Grammarly offers useful feedback and explanations, and it is quick, convenient, accessible, and easy to use. On the other hand, participants articulated fewer reasons for disliking Grammarly. Among the reported concerns (11 references), participants pointed out that feedback provided by Grammarly was unhelpful, inaccurate, and decontextualized (7 references), that they had to pay to use the additional features of Grammarly Premium (3 references), and Grammarly focused on local issues only (1 reference), etc.
Descriptive Data: ESL Students’ Perceptions of F2F Tutoring at the Writing Center
Statistical Analysis: Perceptions of F2F Tutoring at the Writing Center
Table 3 in Appendix B demonstrates the means, medians, and standard deviations of the rated items related to participants’ perceptions of F2F tutoring at the writing center.
Table 3 shows that Item 1 had the highest mean (M = 4.03), indicating that participants somewhat agreed that tutors make helpful suggestions for improving ESL writers’ works. Table 3 also reveals that participants tended to somewhat agree that tutors provide good explanations about their suggestions (M = 3.92), and that participants trust tutors because their feedback is accurate (M = 3.81). Results reveal that Item 9 had the lowest mean (M = 3.58), indicating that participants were uncertain whether tutors help them understand grammar rules better.
Thematic Analysis: Likes and Dislikes about F2F Tutoring
To understand the reasons why participants liked and disliked F2F tutoring at the writing center, their responses to the open-ended question (“Why do you like and dislike face-to-face tutoring at the writing center?”) were coded through an exploratory thematic analysis.
Table 4 in Appendix B demonstrates participants’ reported reasons for liking and disliking F2F tutoring at the writing center. Among the 31 references, two reasons were the most salient:
F2F tutoring makes it easy to communicate, discuss, and ask questions about writing
Tutors provide helpful feedback and explanations.
Participants also reported benefiting from F2F tutoring because they better understood assignments and comments on their papers (2 references) and got inspirations (1 reference) by working with a tutor. Again, participants focused less on reasons why they disliked F2F tutoring at the writing center (13 references), with time concerns as a major reason such as the wait time and the limited opening hours of the writing center. Other reasons included the inconvenience to visit the writing center (3 references) and tutors’ lack of expertise (3 references).
Comparison: Grammarly vs. F2F Tutoring
A Big Picture: Frequencies and Purposes of Using Grammarly vs. F2F Tutoring
Twenty-seven participants reported using both Grammarly and F2F tutoring at the writing center. Table 5 displays the frequencies and percentages of these participants’ use of Grammarly and the writing center.
Table 5 (Appendix B) indicates that among the ESL writers who reported using both services, six participants used Grammarly on a daily basis, while no participants reported visiting the writing center daily. Table 5 reveals that more than half of the participants (55%) reported that they visited the writing center once a semester. Participants who used both services used Grammarly more frequently than visiting the writing center.
Table 6 (Appendix B) displays the purposes of using Grammarly and F2F tutoring at the writing center for the 27 participants who reported using both Grammarly and F2F tutoring at the writing center.
Table 6 reveals that participants’ purposes to use Grammarly included grammar (31%), spelling (27%), and punctuation (19%). Participants reported that they also used Grammarly for word choice (14%), style (8%), and other (to check plagiarism) (1%). It is seen in Table 6 that writing center visitors used F2F tutoring for grammar (15%) and word choice (14%). In addition, participants used F2F tutoring for a variety of purposes such as format (9%), punctuation, style, and organization (8%), spelling, transitions and flow, and developing ideas (7%), writing a thesis or main idea (6%), using examples and details, introductions and conclusions, and incorporating research (3%). Our data show that while ESL writers used both Grammarly and F2F tutoring mainly for grammar issues, they tended to use these services for different ranges of purposes.
Comparison: Statistical Analysis of Responses to Rating-Scale Questions
Table 7 in Appendix B presents participants’ perceptions of Grammarly in relation to F2F tutoring at the writing center.
Table 7 reveals that participants tended to agree that both Grammarly and F2F tutoring at the writing center were important to them (M = 3.88) and they would probably use Grammarly more often than working with a tutor face to face at the writing center (M = 3.85). Results indicate that participants were uncertain whether they use Grammarly instead of working with a tutor face to face at the writing center (M = 3.44).
Table 8 in Appendix B presents participants’ responses to “How satisfied are you with Grammarly/F2F tutoring at the writing center?”, “Would you recommend Grammarly/F2F tutoring at the writing center to your peers?”, “How likely are you to use Grammarly/F2F tutoring at the writing center in the future?” on a 5-point Likert-scale.
Table 8 reveals that participants tended to be somewhat satisfied with using Grammarly (M=3.74) and F2F tutoring at the writing center (M=3.94). Results indicate that participants would probably recommend using Grammarly (M=4.06) and visiting the writing center (M=4.47) to their peers. Participants reported that they were somewhat likely to use Grammarly (M=4.03) and visit the writing center (M=4.22) in the future. It was seen that the participants rated all the items slightly higher for F2F tutoring. To examine the significance of the differences between participants’ perceptions of Grammarly and F2F tutoring in terms of satisfaction, recommendation, and likeliness of future use, a Mann Whitney U test was conducted. Results reveal that there was no significant difference in satisfaction U=492.0, p=.123, recommendation U=493.5, p=.127, and likeliness of future use U=575.5, p=.642.
Comparison: Thematic Analyses of Responses to Open-ended Questions
When asked to compare their perceptions of Grammarly in relation to F2F tutoring at the writing center, four themes emerged from the collected responses (22 references):
Participants saw both Grammarly and F2F tutoring at the writing center with respective benefits and made choices depending on their specific writing needs and contexts (10 references), illustrated by one of the participants’ quotes:
Grammarly is convenient when you need instant responses and you can use it more portably. Working with a tutor F2F is the most helpful for me to revise my writing, …
Participants perceived both resources with limitations and hoped for improvement (4 references), as one of the responses went, “It (Grammarly) is mkre [more] convenient, though has many drawbacks. I wish I had tutor availability 24/7.”
Five participants reported favoring F2F tutoring at the writing center for its specific feedback and opportunities to discuss and communicate with tutors. Below is a compelling and profound quote from a participant, highlighting the advantage of F2F tutoring over Grammarly in that the former attends to issues beyond grammar and enables writers in discussions and negotiations:
Grammarly cannot replace working with tutors at the writing center. Grammarly can be used to check basic grammatical errors. However, writing is more than grammar. Writing involves so many issues that Grammarly cannot solve at all. Working with tutors is also about negotiation. During the tutoring sessions, students can ask questions and exchange ideas rather than take or not take the recommendations in Grammarly. (In Grammarly, there are two choices: take/deny).
Three participants reported favoring Grammarly, without giving specific reasons besides its convenience.
Therefore, results rendered from our data analysis revealed that participants perceived both services with advantages and limitations. In addition, they used Grammarly more frequently than visiting the writing center, while they used F2F tutoring for a wider variety of purposes compared to Grammarly. Furthermore, participants reported using these two writing resources to meet different needs in different contexts.
Discussion and Conclusion
The present study aimed to answer three research questions:
What are ESL writers’ perceptions of Grammarly?
What are ESL writers’ perceptions of F2F tutoring at the writing center?
How do ESL writers perceive Grammarly in relation to F2F tutoring at the writing center?
In this section, major findings are presented regarding ESL writers’ perceptions of Grammarly, F2F tutoring at the writing center, and their comparison of these two writing resources. In addition, the major findings are synthesized to make connections to the extant research.
RQ 1: Grammarly
In general, participants demonstrated a positive attitude toward their use of Grammarly. Participants did not show disagreement on any of the items about Grammarly because the lowest mean score was 3.24, which corresponded to “neither agree nor disagree.” Regarding their perceptions of Grammarly, both statistical and thematic analyses indicate that participants favored the advantages of Grammarly such as its ease of use and useful suggestions, which improved their confidence in writing. However, despite the relatively high means of ratings on the helpfulness of Grammarly’s suggestions, accuracy, and the quality of its explanations, the biggest reason that participants disliked Grammarly was its unhelpful, inaccurate, and decontextualized feedback. This might explain why participants gave relatively low ratings when it came to Grammarly’s helping them understand grammar rules better, its effectiveness in improving their grades, and improving their writing skills. Still, the overall results indicate that participants reported more reasons for liking Grammarly than disliking it.
RQ 2: F2F Tutoring at the Writing Center
Participants’ reported perceptions toward F2F tutoring at the writing center were also positive, with the means of all the ratings above 3.58. The thematic analysis revealed that participants’ reported reasons of liking F2F tutoring focused on the convenience and richness of face-to-face communication and the helpful feedback offered by tutors, which was corroborated by the relative high ratings on tutors’ helpful suggestions, good explanations of their suggestions, and their accurate and detailed feedback. On the other hand, participants disliked consulting tutors at the writing center mainly due to physical inconvenience such as time and location and a lack of confidence in tutors’ expertise, coupled with participants’ relatively low ratings on the convenience of the writing center. Generally, participants perceived the effectiveness of tutoring favorably, with tutoring increasing their writing skills, helping them get better grades, giving them more confidence in writing, and helping them understand grammar rules better.
RQ 3: Comparing Grammarly and F2F Tutoring at the Writing Center
Despite participants’ generally positive attitudes toward Grammarly and F2F tutoring at the writing center, their uses and perceptions of these two writing resources did reflect nuanced differences worthy of discussion.
To begin with, participants reported using Grammarly more frequently than visiting the writing center in general, while consulting writing center tutors about a wider range of issues than Grammarly. These two patterns correspond to the findings above: with the reported convenience to use Grammarly and the inconvenience to visit the writing center, it is reasonable that participants used Grammarly more often; similarly, with the convenience and richness of face-to-face communication with tutors and with Grammarlyonly focusing on grammar, spelling, punctuation (free version), it seems natural for participants to capitalize on the versatility of tutors and bring a wider range of issues to the writing center.
When participants were asked to compare their perceptions of using Grammarly and F2F tutoring, statistical analysis indicates that they tended to value both, with a higher frequency of using Grammarly and a lower tendency to replace the tutoring service at the writing center with Grammarly, with the differences being statistically insignificant. These findings align with our thematic analysis, where the majority of the participants pointed out both advantages and limitations of these two writing resources and reported choosing one to use depending on their specific writing needs and contexts. Therefore, instead of an either/or stance that leads student writers to choose one and give the other up, participants tended to adopt a both/and stance and incorporated both Grammarlyand F2F tutoring as useful resources in their writing processes. Additionally, when participants were asked to compare using Grammarlyand F2F tutoring regarding satisfaction, recommendation, and the likeliness of future use, they gave slightly higher ratings of tutoring than Grammarly, but the differences were not statistically significant.
Contribution to the Field
Our analyses above show that the majority of the participants found Grammarly to be a useful tool for ESL students in terms of providing helpful suggestions and detailed feedback, which supports findings from Cavaleri and Dianati, who indicate that Australian college students considered Grammarly to be easy to use. Based on the results of Cavaleri and Dianati and the current study, it can be suggested that Grammarly was perceived to be easy to use by both L1 and L2 speakers of English. In addition, Participants’ positive responses such as “Grammarly offers useful feedback,” “is convenient,” “accessible,” and “easy to use” confirm the usefulness of Grammarly for ESL students in previous literature (Cavaleri and Dianati; Soleimani; Razali; Karyuatry). However, the Australian college students in Cavaleri and Dianati’s study seemed to favor Grammarly more than the participants in the current study, because the majority of the Australian participants agreed or strongly agreed on the rated items, while the majority of our participants remained neutral on them.
Meanwhile, our participants provided some examples for disliking Grammarly. For instance, some participants reported that Grammarly is “inaccurate” and provides “decontextualized feedback.” This result supports previous findings on user experiences with Grammarly which emphasize the inaccuracy and the decontextualized nature of feedback provided by this online grammar checker (Nova; Schraudner). These comments confirm Perelman’s argument that grammar computer checkers can provide unreliable information. Our results confirm Grammarly’s drawbacks to provide imprecise feedback on errors and its inability to take contextual background into account. Thus, producing accurate and contextualized comments on writing remains as an area yet to be developed by the program developers of Grammarly.
Besides our participants’ generally positive perceptions of F2F tutoring at the writing center, they also reported negative feelings about tutors’ lack of expertise, which reflects a much-discussed issue in writing center scholarship. Empirical research reveals that tutors’ disciplinary expertise plays an important role in facilitating the efficacy of tutorials (Dinitz and Harrington) and that tutors develop strategies in order to tutor writing in which they lack subject knowledge (Summers). Consequently, based on our participants’ reported lack of confidence in tutors’ expertise, it can be suggested that tutors need to horn their strategies to help writers from diverse disciplinary backgrounds.
Additionally, moving beyond studies that equated the reduction of grammar errors to the improvement of writing (Jayavalan and Razali; Karyuatry et al.), our study probed into participants’ perceptions of Grammarly in a more nuanced way. The results of our study indicate that participants acknowledged both advantages and limitations of Grammarly. On one hand, they perceived feedback from Grammarly to be useful; on the other hand, their ratings were lower when they considered the effectiveness of Grammarly in helping them get better grades and improve their writing skills. In other words, participants’ positive perceptions of Grammarly did not seem to translate into actual improvement of their writing or their development as writers, which counters studies such as Jayavalan and Razali and Karyuatry et al. Consequently, while Grammarly holds potential for improving students’ writing ability, due to the shortage of empirical studies exploring Grammarly’s influence on student writing, the conversations surrounding the effectiveness of Grammarly in student writing development are inconclusive. This evinces the need for more research to investigate whether and in what ways using Grammarly might help cultivate writing development in students. Thus, with our findings about ESL students’ self-reported perceptions of Grammarly as a starting point for more scholarly discussion of Grammarly’s impact, we suggest that researchers conduct more systematic empirical inquiry into Grammarly’s effects on students’ writing improvement and development.
Another important contribution of this study lies in its reflection of a both/and approach toward Grammarly vs. F2F tutoring at the writing center. Among the few studies that examine or touch upon the relationship between Grammarly and the writing center, scholars seem to conduct comparisons based on an either/or approach that aims to judge which writing resource is better. For example, Dembsey conducted a rigorous and nuanced comparison between feedback provided by Grammarly and by writing center tutors asynchronously, indicating that tutors were more advantageous than Grammarly in that tutors were trainable, flexible, and interactive. Dembsey’s findings contribute significantly to our understanding of Grammarly in relation to the writing center and pose interesting questions: when we compare Grammarly against tutors and decide that one is more advantageous than the other, are we to choose between automated grammar checkers and human tutors? Our study suggests otherwise. Based on our findings, the majority of our ESL participants perceived that Grammarly and tutors both had advantages and limitations and participants reported choosing one to use depending on their specific needs and situations. For example, in situations where they needed to “check basic grammar,” they used Grammarly and benefited from its convenience and speed, thus using Grammarly to lessen the cognitive burden arising from grammar issues that they have to deal with while writing in English; when they needed to engage in in-depth discussions about their writing, they consulted tutors face to face. In this sense, the majority of our participants adopted a both/and approach: they treated both Grammarly and F2F tutoring as useful writing resources and they used them under different circumstances to their advantages. As compositionists and scholars, we endorse such a both/and approach toward Grammarly and the writing center, because such an approach enables ESL students to make full use of a wide range of available writing resources in their writing processes and to base their decisions on their needs and contexts. Therefore, we call on researchers to conduct more comparison studies to explore the affordances and effectiveness of Grammarly and F2F tutoring at the writing center, so that students—both L1 and L2 students—can make informed decisions when they utilize writing resources to aid their writing processes.
Implications
The results from this study contribute to discussions over “grammar checkers vs. writing tutors” in literature (Dembsey; Perelman), or even “technology vs. humanity” discussions in broader contexts. The advances in technology result in the creation of digital tools that aim to facilitate learning, such as online grammar checkers, and learners naturally desire to take advantage of these tools. Rather than arguing against the use of automated grammar checkers to improve writing, the current study advocates an open attitude toward available writing resources for ESL writers. Based on the results of this study, we propose that both Grammarly and F2F tutoring at the writing center can be used to support the development of ESL writers, helping them to become more savvy and resourceful writers. For example, we can encourage ESL writers to use feedback from Grammarly as heuristics to generate a list of concerns to bring to the writing center to discuss with tutors. Especially for ESL writers who might lack the metalanguage to discuss writing in English, Grammarly can provide them with terminologies that can help them to conduct more productive tutorials with writing center tutors. In other words, we encourage ESL writers to explore all writing resources available to them and incorporate them into their writing processes to their advantage, be them automated checkers or human tutors. For instructors and writing center tutors, we recommend that they explore the potential to utilize Grammarly in their teaching/tutoring pedagogy and provide guidance to help ESL writers make informed decisions about using different writing resources to meet different needs in different contexts. For instance, we suggest that instructors and writing center tutors engage in discussion with EFL students about their Grammarly usage, raising students’ awareness of the benefits and limitations of Grammarly and encouraging them to visit the writing center for more individualized and in-depth discussion about their writing. In addition, the limitations that participants reported about Grammarly and F2F tutoring raise important implications for both Grammarly program developers and the writing center administrators. For Grammarly, programmers need to improve the accuracy of Grammarly’s feedback to provide more reliable support for its users.
Limitations and Future Research
Most empirical inquiries have limitations and the current study is not an exception. The main limitation of this study is the small sample size which decreases the generalizability of the findings to larger contexts. With a relatively small sample (N = 43), this study offers preliminary findings only; thus, larger-scale studies are needed to further inquire into students’ perceptions of Grammarly and F2F tutoring at the writing center. We encourage writing center and second language writing researchers to conduct replication studies to establish more generalizable patterns on user perceptions of online grammar checkers and F2F tutoring. Another limitation of the study is that its analysis remains mainly at a descriptive level to identify and describe perceptions. More empirical research that employs inferential statistics is needed to draw causal-comparative relationships and make stronger arguments about the use of these services. Despite these limitations, this study offers some insights about ESL writing resources and development and initiates an important line of inquiry for future research to build on. Besides examining ESL/EFL students’ perceptions, future research could also look into L1 students’ perceptions’ of Grammarly in relation to the writing center. Particularly, we call on researchers to examine how a both/and approach to Grammarly and the writing center influence students’ writing processes and development. Another interesting line of research is to examine how the services provided by Grammarly, i.e., grammar, spelling, and punctuation, etc., are attended to in F2F tutoring at the writing center.
Acknowledgement
The authors extend great thanks to their professors, family, and friends, including Dr. Mary Stewart, Dr. Ben Rafoth, Krista Sarraf, 赖财兰, 王丽文, 王宝, 张云生, 阎杰. We also thank the attendees of our presentation at the International Writing Centers Association Conference in Columbus in 2019 and the two anonymous reviewers and the editors of Praxis: A Writing Center Journal for their valuable feedback on this article.
Works Cited
Bigert, Johnny, et al. “Grammar checking for Swedish second language learners.” 2004, pp. 33-47.
Burston, Jack. “Review of BonPatron: An online spelling, grammar, and expression checker.” Calico Journal, vol. 25, no. 2, 2013, pp. 337-347.
Cavaleri, Michelle, and Saib Dianati. “You want me to check your grammar again? The usefulness of an online grammar checker as perceived by students.” Journal of Academic Language and Learning, vol. 10, no. 1, 2016, pp. A223-A236.
Chen, Hao-Jan Howard. “Evaluating two web-based grammar checkers- Microsoft ESL Assistant and NTNU statistical grammar checker.” Computational Linguistics and Chinese Language Processing, vol. 14, no. 2, 2009, pp. 161-180.
Daniels, Paul, and Davey Leslie. “Grammar software ready for EFL writers.” OnCue Journal, vol. 9, no. 4, 2013, pp. 391-401.
Dembsey, J. M. “Closing the Grammarly® gaps: A study of claims and feedback from an online grammar program.” Writing Center Journal, Vol. 36, no. 1, 2017, pp. 63-96, 98-100.
Dinitz, Sue, and Susanmarie Harrington. “The role of disciplinary expertise in shaping writing tutorials.” Writing Center Journal, vol. 33, no. 2, 2014, pp. 73-98.
Duffy, Jill. “I improved my writing with Grammarly, and so can you.” PC Magazine, 2016. https://www.pcmag.com/commentary/342625/i-improved-my-writing-with-grammarly-and-so-can-you
“Great Writing, Simplified.” Grammarly, www.grammarly.com. Accessed 12 May. 2019.
Jayavalan, Kalpana, and Abu Bakar Razali. “Effectiveness of online grammar checker to improve secondary students’ English narrative essay writing.” International Research Journal of Education and Sciences, vol. 2, no. 1, 2018, pp. 1-6.
Karyuatry, Laksnoria, Muhammad Dhika Arif Rizqan, and Nisrin Adelina Darayani. “Grammarly as a tool to improve students’ writing quality: Free online-proofreader across the boundaries.” Edulitics Journal, vol. 3, no. 1, 2018, pp. 36-42.
Matsuda, Paul Kei, and Michelle Cox. “Reading an ESL writer’s text.” ESL writers: A guide for writing center tutors. 2nd ed., edited by Shanti Bruce and Ben Rafoth, Boynton/Cook, 2009, pp. 42-50.
North, Stephen M. “The idea of a writing center.” College English, vol. 46, no. 5, 1984, pp. 433-446.
Nova, Muhamad. “Utilizing Grammarly in evaluating academic writing: A narrative research on EFL students’ experience.” Premise: Journal of English Education, vol. 7, no. 1, 2018, pp. 80-97.
O’Regan, Brendan, Annick Rivens Mompean, and Piet Desmet. “From spell, grammar and style checkers to writing aids for English and French as a foreign language: Challenges and opportunities.” Revue Française de Linguistique Appliquee, vol. 15, 2010, pp. 67-84.
Perelman, Les. “Grammar checkers do not work.” WLN: A Journal of Writing Center Scholarship, vol. 40, no. 7-8, 2016, pp. 11-20.
Potter, Reva, and Dorothy Fuller. “My new teaching partner? Using the grammar checker in writing instruction.” English Journal, vol. 98, no. 1, 2008, pp. 36-41.
Qassemzadeh, Abolfazl, and Hassan Soleimani. “The impact of feedback provision by Grammarly software and teachers on learning passive structures by Iranian EFL learners.” Theory and Practice in Language Studies, vol. 6, no. 9, 2016, pp. 1884-1894.
Radi, Odette Bourjaili. Teachers’ Perceptions of the Individual Case Studies’ Literacy Performance and Their Use of Computer Tools. International Association for the Development of the Information Society, 2014.
Radi, Odette Bourjaili. “Studies Relating to Computer Use of Spelling and Grammar Checkers and Educational Achievement.” International Association for Development of the Information Society, 2015.
Saldana, Johny. The Coding Manual for Qualitative Researchers. Sage, 2009.
Schraudner, Michael. “The online teacher’s assistant: Using automated correction programs to supplement learning and lesson planning.” CELE Journal, vol. 22, 2014, pp. 128-140.
Summers, Sarah. “Building expertise: The toolkit in UCLA’s graduate writing center.” Writing Center Journal, vol. 35, no. 2, 2016, pp. 117-145.
“Cool Tool: Grammarly.” TD Magazine, 2017. https://www.td.org/magazines/td-magazine/cool-tool-grammarly
Vernon, Alex. “Computerized grammar checkers 2000: Capabilities, limitations, and pedagogical possibilities.” Computers and Composition, vol. 17, 2000, pp. 329-349.
Appendix A: Figures
Figure 1: Screenshot: Grammarly enables users to create a rhetorical situation by making choices about intent, audience, style, emotion, and domain
Figure 2: Screenshot: Interface of Grammarly
Appendix B: Tables
Table 1. Descriptive Statistics for ESL Writers’ Perceptions of Grammarly (N=34)
Table 2. Thematic Analysis: Likes and Dislikes about Grammarly
Table 3. Descriptive Statistics for ESL Writers’ Perceptions of the Writing Center (N=36)
Table 4. Thematic Analysis: Likes and Dislikes about F2F Tutoring at the Writing Center
Table 5. Frequency of Use: Participants Who Used Both Grammarly and the Writing Center (N=27)
Table 6. Purpose of Use: Participants Who Used Both Grammarly and the Writing Center (N=27)
Table 7. Descriptive Statistics: ESL Writers’ Perceptions of Grammarly in relation to F2F Tutoring at the Writing Center (N=27)
Table 8. Descriptive Statistics: ESL Writers’ Perceptions of Their Satisfaction, Recommendation, and Future Use of Grammarly and F2F Tutoring
