ESL 610

Prof. G. Crookes

May 1997

ESL College Writing Portfolios:

A First-Attempt Saga

Background and Introduction

I first experimented with the use of portfolio assessment while teaching an advanced-level ESL academic writing class consisting of eight college students from East and Southeast Asia. My choice to veer away from traditional grading resulted from two events which roughly coincided with my students' completion of their first of five major take-home writing assignments. One catalyst for my interest in portfolio assessment came from the instructor of a course in which I was enrolled, who had rather rockily started the semester by challenging my classmates and I to design much of our own syllabus, to assign ourselves individual or group term projects which we would find personally meaningful, and, finally, to monitor our own progress throughout the term, perhaps culminating in a term-end self assessment, in which each of us would evaluate the extent to which we had met our own stated goals for the course. Feeling abandoned in the absence of clearly defined objectives and externally determined assessment criteria, I panicked and resisted. Seventeen and a half years of successfully "playing the game" of discovering what my instructors wanted and doing my best to give it to them had left me totally unprepared for a situation in which my objectives were centrally important to the course of my study. Much as I recoiled against this newfound academic uncertainty, it is ironic that within a few weeks, I was putting my own unsuspecting students in a very similar situation.

I had started teaching that term with the most traditional of intentions. I had dutifully photocopied the administratively prescribed course objectives and grading policies, with flat percentage values attached to each major upcoming assignment. Papers would be assigned letter-grades which would later be converted into point values for the final course-grade calculation. These were exactly the specifications my students had expected; no questions asked, they got to work on the first writing assignment. Little did they know that by the time they handed those papers in, I would encounter the second impetus for a change in my assessment strategy.

Reading for a research project comparing ESL student responses to instructor- vs. peer-provided feedback on writing assignments, I stumbled upon alarming studies which found that student interest in instructors' feedback tended to begin and end with a glance at the letter-grade. Students reported that they rarely, if ever, looked back on previous assignments, and most reported that, at best, they merely "mentally noted" any corrective feedback their instructor had provided (Leki 1990, Cohen 1992). Furthermore, ESL writing instructors had been found to orient their feedback around the perception that each of their students' assignments existed in isolation, rather than as a link in a developmental process (Zamel, 1995). Finally, reading on the process approach to writing (Kutz, Groden, & Zamel, 1993) convinced me that each draft my students wrote should be looked upon as practice for the next they would write. Even the final draft of a paper could be considered as a source of feedback through which they could improve future papers. I settled on portfolio assessment as a means to foster these same considerations in the minds of my students. With the ungraded (but heavily commented-upon) final drafts of their first major writing assignment, I passed out copies of the following explanation of my change in plans:

Portfolio Grading

Too often, when the final draft of a writing assignment is returned to a student, the student glances at the grade, briefly experiences either joy or dismay, and then files the assignment away where it will never be seen again. In a class such as this one, where the process of writing is our main focus, individually grading assignments could seriously undermine our goal by suggesting that the process ends with the grade on each paper, and begins again as we start a new paper. Of course, the process of learning to write well should be ongoing, with each assignment building on previous ones. Therefore, in order to encourage you to refer back to old assignments and to seriously consider all comments on every draft, I propose adopting a policy of "portfolio grading" in this class.

By "portfolio grading," I mean that your grade in each area of the course (i.e. out-of-class writing, in-class writing) will reflect the sum of all your work in that area, taking into consideration your improvement over time, and your attention to targeting and conquering your own weak spots. Individual assignments will be returned to you ungraded; you will be responsible for keeping this work, for using it as a reference as you develop future assignments, and finally, for returning all of the work to me in a portfolio at the end of the semester. The portfolio grades will make up the percentages of your final course grade, as indicated in your syllabus.

If at any time you feel anxiety about the progress of your portfolio, I encourage you to bring your work to me for discussion in my office hours. This policy is not designed to cause discomfort, but rather to give greater purpose to each of your assignments, by showing you how each is part of a bigger picture.

Watching my students' faces as they read the above statement, I saw expressions transform from sympathetic interest to confusion to rejection. We spent half an hour of class time discussing the new policy. Students wanted to know why I wouldn't give them grades. How would they know how they were progressing in the class? How would they know if they needed to work harder? I explained that my extensive feedback on each of their forthcoming papers would answer all those questions, along with many others which could never be communicated through A's, B's, C's. I encouraged them to come talk to me about individual assignments or overall questions about performance. For one who had so recently felt their pain in my own academic pursuits, I really presented an impressive sales pitch, I thought. Nevertheless, my new grading policy was met with skepticism and mistrust.

In hindsight, armed both with one semesters' experience with portfolio assessment and with a fair amount of reading on the topic, I understand that my students' resistance to the notion of self-reflection in the place of explicit instructor evaluation (and my own resistance, back in the class I was taking) cannot be blamed entirely on their previous educational expectations and the newness of the portfolio proposal. Specialists in educational assessment as well as practitioner-researchers who have experimented with portfolio development generally agree that several components are essential to the successful implementation of a portfolio system with both instructive value and assessment validity. The absence of some of these components in the portfolio system I implemented explain certain areas in which my system broke down, while the presence of other important components explain how, while unguided by prior experience or knowledge of the literature on alternative assessment, I was able to extract certain meaningful and instructive results through the use of portfolios in my ESL writing classroom.

Literature Review

Portfolios have been proposed as an alternative to tests as a means of educational assessment, due to several problems with the latter, as succinctly outlined by Wolf (1989:36):

... much school-based assessment actually prevents students from becoming thoughtful respondents to, and judges of, their own work. The "surprise" nature of many test items, the emphasis on objective knowledge, the once-over and one-time nature of most exams- all offer students lessons that are destructive to their capacity to thoughtfully judge their own work: (1) assessment comes from without, it is not a personal responsibility; (2) what matters is not the full range of your intuitions and knowledge but your performance on the slice of skills that appear on tests; (3) first-draft work is good enough; and (4) achievement matters to the exclusion of development.

While the multiple drafting inherent to most ESL writing courses appears to separate them from courses which rely on exam-based assessment, Wolf's concerns transfer to writing classrooms remarkably well. Certainly, in most writing classes, assessment comes exclusively from the instructor. Furthermore, even in courses which claim a "process approach," this external assessment may appear on every draft of an assignment, with full points granted to a "first draft" only if it is already nearly perfect, and full points assigned to a final draft only if the student-author has understood and meticulously incorporated all of the instructor's corrective feedback. In other words, every paper or even every draft may be graded as a "test," representing only a slice of skills and seen only from the instructor's perspective. In such cases, achievement does matter "to the exclusion of development," as assignments are never revisited once the grade goes down in the instructor's book. Each paper is read in isolation, one paper sometimes receiving a lower holistic grade than a previous one despite improvements in a specific area which had caused problems in the holistically "superior" earlier paper.

Portfolios, on the other hand, "can become a window into the students' heads, a means for both staff and students to understand the educational process at the level of the individual learner" (Paulson, Paulson, & Meyer 1991: 61). However, portfolios must be carefully designed in order to meet this lofty potential. First of all, portfolios must be separate and different from cumulative folders; they should never become "a place to collect more isolated skills tests" (Valencia 1990: 339; Jongsma 1989, Paulson et al. 1991, Smolen et al. 1995). Rather, choices must be made in determining the ultimate contents of student portfolios, and ideally, these choices should be made, partially or completely, by the students themselves (Paulson et al. 1991:61, Valencia 1990).

In a well-designed and managed portfolio system, students will learn how to honestly and effectively evaluate their work against a clearly defined set of specific criteria. "If the goals of instruction are not specified, portfolios have the potential to become unfocused holding files of odds and ends..." (Valencia 1990: 339). Therefore, instructors and students must work together from the very beginning of the semester to solidify and document the individual and collective goals of the course. (Jongsma 1989, Arter and Spandel 1992, Lynn 1995). These goals can then function as tangible points of reference from which students and instructors alike can measure progress (Arter and Spandel 1992, Smolen et al. 1995).

Arter and Spandel (1992) emphasize that the criteria collaboratively determined by instructors and students must lead to authentic portfolio contents in order for portfolios to have any instructive or assessment value. To accomplish this aim, "the content of the portfolio should mirror the emphasis in the curriculum and classroom," instructors must be sure to assign tasks which match the goals for portfolio contents, and "the portfolio system must not be seen as an add-on to the `real' instruction taking place in the classroom (38). The most effective portfolios will be those which are treated as a regular, ongoing and integral assignment which compliments and enhances students' other work.

Even when presented with clearly defined achievement criteria, students being introduced to portfolios for the first time are unlikely to have the skills or even the desire to become involved in their own assessment. Lynn (1995: 37) reminds instructors experimenting with portfolios that :

...in most cultures teachers and students are not considered equivalent in terms of mastery of the subject matter... the teacher is viewed as an expert in the subject at hand, a fact readily accepted by students who want to learn from someone who knows what they are talking about. ... Any attempt, therefore, to introduce new trends of looking to the student for... assessment must be handled carefully and with expertise. The very idea of incorporating students into the decision-making process of doing things in class may strike students as odd.

Needless to say, most students have not had experience in evaluating their own work. Tracey Whathen (Smolen et al. 1995) found that when she first introduced her middle-school ESL class to a portfolio system in which students wrote their personal goals for each assignment on index cards and later reflected on whether they had met those goals, "the goal cards initially failed to live up to expectations because students were writing simplistic goals" (23). At first, students' goals tended to focus merely on completing assignments rather than on completing the work in a specific way, for a specific learning purpose. Whathen guided her students to a better understanding of the "goals" she expected them to set and meet by leading the class in a brainstorming session about the factors which constituted good work, and then listing the factors which emerged on large posterboards around the classroom. McNamara and Deane (1995) have found that asking students to write letters to the teacher in the first week of class, discussing their strengths and weaknesses in English and the area of the language in which they feel they need improvement, serves as an important first step towards "placing the responsibility for learning with the students, not the teacher" (17). Arter and Spandel (1992) recommend showing students samples of "good" self-reflection so they will know what it looks like as they begin trying to produce it. They further provide the following five questions to guide novices into reflexive assessment (Arter & Spandel 1992:3):

* Describe the process you went through to complete this assignment. Include where you got ideas, how you explored the subject, what problems you encountered, and what revision strategies you uded.

* List the points that made by the group review of your work. Describe your response to each point- did you agree or disagree? Why? What did you do as a resut of their feedback?

* What makes your most effective piece different from your least efective piece?

* How does this activity relate to aht you have learned before?

* What are the strengths of your work? What still makes you uneasy?

Using guiding questions such as these can help students begin to recognize and compare their own strengths and weaknesses and to reflect on thier own learning processes.

Finally, the reflexive assessment involved in selecting portfolios should be continuous throughout the term, with instructors and students meeting individually at several points to discuss the student's portfolio selections, and whether those selections are truly justified by evidence of growth in the student's abilities. (Jongsma 1989, Valencia 1990). In these ways, students can receive feedback on thier self-assessment itself, thereby gaining training in critical thinking, problem solving, and independent thinking simultaneously to mastering course content material (Spandel et al. 1992).

In order for students to feel comfortable selcting materials to place in thier portfolios, the purpose(s) and ownership of the portfolio must be clarified. Possible purposes for keeping student portfolios might be instructional, assessment-based, administrative, display (Gottleib 1995). The purpose of the portfolio may even change at different points during the semester; for example, an instructional portfolio which might contain problematic or partially completed work during the term could become a display portfoilio at the end of the term, containing only the work in which the student takes greatest pride (Paulson et al. 1991). In this case, the portfolio might undergo a change in ownership as well as in purpose. What was previously a collection of materials to be accessed and discussed by only the student and her instructor to manitor the student's progress through the course becomes a public record, available to classmates or even to school administrators or to parents as evidence of the student's growth. Knowing the purpose of the portfolio and its intended audience will help students to select the contents they feel comfortable sharing within that context.

Ideally, regardless of the purpose of a portfolio system, the portfolios should contain selections which show a broad range of the students' abilities, and should include examples of activities related to the processes as well as the products of learning (Jongsma 1989). Gottleib (1995) recommends "brainstorm[ing] with students and other teachers to generate a list of tasks, projects, or exhibits for the selected learning area" (13). In a writing class, these might include journals, brainstorming ecercises, and outlines in addition to both in-class and take-home essay assignments. Some instructors even allow materials not specifically produced for the course to be placed in the portfolio; for example, a letter to a friend might be a writing sample the student finds meaningful as documentation of confidence in her writing ability. If materials from outside the class are not permitted as portfolio contents, Arter and Spandel (1992) insist that classroom assignments should be tailored to mirror real-life contexts, so that the portfolio data will represent a student's preparation for the outside world as well as for advancement through school.

One further advantage to roprfolios which strive to meet the specifications explained thus far is that analysis of these portfolios will offer instructors opportunities to "review their instructional practices and revise curriculum to reflect content and performance standards (Gottleib 1995). Instructors who use portfolios and self-assessment execrises in thier classes will have the rare opportunity to see not only which assignemtns went well or poorly according to the products students produces, but also the students' own perceptions of factors which contributed to their relative successes and failures.

What I did "Wrong"

Plunging directly into a portfolio-style assessment style in my classroom without investigating previous literature on the issue beforehand, I fell into several of the "traps" which the articles cited above warn against. Looking back at the original statement I handed out to my students explaining my perception of "portfolios," one problem which comes immediately to my attention is the cumulative nature of the portfolios I proposed. I actually told my students that they would have to turn in all of their work for a grade at the end of the semester. Fortunately, I revised this position slightly before collecting the portfolios.