Dainan Skeem

Masashi Shimonao

Adnan Raza

LIS 665-Teaching Info Literacy

LILO Rubric Report

Spring 2006

Introduction

There are many ways to assess students’ work. However, as Trudi Jacobson and Lijuan Xu explain, “If you provide students with the criteria by which they will be graded, they will feel a sense of control over the work that you expect of them” (105). The best way of helping them to feel a sense of control, which happens to be the easiest for both instructor and student, is with a rubric. “If you give students a rubric, or set of criteria, by which their paper or project will be judged, they will understand much better what you are looking for, and will have an increased sense of control” (105).

In an interesting article from Ahrash Bissell and Paula Lemons, one can see the importance of creating measurable assessment for students. Bissell and Lemons, as college biology instructors, were interested in teaching their students critical thinking, something rarely done in their line of work. However, they quickly ran into a problem. “When trying to implement critical thinking as an explicit goal in introductory biology, we found ourselves without a well-defined scheme for its assessment” (66). They also realized that many others were having the same problem: how to assess students’ critical thinking.

Together, both instructors devised a method in which they would write questions for their students that contained the content of the biology course but that would also require critical thinking. These questions were given to other colleagues to be examined for accuracy and then a rubric was designed on how to grade the students’ responses. These were the results the instructors found: “Thinking in advance about what we want questions to accomplish in terms of both content and critical thinking has enabled us to be explicit with students about the skills they need to develop in order to succeed in the course” (70). Not only did this help the instructors, but it also helped the students.

We have reviewed questions and grading rubrics in our lectures and made examples of them available to students outside of class. As a result of this exposure, students were more aware of the quality of responses we expected for questions and could easily cross-reference their own responses with our explicit guidelines. Many students have communicated that they never understood how to "think critically" in other courses, even if they were asked to, because they were never shown what it means. (70-71)

It is interesting to see the impact a simple rubric can have on students. In their conclusion, the instructors stated, “the assessments are flexible, in that they can be easily amended to accommodate unforeseen answers, and can be weighted to favor either the critical-thinking component or the content component” (71). For all of these reasons, we are excited to be able to work with rubrics in this LILO report.

Procedures

In our assessment of LILO, we analyzed the section on plagiarism. This matched the Association of College and Research Libraries’ fifth standard:

“The information literate student understands many of the economic, legal, and social issues surrounding the use of information and accesses and uses information ethically and legally” (American Library Association).

There were two Performance Indicators within this standard that helped guide us in creating this rubric as well.

The first was PI 2: “The information literate student follows laws, regulations, institutional policies, and etiquette related to the access and use of information resources” (ALA).
- Within the PI was the specific Outcome F: Demonstrates an understanding of what constitutes plagiarism and does not represent work attributable to others as his/her own” (ALA).
- The questions asked in the LILO software that correspond to this Outcome can be seen in Tables 1-3 below.

Discussion

In her discussion about Rubrics as an assessment tool, Gratch-Lindauer emphasizes that it is very important for rubrics to have clear descriptions of evaluation criteria with examples in order to achieve high inter-rater reliability (33). Following that ideal as well as the instructions for this project, we created these criteria based on close analysis of students' answers for each question. Examples from students' answers were chosen to best exemplify each criterion.
In order to differentiate those who do not provide any answers or give nonsensical from those who provide more reasonable answers and show a minimal understanding of the nature of the questions, we set up two different points (0 and 1) for beginning level instead of just one. As a result, those who are considered Proficient and Advanced were given 2 points and 3 points, respectively.

Table 2 As Illustrative of Rubric Creation Process

Table 2 is based on the answers the students gave to the LILO question "Describe, in your own words, the differences among quoting, summarizing, and paraphrasing."
Incorporating Jacobson and Xu's ideas, it is important to be clear on what we are measuring to create good rubrics. Meeting this aspect was not very difficult for most of our prompts. For example, here, what we are measuring is how correct the definitions of the three different citation styles students provided are.
For 0 point, student either gives no answer, a totally irrelevant answer, or nonsensical answer. An example of such an answer is shown in the examples in the table 2 at the next section.
We give 1 point when student answers show at least a general understanding about what is being asked although they fail to articulate the particular definition to any of the citation styles. The answers are either too broad or show misunderstanding about the concepts.
We give 2 points, when students are successful in giving at least one or more parts of the correct definitions to the any of the citation styles but are still missing some important points.
We give 3 points, when students have given comprehensive coverage on the definitions for the three styles of citations.

Rubrics

Table 1 Student Identification of Plagiarism

Evaluation Criteria

Beginning

0-0.5-1

Proficient

Advanced

Student Learning Outcomes

Identifying the most common example of plagiarism

Student gives no explanation, explanation is contradictory, or explanation is unintelligible.

Student gives at least one example of plagiarism.

Student gives an example and justifies his/her reasoning.

LILO 6.1

(ACRL 5-2-f)

Defining Examples

0 -

I am not sure what this means I am not sure what this means I am not sure what this means I am not sure what this means I am not sure what this means I am not sure what this means I am not sure what this means I am not sure what this means

1 – I think that in advertising companies like to intentionally misrepresent their competitors to make the own company look better. I see this even more in political advertising.

Copying someone else’s ideas without giving credit is probably the biggest type of plagiarism because students love to procrastinate and then get in a jam and can’t do the work by themselves by deadline.

Accidentally or intentionally misrepresenting someone else's words or ideas.

Table 2 Differentiating citation styles

Evaluation Criteria

Beginning

0-0.5-1

Proficient

Advanced

Student Learning Outcomes

Differentiating among quoting, summarizing, and paraphrasing

Student does not articulate the distinction between any of these three terms.

Student explains one or two of the terms but misses the third.

Student explains all three terms.

LILO 6.2

(ACRL 5-2-f)

Defining Examples

0 –

using the authers words and citeing, summerising is mostly your ideas from paper jsut read

quoting is when someone said something, summarizing is just a summery,

1 –

These are restating the message what the author is trying to say.

quoting you are taking the words from the mouth, summarizing is like a theisis statement and parapharsing is reiterinating what has already been said.

Quoting is just a few words....summerizing can be a whole book...and paraphrasing is saying the same thing in a differen't way.

A quote is exactly like the original. Summaries are in your own words.

quoting using the authors own words. Summarizing is using your own word after reading the authors original work. Your saying the same things that the authors saying but in different order.

When you quote an author verbatim ("word for word") , take the source and add quotation marks at the beginning and ending of the passage. To avoid plagiarism, you must identify the source of the quotation using a citation format, like MLA or APA (more abo

Quoting is using the authors exact words, it is shown by using quotation marks. Summarizing is using the authors main points or ideas. Paraphrasing is just restating the authors ideas, points, and words. Summarizing and paraphrasing do not need quotation

quoting is direcly using exactly the same words as the origial author. summarizing is using your own words to sum up an idea made by an author, and paraphrasing restates all the informatin that an author said, not just the main idea. with both paraphrasing

Table 3 Knowing when to cite

Evaluation Criteria

Beginning

0-0.5-1

Proficient

Advanced

Student Learning Outcomes

Knowing why to quote, summarize, and paraphrase

Student gives no explanation, or explanation is nonsensical.

Student articulates some rational for using any of these terms.

Student specifically states why he/she would quote, summarize, or paraphrase.

LILO 6.3

(ACRL 5-2-f)

Defining Examples

0 – It would depend on how the information in my head compares to what the author wrote.

1 – word for word. general. rearrange.

It depends on how long the article is, how much information you need, and what you need from the article.

if I could not explore the idea in my own words I would quote, if the only information I needed for the paper was the main idea I would summarize, a key section with many important points I would paraphrase.

Future Research

One of the questions that we discussed in group meetings but ultimately did not include in our rubric design was “Which citation style are you familiar with?” Realistically, the answer to this question serves no purpose to the evaluators. All the student has to do is type one in. It wouldn’t matter if they really knew the particulars of the style or not. Therefore, this question cannot be accurately assessed and LILO should consider eliminating it.

The other questions seemed to be good questions to judge the students’ understanding of the subject of plagiarism. As can be seen from the majority of these responses, though, not many of the students are learning what the instructional sequence of LILO is trying to teach. A very large portion of the responses were incorrect. It seemed many were just typing to fill in the space. Future research should be done to find out if this tutorial on plagiarism is really effective.

Conclusion

Having created rubrics from the LILO tutorial this is a list of things that we have found beneficial as future academic librarians:

· Fully understanding the design of a rubric.

· How a rubric can be implemented into an assignment.

· What to expect from students’ responses (quality of content).

· Why rubrics are important in being a consistent grader.

· How powerful rubrics can be for the instructors, but even for the students.

· Becoming familiar with this useful tool that can make it possible for easier, more objective and thus better grading. As is discussed in the online article from California State University at Fresno, good rubrics can allow us to do "criterion-referenced grading," so that teachers do not resort too much to "norm-referenced grading" in which students' performance is judged by the vague standard of how each of student performed compared to the other students.

How will rubrics contribute to assessing information literacy competence for instruction programs?

· Since the LILO tutorial considers complex student behavior and the demonstration of practical knowledge about writing research papers, evaluating students' answers is a highly challenging task. We learned to appreciate the value of creating rubrics which can provide clear evaluation criteria for students' expected achievements. Without rubrics, the efficient grading and evaluation of students' performance for online tutorials would be nearly impossible.

References

American Library Association. “Information Literacy Competency Standards for Higher Education.” Association of College and Research Libraries. 2005. Accessed 3 May 2006. < http://www.ala.org/ala/acrl/acrlstandards/informationliteracy competency.htm#useofst>

Bissell, Ahrash and Paula Lemons. “A New Method for Assessing Critical Thinking in the Classroom.” BioScience. 56.1 (2006): 66-72.

Jacobson, Trudi and Lujuan Xu. Motivating Students in Information Literacy Classes. New York: Neal-Schhuman Publishers, Inc., 2004.

Gratch-Lindauer, Bonnie. "Selecting and Developing Assessment Tools." Chapter 3 in Elizabeth Fusler Avery, Ed., Assessing Student Learning Outcomes for Information Literacy Instruction in Academic institutions. Chicago, IL: ALA/ACRL, 2003. pp. 22-39.

California State University at Fresno. 2002. Teaching, Learning and Technology: Using Scoring Rubrics. http://www.csufresno.edu/cetl/assessment/UsingScoringRubrics.pdf