Mini Merlin

back to home

Members
Joy Agustin (jagustin@hawaii.edu)
Astrid Tomsic (astrid_tomsic@yahoo.com)

Memos

Plans

Requirements Analysis

Usability Testing
Instructions
Consent Form
Background Survey
Participant Script
Evaluator Script
Data Collection Form
Post-Usability Survey
   Interviews
   Interview-faculty
   Interview-alumni
   Interview-student
   Usability Testing 1
   Background Survey
   Data Collection Form
   Post-Usability Survey
   Usability Testing 2
   Background Survey
   Data Collection Form
   Post-Usability Survey
Post-Usability Survey
Expert Evaluation

Conceptual Design

Prototype

Participant ID: 3
(Student)
Session Date: 11/04/2002
Start Time: 16:30
End Time:
Task Description Number of Errors (1) Elapsed Time Participant’s Actions and Comments Evaluator’s Observations
1. Locate Resume Index 0 2 secs

User quickly scanned the page and found the "Resume Index" link.

Note: user previously added a resume to the system, so already familiar with the location of the "Resume Index" link.

2. Add a Resume 0 69 secs

Scanned the ICS home page for the Resume Index link. Clicked on the link to get to the Resume Index page. Scanned the instructions at the top to find the login link. In an attempt to log in, the user paused to remember what password to use. After seeing the Preferences page, the user indicated the Resume section and claimed that to add a resume, one would have to type in the information, check the "Show Resume" box, and then click on the "Make Changes" button.

Note: since the user is already familiar with the steps to add a resume, the task was modified to repeating the action of how to add a resume.

3. Update Personal Preferences 1 60 secs

Repeating the steps of the previous task, the user quickly updated the Preference section with the desired information. After submitting the changes, the user tried clicking the "career zone" link on the left side to return to the Resume Index. Remembering that the link is broken the user commented that he should have remembered it was broken, and then pressed the back button on the browser.

After this task was completed, the user quickly went back to the Preference page to revert back to the original data. When asked why the data needed to be changed quickly, the user replied that since he didn't use the service much, he might forget to change the data at a later date. He didn't want to be known as "bob".

Although the user has previous experience with the service, twice was the "broken" "career zone" link on the Preference page clicked to return to the Resume Index page. The first time was to check that the update was committed, the second when the data was returned to its previous values and the user wanted to ensure the changes were committed.

Both times the data was updated, the user needed to make sure the Resume Index web page reflected those changes.

4. Find and view a Resume ("HTML" skills) 0 15 secs

After reaching the Resume Index page, the user scanned the skill sets that were visible by default on the page. The HTML skill was found in the first skill set. To view the resume, the user clicked on the resume icon.

The first thing the user did was scan the page before trying any finding tools.

5. Find and view a Resume ("Pascal" and "COBOL" skills) 0 15 secs

After reaching the Resume Index page, the user used the Control-F combination to bring up the browser's find in page tool. First searched for "pascal". When no results showed up, searched for "cobol". When no results showed up again, the user claimed that he gave up, there was no resume with those skills.

The timer was stopped after the user defeat.

When the evaluator erroneously insisted that a resume had to be there, the user then scanned the instructions for the "Search Resumes" link and tried searching with that. After that didn't return any results, the user insisted that there were no resumes fitting that criteria. The user had faith that the searching tool on the web page worked properly.

5b. Find and view a Resume ("SQL" and "java")

 

 

0 33 secs

After reaching the Resume Index page, the user used the Control-F combination to bring up the browser's find in page tool and typed in "Java, sql". When no results showed up, the user then tried "Java" by itself in the find in page tool. After a couple of tries, he realized that this approach wouldn't work, so tried the "Search Resumes" link. In the search he typed in "java,sql", following the example on the page. That returned every resume that contained either "java" OR "sql". Commenting that the search was not good since he expected "java" AND "sql", the user then manually scrolled through the results on the page to find the resume and opening it. He mentioned that the search should allow for both options so it's easier to narrow down searches.

This task was added in to see what the user would have done to find a resume according to multiple skills.
6. Find and view a Resume ("SQL" skills and most recent graduate) 0 47 secs

From the Resume Index page, the user automatically clicked on the "Grad Date" column header to sort the column by graduation date. After scrolling a little to verify that the column was sorted, and in what order, the user headed straight to the bottom of the page and manually looked through the skills. Every so often he needed to stop to remember what date he was looking at because expected graduation dates are also posted. The resume viewed belonged to "Steven A. Miyakawa".

The evaluator should have recorded the names of the owners of previous resumes for verification later. However, due to an oversight, this wasn't done until now.
7. Find Resume Count 0 22 secs From the Resume Index page the user scanned the Instructions for the "Search Resumes" link, then clicked on it. After typing in "javascript", the user manually counted the number of results (5 resumes). The user didn't consider any other option but the service's search tool.
8. Remove a Resume 0 22 secs After successful login, the user scanned the Preference page for the "Show resume" checkbox. After clicking on the box, he submitted his changes, clicked the back button to verify that his name was no longer displayed. The biggest hang up here was typing in the password to log in.
(1) An error or negative critical incident is something that happens while a participant is working that has a significant negative effect on task performance or user satisfaction, and thus on usability of the interface.
Last modified: Mon Dec 09 16:37:01 Hawaiian Standard Time 2002