|
|
| Project Report |
| (MS
Word version) |
| 1. Introduction |
| |
|
Telementoring is a mentoring relationship or program in which the
communication between mentor and mentee is computer mediated, where
mentoring is defined as a sustained relationship between a youth
and an adult. In a well-structured mentoring relationship, the adult
provides help, support and guidance to the mentee.
The idea of telementoring is to use telecommunications technology
to support mentoring relationships between students and volunteering
field experts. This is particularly useful in situations where face-to-face
mentoring would be hard to carry out. We predict that telementoring,
both within traditional education and the business world, will be
a growing area in the future. It will connect mentees with field
experts in a worldwide context. As there currently does not exist
any customized software to support telementoring we believe this
is a great opportunity to get a competitive advantage and lead the
way for telementoring software. We will focus on the development
of a new software tool for the use of educational telementoring.
Typically students of lower grade levels have teachers with knowledge
of general topics in education. Therefore when students are working
on projects or field studies that require expert knowledge of a
particular field, their teachers cannot always provide them with
the assistance they need. In these cases students are often left
to seek additional knowledge via sources like the school library
and the Internet, where the abundance, or possibly lack of, information
can make it difficult and time consuming for the student to find
what is relevant for their particular project. Scientists and other
field experts may not be available in the local community, and students
might not know where and how to get in touch with authorities within
their field of interest. Currently the communication between students,
teachers, and field experts in a telementoring relationship is primarily
done by e-mail or in discussion groups. Our proposal is to make
software specifically for telementoring, thus considering the needs
of the various user groups in a telementoring relationship throughout
the design process. The software will provide a workspace where
this interaction between telementor and mentees can take place.
Early attention was given to usability, since a key advantage over
other existing tools will be that our tool is simpler for mentors
and mentees to use.
The methods used throughout this project includes root concept,
interview guide, user role modeling, artifact analysis, problem
and activity scenarios accompanied by claims analysis, essential
use cases, entity-relationship modeling, contexts, prototyping,
and usability testing. We believe that prioritizing spending time
on these methods allowed us to design a system that has clear advantages
over existing systems. With a growing need for telementoring software
due to the shortcomings of existing tools, we believe moving on
this now will give us the opportunity to gain a key market advantage
being the first to create this kind of tool. We have the opportunity
to be the first and set the standard, as well as be the only offering
such a tool
|
| |
| 2. Requirements |
| |
|
We started out defining the root concepts of our project in order
for us as a development team to have a common understanding of what
we were trying to achieve. Then we continued to define the requirements
for Telementoring Workspace software as we conducted interviews
with representatives for the core stakeholders in our project. Based
on the results of these interviews we were able to conduct a requirements
analysis for the system.
We prepared a user role model document that describes the roles,
backgrounds, expectations, profile, objectives and knowledge of
the various stakeholders. We also prepared an artifact analysis
that surveys some existing system offering features useful for telementoring.
In addition to this we prepared three problem scenarios, three activity
scenarios, and a derived claims analysis, which served to find problems
with the existing technology and propose solutions to these.
|
| |
| 2.1 Root Concept |
| |
| The first step we took in our project
was to write a Root Concept. This defined
what the goals and aim of our project are. We defined our vision,
rationale and our stakeholders for the project. Also, we defined our
starting assumptions and constraints for the project. This part of
the project gave us a starting point for the project- our starting
point, what we envisioned ourselves accomplishing, why it was necessary,
and who was involved. |
| |
| 2.2 Interview |
| |
|
We wanted to interview users to gain knowledge of some of the limiting
factors that we would have for our project. For example, we wanted
to know what computer equipment our users had, what kind of Internet
connection they had, and how much access they had to the equipment.
This allowed us to get a feel for what their needs were. We also
wanted to get an understanding for their knowledge background related
to telementoring. We were able to find out if our sample users were
familiar with telementoring (after explaining the definition), and
see how they currently interact with each other for similar types
of projects.
Interviewing also helped us to determine what type of information
is exchanged in related activities so we can design a system that
supports this type of information exchange in a more efficient manner
than the existing tools. This lead us to ask our stakeholders what
features on current systems were likeable or dislikeable, so we
could design a system that had advantages over the existing technology.
In preparation for the interviews we made an interview
guide which took into consideration the various stakeholders
recognized in our root concept.
Because of the short time available for this project we decided
to limit the interviews to a couple
of representatives for each of our three main stakeholders; mentors,
mentees, and the mentees supervisors. If there had been more time
available for this project we would have liked to do more interviews
in order to find general trends rather than individual preferences.
As representatives for Mentors we interviewed two graduate students,
as Mentees we interviewed a high school student and an undergraduate
student, and two K-12 teachers and a university instructor represented
the supervisor stakeholder group.
Before each interview we explained the term "Telementoring"
to ensure a common understanding of what it is.
We found that half of the interview subjects have experience with
telementoring / mentoring using Telephone, e-mail, and/or chat room.
Most expect telementoring to be some sort of relationship between
a student who could use help from an expert, and the expert with
knowledge in a particular area, and they would all like a tool that
is easy to use, and some expect some sort of a collaboration tool.
Most of our subjects expect a project to last from half a semester
to a semester in length, with some sort of presentation at the end.
Collaboration takes place throughout. All expect the outcome of
the project to be students having gained knowledge on a particular
subject, while utilizing tools such as mentors to aid them in their
work.
All our interview subjects have access to newer computer equipment
with a fast Internet connection whenever they wish. Some are experts,
the rest have a moderate amount of computer expertise mostly using
text editing software, browsing the Internet, and playing computer
games. The subjects had varying likes and dislikes when it comes
to software and features, but they all agreed that being easy to
use is an important trait.
Most of our subjects liked meeting in person, and disliked using
chat rooms because they weren't private. All are familiar and like
to use e-mail, and all but one are familiar with discussions, and
think they have advantages / disadvantages, such as anonymity. All
would appreciate a mixture of chat and discussions to accommodate
the most number of people. Most would like to know whom they are
working with, some personal information, and some credentials. If
participating in a telementoring they would like to know some personal
information, then some information on the details of the project.
As a mentor they would like to know about the relationship between
stakeholders, and the mentee's general knowledge level, so information
can be presented for their level, and they would expect to spend
around two hours per week telementoring.
|
| |
| 2.3 Artifact Analysis |
| |
Because of the lack of existing telementoring
software we did an artifact analysis
of systems containing aspects and features useful for mentors, mentees,
and supervisors that are looking for ways to communicate with each
other. We found that the various tools are mostly poorly integrated.
The free tools has a bonus for those with a low budget, but they all
contain components which require downloading and installing software,
which requires memory space and may for some users also require retrieving
permission and assistance from a system administrator. Any of the
three free suites analysed provide a bit of simplicity in the fact
that you only need one login / password to access any of the services.
But they all suffer from "sneaking featurism" because of
their commercial nature and the expectations of continuously upgrading
their products by adding more features. WebCT also has bundled features
that are available under the same login and password.
We definitely see the need for a fully integrated system tailored
for telementoring, offering artifact sharing and asynchronous and
synchronous communication that can be archived in order to preserve
useful information for the benefit of others. The system should be
accessible to the users via an Internet browser, and not require installing
additional software. The system needs to be easy to use and easy to
navigate in order to retrieve information, and should be stripped
for unnecessary features and functionalities that may confuse and
distract the users. |
| |
| 2.4 Problem/ Activity Scenarios |
| |
Based on the findings in the interviews
we wrote three problem scenarios (1 ,2
,3)describing problems encountered by
users when using software that is designed for other purposes than
telementoring. Writing these scenarios was an iterative process as
we consulted users in order to make them as representative as possible.
As a response to the problem scenarios we then wrote three activity
scenarios illustrating how we proposed the Telementoring Workspace
software to provide improved support for telementoring. These scenarios
were presented to some users in order to get their opinion about the
solutions they proposed |
| |
| 2.5 Claims Analysis |
| |
| Both problem- and activity scenarios were
accompanied with claims analysis emphasizing
the positive and negative aspects of the features presented in them.
The claims analysis made it clearer what tradeoffs were connected
to the various features present in the scenarios. |
| |
| 2.6 User Role Models |
| |
For our User
Role Model forms, we put our stakeholders into three categories:
mentor, mentee, and supervisor. First, we viewed our mentor as the
'outside expert' who voluntarily was brought into a project to assist
someone (the mentee) who is in need of expert help or guidance. Second,
we thought of the mentees as those who would be benefiting from outside
expertise- those with the projects that they would need expert assistance
with. Typically the mentees would fall into the category of being
in school. Our third stakeholder was the supervisor. We envisioned
the supervisor role as anyone who was overseeing the project. In the
case where the mentee is a student, the supervisor would probably
be the student's teacher / professor. The supervisor could oversee
multiple projects, and interact in the communication if they chose
to do so.
Overall, the User Role Model forms helped us categorize each of our
three users, and see more clearly the properties of the stakeholder
that we are designing our system for. |
| |
| 3. Design and Prototype |
| |
For the design phase we prepared some
essential use cases, which served to isolate the design issues we
determined in our activity scenarios. Further we created an entity-relationship
diagram showing the components of the new system and how they inter-relate
with each other. Based on these documents, along with the Activity
Scenarios, we made some Contexts. The relations and transitions between
the various the contexts were shown in a Navigation map.
After revisiting documentation from our requirements analysis and
design process with representatives from our user group we developed
an interaction scenario with a related prototype describing a new
user's meeting with the Telementoring Workspace. In addition we have
a second view of the same prototype showing an experienced user logged
in at TW. Finally we did some usability testing of aspects of our
prototype. The test results emphasized strength and weaknesses in
the Telementoring Workspace software, and gave us a better impression
what aspects of the system need to be improved.
The results of these methods can be seen in the appendix. |
| |
| 3.1 Essential Use Cases |
| |
| We developed Essential
Use Cases for the purpose of re-designing some of the issues we
came up with in our Problem and Activity Scenarios, and the Claims
Analysis. The Essential Use Cases show the envisioned user action
and system response required for the system's 'essential' tasks. However,
they are still very abstract, so that implementation decisions are
not made. Our Essential Use Cases show the main features of our proposed
system, such as using the chat, discussion, searching for other discussions,
adding a friend to your 'friend list', uploading a file, creating
new workspaces, and viewing recent activity. These Essential Use Cases
provide us with an abstract picture of how our envisioned system will
work. A lot of our design work was done on this part of the process,
and takes one step closer to creating a prototype. |
| |
| 3.2 E-R Model |
| |
| We created an entity-relationship
model showing the main components of the new system and how they
inter-relate with each other. This was used as a communicative tool
for the designers to have a common understanding of what the main
elements in the system would be and how they would interact. |
| |
| 3.3 Context Navigation Map |
| |
| The Context
Navigation Map provides navigation between the Contexts we created.
On our individual Contexts, the dark purple bars designate links to
external Contexts. All of these links can be seen together in the
Context Navigation Map, and provide a good high-level view of the
system and how one can move through it. For a description of the Contexts,
see below. |
| |
| 3.4 Contexts |
| |
Our Contexts
were based on our Essential Use Cases, and are based on the nouns
and verbs that were in the Essential Use Cases. The nouns turned into
the items in the Contexts, and the verbs were the actions that one
can perform. Our Contexts were drawn with four colors. Dark yellow
designates the Context, light yellow designates a content display
area, dark purple designates a command tool that takes you to another
Context (see Context Navigation Map), and light purple designates
a command tool that carries out actions within the current Context.
The Contexts took the design we came up with in our Essential Use
Cases, and translated them into something visual that was very helpful
to us in making a prototype. |
| |
| 3.5 Interaction Scenario |
| |
| Our Interaction
Scenario describes some features that we felt were necessary to
include in the Prototype
that were not covered by the Essential Use Cases. Basically, they
covered a user using particular features of the envisioned system.
In our case, it tied in very closely to our prototype, and provided
a glimpse of a hypothetical user using our system. We were able to
link this to the prototype we made, so that when reading one can go
back and forth between the two to clarify what is being read. |
| |
| 3.6 Prototype |
| |
| The prototype
is an implementation of the system we designed in the steps above.
Being an initial prototype, we expected some problems with the interface,
which will hopefully be discovered later on in our usability testing.
We conducted icon testing to see the effectiveness of our chosen icons,
and followed with prototype testing for feedback on the complete prototype
/ implementation. |
| |
| 4. Evaluation and Redesign |
| |
| Software development is an iterative process
and user participation is recommended along the way. We conducted
a usability test on the Telementoring Workspaces prototype, and received
some useful feedback. |
| |
| 4.1 Icon Testing |
| |
We conducted icon
testing, for the purpose of evaluating the icons we chose to use
on our prototype. We tested a handful of users, and based on their
feedback, we were able to determine how effective our icons were.
We tried to choose icons in our prototype that represented well the
items they corresponded to, so that a new user would be able to figure
out what an item was without too much investigation.
We tested the site logo, the discussion icon, the workspace icon,
the chat icon, the people icon and the artifact icon. Overall, the
site logo was interpreted as something with sharing / collaboration
(good), the discussion icon was interpreted incorrectly (bad), the
workspace icon was interpreted as a place for collaboration (good),
the chat icon was interpreted as a place for dialogue / chat (good),
the people icon was interpreted as people (good), and the artifact
/ file icon was thought of as a document (good). Thus, if we were
to improve our prototype, we could start by changing our discussion
icon. |
| |
| 4.2 Prototype Testing |
| |
|
We tested our prototype by conducting usability
tests with sample users, and had the testers complete a set
of sample tasks. While they were testing, they were encouraged to
think aloud, and prompting questions were asked to help them talk
out their thought process. Afterwards, the users were asked a few
exit interview questions. This gave us some very good feedback on
our prototype, and helped us identify what parts needed changing
or updating. The comments told us that certain parts of our interface
were flawed, and some of the parts we thought were easy to figure
out and use were actually in need of some help, and were not very
easy to use. See appendix for details.
Some very good information was gained by our prototype testing,
and by the discussion area in disCourse. Reviewing the suggestions
and making some changes to a second version of the prototype would
lead us to an even better solution. From here the tests could be
conducted again for additional feedback that may have been missed
during the first set of tests.
|
| |
| 5. Conclusion: Discussion of Methods |
| |
|
The design methods we used worked well for us. First, the Root
Concept turned out to be an essential portion of our project, because
it gave us a starting point and provided a framework for our project.
Looking back, this portion was fundamental to our project.
The interviewing process provided us with great user feedback from
potential stakeholders. However, the interviewing is only as effective
as the quality of the questions being asked, and we found the interview
guide to be of great help here. The interview guide also helped
us stay on track and get all the interviewees view on the various
question topics. Since users don't always know all tools available
or what features they want, it is a good idea to accompany interviews
with an artifact analysis in order to provide another viewpoint
of existing technology, and what its benefits and shortcomings are.
The results from the interviews and artifact analysis turned out
to be extremely helpful in analyzing what to design in a new system.
However, as mentioned earlier both the interview guide and the artifact
analysis have shortcomings, but when done together they are proved
an effective tool in gathering information necessary for defining
system requirements.
The Problem and Activity Scenarios were good tools to show us what
claims need be addressed. The dialogue can be a bit long, but it
is very helpful for communicating problems and solutions between
developers and users since the latter most commonly is not familiar
with more formal modeling tools.
The Claims Analysis summarizes the key points of the scenario,
emphasizing the various tradeoffs. We feel this was a very important
part of our project- because it identified the key problems and
features that made tools successful or unsuccessful, and allowed
us a turning point between analysis and design. The Claims Analysis
allowed us to create good Essential Use Cases, which was key in
determining what our end product would look like.
Our Essential Use Cases turned out to be an 'essential' part of
our project. Since the latter portions of the project were determined
from this design, they ended up being a very useful tool in our
project. The key is to keep them abstract- without making any implementation
decisions- and to keep them 'essential', where each step in the
use case an essential one, for an essential task of the user / system.
We struggled a bit with keeping the Essential Use Cases 'essential',
not just regular use cases. This was probably because the members
of the development team had previous experience with more traditional
use cases. However, once we worked through this, we were left with
something that was very clear in what our system needed to do, and
how (abstractly) it would do it.
The Contexts were a helpful tool in creating our end Prototype,
since we basically took our Essential Use Cases, and turned them
into visual Contexts. This allowed a visualization of what we were
designing, and much of the system's responsibilities actions and
the user responsibilities and actions could be seen visually. We
believe it was important to do this on actual paper- not even in
a drawing tool on a computer, so that one can play around with the
envisioned interface. The best part of Contexts is that implementation
decisions are still avoided, and your design work can be seen visually.
Looking back, this was a great tool for the purpose of our project.
The Context Navigation Map was also a great tool for our project.
It was easy to make after doing the Contexts, and provided a high
level layout of how the Contexts will interact with each other.
If you're making more than one Context it makes sense to draw a
Context Navigation Map.
The Interaction Scenario was used to present our prototype, allowing
us to better communicate our vision of the prototype. Basically,
it wrapped up the stuff that we were planning to implement, but
we needed to cram it into the design before we actually implemented
it. The other tools we used were enough for us to get started on
a prototype, and at this point in our project, time would have been
better spent making several prototypes demonstrating some of the
most central tradeoffs in the system, so we could get the users'
help in making the right decisions.
The Prototype also turned out to be a vital portion of our project-
it was the summation of our analysis and design work. This gave
something hands on that our stakeholders could play with, and it
opened up the door for more user feedback in our testing phase.
Also, it gave something visual and concrete to look at and allowed
for iterative design based on user feedback.
The Usability Testing was probably one of the most important parts
of our project. It allowed some really useful feedback. Piece for
piece it either validated or rejected everything we had done so
far, and allowed for iterative design in the project. It is difficult
to please everyone and create something spectacular on the first
try, and the user feedback we obtained here allows us to go back
and refine the product until it is the best it can be. In retro
perspective, this portion of the project was extremely valuable.
Looking back the methods we used turned out to be very effective
for our purposes. The methods we selected turned out to be effective
tools in creating good telementoring software. If we had to re-do
the project we would have liked to bring a higher number of users
in the development process in order to better eliminate personal
preferences and get more generally representative feedback.
We believe our project has potential to be continued and made into
a functional system that will benefit those in need of a good telementoring
software tool.
|
| |
|