Telementoring Workspace
Networked Education for Remote Destinations

Project Report
(MS Word version)
1. Introduction
 

Telementoring is a mentoring relationship or program in which the communication between mentor and mentee is computer mediated, where mentoring is defined as a sustained relationship between a youth and an adult. In a well-structured mentoring relationship, the adult provides help, support and guidance to the mentee.
The idea of telementoring is to use telecommunications technology to support mentoring relationships between students and volunteering field experts. This is particularly useful in situations where face-to-face mentoring would be hard to carry out. We predict that telementoring, both within traditional education and the business world, will be a growing area in the future. It will connect mentees with field experts in a worldwide context. As there currently does not exist any customized software to support telementoring we believe this is a great opportunity to get a competitive advantage and lead the way for telementoring software. We will focus on the development of a new software tool for the use of educational telementoring.

Typically students of lower grade levels have teachers with knowledge of general topics in education. Therefore when students are working on projects or field studies that require expert knowledge of a particular field, their teachers cannot always provide them with the assistance they need. In these cases students are often left to seek additional knowledge via sources like the school library and the Internet, where the abundance, or possibly lack of, information can make it difficult and time consuming for the student to find what is relevant for their particular project. Scientists and other field experts may not be available in the local community, and students might not know where and how to get in touch with authorities within their field of interest. Currently the communication between students, teachers, and field experts in a telementoring relationship is primarily done by e-mail or in discussion groups. Our proposal is to make software specifically for telementoring, thus considering the needs of the various user groups in a telementoring relationship throughout the design process. The software will provide a workspace where this interaction between telementor and mentees can take place. Early attention was given to usability, since a key advantage over other existing tools will be that our tool is simpler for mentors and mentees to use.

The methods used throughout this project includes root concept, interview guide, user role modeling, artifact analysis, problem and activity scenarios accompanied by claims analysis, essential use cases, entity-relationship modeling, contexts, prototyping, and usability testing. We believe that prioritizing spending time on these methods allowed us to design a system that has clear advantages over existing systems. With a growing need for telementoring software due to the shortcomings of existing tools, we believe moving on this now will give us the opportunity to gain a key market advantage being the first to create this kind of tool. We have the opportunity to be the first and set the standard, as well as be the only offering such a tool

 
2. Requirements
 

We started out defining the root concepts of our project in order for us as a development team to have a common understanding of what we were trying to achieve. Then we continued to define the requirements for Telementoring Workspace software as we conducted interviews with representatives for the core stakeholders in our project. Based on the results of these interviews we were able to conduct a requirements analysis for the system.

We prepared a user role model document that describes the roles, backgrounds, expectations, profile, objectives and knowledge of the various stakeholders. We also prepared an artifact analysis that surveys some existing system offering features useful for telementoring. In addition to this we prepared three problem scenarios, three activity scenarios, and a derived claims analysis, which served to find problems with the existing technology and propose solutions to these.

 
2.1 Root Concept
 
The first step we took in our project was to write a Root Concept. This defined what the goals and aim of our project are. We defined our vision, rationale and our stakeholders for the project. Also, we defined our starting assumptions and constraints for the project. This part of the project gave us a starting point for the project- our starting point, what we envisioned ourselves accomplishing, why it was necessary, and who was involved.
 
2.2 Interview
 

We wanted to interview users to gain knowledge of some of the limiting factors that we would have for our project. For example, we wanted to know what computer equipment our users had, what kind of Internet connection they had, and how much access they had to the equipment. This allowed us to get a feel for what their needs were. We also wanted to get an understanding for their knowledge background related to telementoring. We were able to find out if our sample users were familiar with telementoring (after explaining the definition), and see how they currently interact with each other for similar types of projects.

Interviewing also helped us to determine what type of information is exchanged in related activities so we can design a system that supports this type of information exchange in a more efficient manner than the existing tools. This lead us to ask our stakeholders what features on current systems were likeable or dislikeable, so we could design a system that had advantages over the existing technology.
In preparation for the interviews we made an interview guide which took into consideration the various stakeholders recognized in our root concept.

Because of the short time available for this project we decided to limit the interviews to a couple of representatives for each of our three main stakeholders; mentors, mentees, and the mentees supervisors. If there had been more time available for this project we would have liked to do more interviews in order to find general trends rather than individual preferences. As representatives for Mentors we interviewed two graduate students, as Mentees we interviewed a high school student and an undergraduate student, and two K-12 teachers and a university instructor represented the supervisor stakeholder group.
Before each interview we explained the term "Telementoring" to ensure a common understanding of what it is.
We found that half of the interview subjects have experience with telementoring / mentoring using Telephone, e-mail, and/or chat room. Most expect telementoring to be some sort of relationship between a student who could use help from an expert, and the expert with knowledge in a particular area, and they would all like a tool that is easy to use, and some expect some sort of a collaboration tool.

Most of our subjects expect a project to last from half a semester to a semester in length, with some sort of presentation at the end. Collaboration takes place throughout. All expect the outcome of the project to be students having gained knowledge on a particular subject, while utilizing tools such as mentors to aid them in their work.
All our interview subjects have access to newer computer equipment with a fast Internet connection whenever they wish. Some are experts, the rest have a moderate amount of computer expertise mostly using text editing software, browsing the Internet, and playing computer games. The subjects had varying likes and dislikes when it comes to software and features, but they all agreed that being easy to use is an important trait.
Most of our subjects liked meeting in person, and disliked using chat rooms because they weren't private. All are familiar and like to use e-mail, and all but one are familiar with discussions, and think they have advantages / disadvantages, such as anonymity. All would appreciate a mixture of chat and discussions to accommodate the most number of people. Most would like to know whom they are working with, some personal information, and some credentials. If participating in a telementoring they would like to know some personal information, then some information on the details of the project. As a mentor they would like to know about the relationship between stakeholders, and the mentee's general knowledge level, so information can be presented for their level, and they would expect to spend around two hours per week telementoring.

 
2.3 Artifact Analysis
 
Because of the lack of existing telementoring software we did an artifact analysis of systems containing aspects and features useful for mentors, mentees, and supervisors that are looking for ways to communicate with each other. We found that the various tools are mostly poorly integrated. The free tools has a bonus for those with a low budget, but they all contain components which require downloading and installing software, which requires memory space and may for some users also require retrieving permission and assistance from a system administrator. Any of the three free suites analysed provide a bit of simplicity in the fact that you only need one login / password to access any of the services. But they all suffer from "sneaking featurism" because of their commercial nature and the expectations of continuously upgrading their products by adding more features. WebCT also has bundled features that are available under the same login and password.
We definitely see the need for a fully integrated system tailored for telementoring, offering artifact sharing and asynchronous and synchronous communication that can be archived in order to preserve useful information for the benefit of others. The system should be accessible to the users via an Internet browser, and not require installing additional software. The system needs to be easy to use and easy to navigate in order to retrieve information, and should be stripped for unnecessary features and functionalities that may confuse and distract the users.
 
2.4 Problem/ Activity Scenarios
 
Based on the findings in the interviews we wrote three problem scenarios (1 ,2 ,3)describing problems encountered by users when using software that is designed for other purposes than telementoring. Writing these scenarios was an iterative process as we consulted users in order to make them as representative as possible.
As a response to the problem scenarios we then wrote three activity scenarios illustrating how we proposed the Telementoring Workspace software to provide improved support for telementoring. These scenarios were presented to some users in order to get their opinion about the solutions they proposed
 
2.5 Claims Analysis
 
Both problem- and activity scenarios were accompanied with claims analysis emphasizing the positive and negative aspects of the features presented in them. The claims analysis made it clearer what tradeoffs were connected to the various features present in the scenarios.
 
2.6 User Role Models
 
For our User Role Model forms, we put our stakeholders into three categories: mentor, mentee, and supervisor. First, we viewed our mentor as the 'outside expert' who voluntarily was brought into a project to assist someone (the mentee) who is in need of expert help or guidance. Second, we thought of the mentees as those who would be benefiting from outside expertise- those with the projects that they would need expert assistance with. Typically the mentees would fall into the category of being in school. Our third stakeholder was the supervisor. We envisioned the supervisor role as anyone who was overseeing the project. In the case where the mentee is a student, the supervisor would probably be the student's teacher / professor. The supervisor could oversee multiple projects, and interact in the communication if they chose to do so.
Overall, the User Role Model forms helped us categorize each of our three users, and see more clearly the properties of the stakeholder that we are designing our system for.
 
3. Design and Prototype
 
For the design phase we prepared some essential use cases, which served to isolate the design issues we determined in our activity scenarios. Further we created an entity-relationship diagram showing the components of the new system and how they inter-relate with each other. Based on these documents, along with the Activity Scenarios, we made some Contexts. The relations and transitions between the various the contexts were shown in a Navigation map.
After revisiting documentation from our requirements analysis and design process with representatives from our user group we developed an interaction scenario with a related prototype describing a new user's meeting with the Telementoring Workspace. In addition we have a second view of the same prototype showing an experienced user logged in at TW. Finally we did some usability testing of aspects of our prototype. The test results emphasized strength and weaknesses in the Telementoring Workspace software, and gave us a better impression what aspects of the system need to be improved.
The results of these methods can be seen in the appendix.
 
3.1 Essential Use Cases
 
We developed Essential Use Cases for the purpose of re-designing some of the issues we came up with in our Problem and Activity Scenarios, and the Claims Analysis. The Essential Use Cases show the envisioned user action and system response required for the system's 'essential' tasks. However, they are still very abstract, so that implementation decisions are not made. Our Essential Use Cases show the main features of our proposed system, such as using the chat, discussion, searching for other discussions, adding a friend to your 'friend list', uploading a file, creating new workspaces, and viewing recent activity. These Essential Use Cases provide us with an abstract picture of how our envisioned system will work. A lot of our design work was done on this part of the process, and takes one step closer to creating a prototype.
 
3.2 E-R Model
 
We created an entity-relationship model showing the main components of the new system and how they inter-relate with each other. This was used as a communicative tool for the designers to have a common understanding of what the main elements in the system would be and how they would interact.
 
3.3 Context Navigation Map
 
The Context Navigation Map provides navigation between the Contexts we created. On our individual Contexts, the dark purple bars designate links to external Contexts. All of these links can be seen together in the Context Navigation Map, and provide a good high-level view of the system and how one can move through it. For a description of the Contexts, see below.
 
3.4 Contexts
 
Our Contexts were based on our Essential Use Cases, and are based on the nouns and verbs that were in the Essential Use Cases. The nouns turned into the items in the Contexts, and the verbs were the actions that one can perform. Our Contexts were drawn with four colors. Dark yellow designates the Context, light yellow designates a content display area, dark purple designates a command tool that takes you to another Context (see Context Navigation Map), and light purple designates a command tool that carries out actions within the current Context.
The Contexts took the design we came up with in our Essential Use Cases, and translated them into something visual that was very helpful to us in making a prototype.
 
3.5 Interaction Scenario
 
Our Interaction Scenario describes some features that we felt were necessary to include in the Prototype that were not covered by the Essential Use Cases. Basically, they covered a user using particular features of the envisioned system. In our case, it tied in very closely to our prototype, and provided a glimpse of a hypothetical user using our system. We were able to link this to the prototype we made, so that when reading one can go back and forth between the two to clarify what is being read.
 
3.6 Prototype
 
The prototype is an implementation of the system we designed in the steps above. Being an initial prototype, we expected some problems with the interface, which will hopefully be discovered later on in our usability testing. We conducted icon testing to see the effectiveness of our chosen icons, and followed with prototype testing for feedback on the complete prototype / implementation.
 
4. Evaluation and Redesign
 
Software development is an iterative process and user participation is recommended along the way. We conducted a usability test on the Telementoring Workspaces prototype, and received some useful feedback.
 
4.1 Icon Testing
 
We conducted icon testing, for the purpose of evaluating the icons we chose to use on our prototype. We tested a handful of users, and based on their feedback, we were able to determine how effective our icons were. We tried to choose icons in our prototype that represented well the items they corresponded to, so that a new user would be able to figure out what an item was without too much investigation.
We tested the site logo, the discussion icon, the workspace icon, the chat icon, the people icon and the artifact icon. Overall, the site logo was interpreted as something with sharing / collaboration (good), the discussion icon was interpreted incorrectly (bad), the workspace icon was interpreted as a place for collaboration (good), the chat icon was interpreted as a place for dialogue / chat (good), the people icon was interpreted as people (good), and the artifact / file icon was thought of as a document (good). Thus, if we were to improve our prototype, we could start by changing our discussion icon.
 
4.2 Prototype Testing
 

We tested our prototype by conducting usability tests with sample users, and had the testers complete a set of sample tasks. While they were testing, they were encouraged to think aloud, and prompting questions were asked to help them talk out their thought process. Afterwards, the users were asked a few exit interview questions. This gave us some very good feedback on our prototype, and helped us identify what parts needed changing or updating. The comments told us that certain parts of our interface were flawed, and some of the parts we thought were easy to figure out and use were actually in need of some help, and were not very easy to use. See appendix for details.

Some very good information was gained by our prototype testing, and by the discussion area in disCourse. Reviewing the suggestions and making some changes to a second version of the prototype would lead us to an even better solution. From here the tests could be conducted again for additional feedback that may have been missed during the first set of tests.

 
5. Conclusion: Discussion of Methods
 

The design methods we used worked well for us. First, the Root Concept turned out to be an essential portion of our project, because it gave us a starting point and provided a framework for our project. Looking back, this portion was fundamental to our project.
The interviewing process provided us with great user feedback from potential stakeholders. However, the interviewing is only as effective as the quality of the questions being asked, and we found the interview guide to be of great help here. The interview guide also helped us stay on track and get all the interviewees view on the various question topics. Since users don't always know all tools available or what features they want, it is a good idea to accompany interviews with an artifact analysis in order to provide another viewpoint of existing technology, and what its benefits and shortcomings are.
The results from the interviews and artifact analysis turned out to be extremely helpful in analyzing what to design in a new system. However, as mentioned earlier both the interview guide and the artifact analysis have shortcomings, but when done together they are proved an effective tool in gathering information necessary for defining system requirements.
The Problem and Activity Scenarios were good tools to show us what claims need be addressed. The dialogue can be a bit long, but it is very helpful for communicating problems and solutions between developers and users since the latter most commonly is not familiar with more formal modeling tools.

The Claims Analysis summarizes the key points of the scenario, emphasizing the various tradeoffs. We feel this was a very important part of our project- because it identified the key problems and features that made tools successful or unsuccessful, and allowed us a turning point between analysis and design. The Claims Analysis allowed us to create good Essential Use Cases, which was key in determining what our end product would look like.
Our Essential Use Cases turned out to be an 'essential' part of our project. Since the latter portions of the project were determined from this design, they ended up being a very useful tool in our project. The key is to keep them abstract- without making any implementation decisions- and to keep them 'essential', where each step in the use case an essential one, for an essential task of the user / system. We struggled a bit with keeping the Essential Use Cases 'essential', not just regular use cases. This was probably because the members of the development team had previous experience with more traditional use cases. However, once we worked through this, we were left with something that was very clear in what our system needed to do, and how (abstractly) it would do it.

The Contexts were a helpful tool in creating our end Prototype, since we basically took our Essential Use Cases, and turned them into visual Contexts. This allowed a visualization of what we were designing, and much of the system's responsibilities actions and the user responsibilities and actions could be seen visually. We believe it was important to do this on actual paper- not even in a drawing tool on a computer, so that one can play around with the envisioned interface. The best part of Contexts is that implementation decisions are still avoided, and your design work can be seen visually. Looking back, this was a great tool for the purpose of our project.
The Context Navigation Map was also a great tool for our project. It was easy to make after doing the Contexts, and provided a high level layout of how the Contexts will interact with each other. If you're making more than one Context it makes sense to draw a Context Navigation Map.
The Interaction Scenario was used to present our prototype, allowing us to better communicate our vision of the prototype. Basically, it wrapped up the stuff that we were planning to implement, but we needed to cram it into the design before we actually implemented it. The other tools we used were enough for us to get started on a prototype, and at this point in our project, time would have been better spent making several prototypes demonstrating some of the most central tradeoffs in the system, so we could get the users' help in making the right decisions.
The Prototype also turned out to be a vital portion of our project- it was the summation of our analysis and design work. This gave something hands on that our stakeholders could play with, and it opened up the door for more user feedback in our testing phase. Also, it gave something visual and concrete to look at and allowed for iterative design based on user feedback.

The Usability Testing was probably one of the most important parts of our project. It allowed some really useful feedback. Piece for piece it either validated or rejected everything we had done so far, and allowed for iterative design in the project. It is difficult to please everyone and create something spectacular on the first try, and the user feedback we obtained here allows us to go back and refine the product until it is the best it can be. In retro perspective, this portion of the project was extremely valuable.

Looking back the methods we used turned out to be very effective for our purposes. The methods we selected turned out to be effective tools in creating good telementoring software. If we had to re-do the project we would have liked to bring a higher number of users in the development process in order to better eliminate personal preferences and get more generally representative feedback.
We believe our project has potential to be continued and made into a functional system that will benefit those in need of a good telementoring software tool.

 

This page was last updated 15 December, 2002