Learning Outcomes Assessment by The NSF TCUP Planning Team

May 2004

Affiliation

Developed and administered by the National Science Foundation Leadership Team, John Rand, Andrew Pak, Robert Franco, and Judith Kirkpatrick with the collaboration of mathematics and Hawaiian Language and Studies faculty, Kapi`olani Community College. Assessment tool implementation and raw data collected through Herve Collin and Judith Kirkpatrick.Item Calculations, Cross Tabulations, and Tables from Frank Abou-Sayf, Institutional Research Office. Survey conducted Online from February 1-21, 2004

Summary

While we had anecdotal information, we were interested in finding where these students were in their mathematics courses, and wanted a baseline in order to further develop data that would help us track our students' progress to achieve their STEM goals. We developed, designed, and administered an online assessment survey of four levels of mathematics students, Math 25, Math 100, Math 140, and Math 205, (86% of students taking the survey) and various Hawaiian Studies and Language courses (14% of students taking the survey).

Goal of Project

The College received a planning grant from the National Science Foundation in summer 2003, to spend a year assessment and planning an implementation grant that would encourage more students at our College to become Science, Technology, Engineering, or Mathematics (STEM) majors when they transfer to four-year institutions. The grant, in particular, encourages the development of our Hawaiian students to prepare for STEM majors and is designated for tribal colleges that serve Native Hawaiian populations. The planning grant leadership team, John Rand (Physics/Engineering), Judi Kirkpatrick (Technology), and Andy Pak (Mathematics), with years of combined experience, want to begin to solve the problem of students who arrive on campus as STEM majors but quickly, usually within one semester, change majors. The team plans to identify ways to better meet the students' needs and to intervene earlier with support in order to encourage student persistence and resilience in order to achieve their goal of being a (STEM) major.

Description of Project

We gathered information on student demographics, coursework, employment demands, family needs, access to technology, use of college services, majors considered, and family educational background. After the removal of duplicates, we ended up with 502 student data sets to analyze.

Resources Used In Spring 2004 To Support Assessment Activities

The STEM/TCUP leadership team developed the assessment. Each of the team members were given three credits of reassigned time by the College to continue the leadership efforts with the science/mathematics faculty to assess the conditions of our math/science/technology programs at the college. The student assessment gathered key pieces of information that are useful to writing the grant. We used a Kirkpatrick/Collin managed Linux server, Kirkpatrick/Collin collaboration at setting up the survey for input, Pak solicitation of mathematics faculty, Tasaka solicitation of Hawaiian faculty, and all of our expertise in designing the questions and the method of collecting the statistics and cross-tabulation reports.

The STEM/TCUP team also used Herve Collin’s programming and statistical skills to program the survey.

Methods/Instruments

There are several items from the survey that we cross tabulated with ethnic data that identified our part-Hawaiian students. The report for the survey is included at the end of this document. We will use several of the items in our request for the NSF/TCUP implementation grant, due in November 2004.

Results

The current spring 2005 survey can be found at:

http://moosurvey.kcc.hawaii.edu/tcup/survey/

Full data results from the survey of over 500 students are available on the web at:

http://moosurvey.kcc.hawaii.edu/tcup/survey/results.html

A complete report is also included at the end of this document.

Plans

The STEM/TCUP leadership team has held four Saturday planning meetings attended by 17 faculty and other invited guests throughout 2003-04. We disseminated the survey results to the faculty and discussed the implementation grant, naming many of the faculty in the grant. We have been gathering commitments from the faculty planning group, asking for narratives, commitments, and needs for training, materials, and support as we develop our plans, and will keep our database of information, our planning grant resource website, and this report available for the group. We are conducting the survey again in early spring 2005, to further validate our claims and to ensure a baseline is established for the STEM possible majors from our mathematics and science courses. The STEM/TCUP leadership team would like to continue gathering baseline statistics on our potentials STEM majors and from the faculty who will teach the courses that we think will be keys to improving our retention of STEM majors, the resilience of students, and the persistence of students in difficult courses. We would like to continue to refine the student data, the faculty data, and the baseline of possible STEM majors. We would like to assure that the survey has the students Banner ID # to begin tracking students over time as they go through their mathematics sequence to reach science courses. The STEM/TCUP leadership team would like to continue its collaboration on assessment, using a database for the finding and interviewing STEM students, using computer-mediated communication from students and faculty, and establishing a good snapshot of where we currently stand in terms of our potential STEM majors.

LONG RANGE PLAN

An essential ingredient of the STEM/TCUP implementation grant will be a baseline and a means of acquiring and maintaining data on the students we identify as STEM transfer majors. The STEM Assessment is the first step of many that will need refinement, analysis, and development as we track students. We will also need to develop a data set that tracks other aspects of the grant, including faculty development and implementation of new courses.

Contacts

Judith Kirkpatrick, kirkpatr@Hawaii.edu, John Rand, jrand@Hawaii.edu Andy Pak, apak@aloha.com

RESULTS

Mathematics and Hawaiian Language/Studies Students

STEM Assessment

Spring 2004

Developed and administered by John Rand, Andrew Pak, Robert Franco, and Judith Kirkpatrick

with the collaboration of mathematics and Hawaiian Language and Studies faculty

Kapi`olani Community College

Survey conducted Online from February 1-21, 2004

Assessment tool implementation and raw data collected through Herve Collin and Judith Kirkpatrick.

Item Definitions, Calculations, Cross Tabulations, and Tables through Frank Abou-Sayf, Institutional Research Office

Summary

Demographics

• A comparison of the ethnicity of Hawaiian/Part-Hawaiian respondents on the survey was made with the ethnicity they reported in Banner. The results indicate that the Hawaiian/Part-Hawaiian ethnicity reported in Banner may be under-reported by as much as 30 percent. (Item 1)(Cross-Validation 1)
• There were more American Indians among the respondents (5%) than their proportion as reported in Banner (0%) among the general KCC population. (Item 1)
• There were more female than male respondents (59% versus 41%), and more than in the general KCC population. (Item 3)
• Over one-third of the respondents came from five neighboring public high schools. Ten percent came from private high schools. (Item 4)
• Almost 60% of the respondents graduated between 2000 and 2003. (Item 5)

Coursework

• The most commonly taken Math courses in high school were Algebra 1, Algebra 2 and Geometry. The most commonly taken Science courses in high school were Biology and Chemistry. (Item 6a and 6b)
• Almost 86% of the respondents to the survey came from a KCC Math course. (Item 7)
• Math 24, 25, and PCM were the first Math courses at KCC, and English 22 and 100 were the first English courses. No significant differences were detected between Hawaiian/Part-Hawaiian students and the other students. (Item 8a, 8b) (Cross-Tab 1 & 8a; Cross-Tab 1 & 8b)

Employment

• About two-thirds of the students were employed. More than half of them worked between 16 and 30 hours per week. (Item 10)
• Although the employment percentage of Hawaiian/Part-Hawaiian students was not significantly different from that of other students, Hawaiian/Part-Hawaiian students worked a significantly larger number of hours per week. (Cross-Tabulation 1 & 10)

People Under Your Care

• Ninety-three percent of the respondents have either no one (81%) or one person (12%) under their care. No significant differences were detected between Hawaiian/Part-Hawaiian students and the other students. (Item 1) (Cross-Tabulation 1 & 11)

Location of Computer

• The overwhelming primary location of the students’ computer is home, followed by school and friends, in this order. No significant differences were detected between Hawaiian/Part-Hawaiian students and the other students. (Item 12) (Cross-tabulation 1 & 12)

Orientation

• As many students did and did not attend an orientation session. Those who attended a session found it useful by a 2/3 to 1/3 margin. More than half of those who did not attend a session did not know about it. No significant differences were detected between Hawaiian/Part-Hawaiian students and the other students. (Item 13) (Cross-tabulation 1 & 13)

Course Selection Advice

• More than 43% of the all students were self-advised. (Item 14)
• More than 38% of all students obtained advice from counselors. (Item 14)
• More than 84% of the students were satisfied with the advice they received. (Item 14)
• More than 52% of the Part-Hawaiian students were self-advised compared to more than 41% of the other students in choosing courses to sign up for. (Cross-tabulation 1 & 14)

Majors Considered

• The primary, secondary and tertiary majors students were considering, in order of preference, are Health Sciences, Computer Science, Biology, Mathematics, Zoology, Medical Technology. (Item 15)
• Close to fifty percent of all students expressed an interest in a STEM major, (250 out of 502), and forty-three out of 88 Part-Hawaiian students stated an interest in a STEM major. (Cross-tabulation 1&15)

Family-Member Education

• For 56% of the students, at least one family member attend a two-year college, 65% a four-year college, and 52% had at least one family member who graduated with a Bachelor’s degree. (Item 16a)
• Of the 502 students who participated in this survey, 209 had a family member who had a Bachelor’s degree in a STEM major. However, because each respondent could choose more than one answer, it is not possible to estimate the proportion of those family members who graduated with a Bachelor’s degree. (Item 16c)

Item

Statement Summary

Results

(n = 502)

1

Ethnicity

As many as apply.

Ethnicity	Total	Percent	Percent out of 502
African American or Black	4	0.50	0.80
Alaska Native	0	0.00	0.00
American Indian	23	2.89	4.58
Asian Indian	1	0.13	0.20
Chinese	121	15.20	24.10
Filipino	117	14.70	23.31
Japanese	171	21.48	34.06
Korean	45	5.65	8.96
Laotian	7	0.88	1.39
Thai	2	0.25	0.40
Vietnamese	12	1.51	2.39
Other Asian	12	1.51	2.39
Caucasian	145	18.22	28.88
Hispanic	22	2.76	4.38
Part Hawaiian	88	11.06	17.53
Samoan	8	1.01	1.59
Tongan	1	0.13	0.20
Guamanian or Chamorro	3	0.38	0.60
Micronesian (except Guamanian or Chamorro)	7	0.88	1.39
OP Other Pacific Islander	7	0.88	1.39
Total	796	100.00

Remarks

More than one ethnic group could be chosen by the same student. The percent out of 502 represents the total number of respondents.

The College’s ethnic data allows only one choice for students, recorded on the application to the University of Hawai‘i system, but a high percentage of the College’s students come from more than one ethnic. The number of American Indians who responded to this survey, (4.58%) is significantly larger than the number of American Indians recorded in the college population statistics (0%). Other Pacific Islanders combined total is 5.17%.

2

Age

Age	Total	Percent	Percent w/o No Data
Under 18	4	0.80	0.84
18	67	13.35	14.05
19	77	15.34	16.14
20	68	13.55	14.26
21	50	9.96	10.48
22	30	5.98	6.29
25	54	10.76	11.32
30	54	10.76	11.32
31-40	49	9.76	10.27
41-50	19	3.78	3.98
51-60	5	1.00	1.05
No data	25	4.98
Total	502	100.00	100.00

Remarks

The largest proportion of respondents (55.77%) were between 18 and 21 years of age.

3

Gender

Gender	Total	Percent	Percent w/o No Data
Female	274	54.58	58.55
Male	194	38.65	41.45
No data	34	6.77
Total	502	100.00	100.00

Remarks

Error margin = +/- 4.46 %, with a confidence level of 0.95. We are 95% confident that there is a larger proportion of females than males.

4

Last High School attended

Hawaii High School	Total
Kaimuki	43
McKinley	34
Farrington	32
Kalani	31
Roosevelt	30
Kaiser	19
Sacred Hearts	11
Castle	9
Kamehameha	9
St. Louis	9
Waipahu	9
Moanalua	8
Aiea	6
Campbell	6
Kahuku	6
Kailua	6
Damien	5
Kalaheo	5
Konawaena	5
Mid-Pacific	5
Our Redeemer	5
Radford	5
Waimea	5
Maui	4
Punahou	4
University	4
Waianae	4
Others	169
No data	14
Total	502

Remarks

170 respondents (37.64%) were from six Honolulu public high schools, Kaimuki, McKinley, Farrington, Kalani, Roosevelt, Kaiser

52 respondents (10%) were from private Honolulu high schools, Sacred Hearts (11), Kamehameha (9), St. Louis (9) and others.

5

Year of High School graduation

Graduation year	Total	Percent	Percent w/o No Data
Before 1950	1	0.20	0.23
1960-1969	3	0.60	0.68
1970-1979	11	2.19	2.51
1980-1989	37	7.37	8.43
1990	3	0.60	0.68
1991	9	1.79	2.05
1992	4	0.80	0.91
1993	7	1.39	1.59
1994	11	2.19	2.51
1995	20	3.98	4.56
1996	15	2.99	3.42
1997	13	2.59	2.96
1998	15	2.99	3.42
1999	24	4.78	5.47
2000	48	9.56	10.93
2001	59	11.75	13.44
2002	61	12.15	13.90
2003	93	18.53	21.18
GED did not graduate	5	1.00	1.14
No data	63	12.55
Total	502	100.00	100.00

Remarks

Almost 60% of the respondents (59.45%) graduated between 2000 and 2003.

6a

Math courses in High School

H S Math courses	Total	Percent of 502
Pre-Algebra	301	59.96
Algebra I	379	75.50
Algebra II	319	63.55
Algebra III	76	15.14
Geometry	326	64.94
Trigonometry	145	28.88
Calculus	59	11.75
AP Calculus	16	3.19
Advanced Math	13	2.59
Consumer Business Math	19	3.78
Total	1,653

6b

Science courses in High School

H S Science courses	Total	Percent of 502
Biology	411	81.87
Physical Science	262	52.19
Earth Science	111	22.11
Chemistry	347	69.12
Physics	193	38.45
Anatomy and Physiology	62	12.35
Astronomy	19	3.78
Environmental Science	44	8.76
AP Biology	14	2.79
AP Chemistry	16	3.19
AP Physics	6	1.20
Total	1,485

7

Course for which you are taking this survey

Course for this survey	Total	Percent	Percent w/o No Data
Math 25	151	30.08	31.66
Math 100	130	25.90	27.25
Math 140	66	13.15	13.84
Math 205	61	12.15	12.79
Hawaiian 101	16	3.19	3.35
Hawaiian 102	11	2.19	2.31
Hawaiian 107	38	7.57	7.97
Hawaiian 201	2	0.40	0.42
Hawaiian 202	2	0.40	0.42
No data	25	4.98
Total	502	100.00	100.00

Remarks

Students from Math courses accounted for 85.54% of all respondents to this survey.

8a

First Math course at KCC

First Math Course	Total	Percent	Percent w/o No Data
PCM 23	77	15.34	16.31
Math 24	111	22.11	23.52
Math 25	95	18.92	20.13
Math 27	4	0.80	0.85
Math 97	4	0.80	0.85
Math 100	38	7.57	8.05
Math 103	21	4.18	4.45
Math 135	10	1.99	2.12
Math 140	46	9.16	9.75
Math 203	2	0.40	0.42
Math 205	33	6.57	6.99
Math 231	2	0.40	0.42
None	29	5.78	6.14
No data	30	5.98
Total	502	100.00	100.00

Remarks

Math 24, Math 25 and PCM were the first courses that were taken by most respondents (59.96%).

8b

First English course at KCC

First English Course	Total	Percent	Percent w/o No Data
PCC 20	23	4.58	4.93
English 21	47	9.36	10.06
English 22	145	28.88	31.05
English 100	195	38.84	41.76
None	57	11.35	12.21
No data	35	6.97
Grand Total	502	100.00	100.00

Remark

English 100 and English 22 are the first English courses that were taken by most respondents (72.81%).

9a

All Math classes taken at KCC

KCC Math Classes	Total	Percent out of 502
PCM 23	75	14.94
Math 24	168	33.47
Math 25	177	35.26
Math 27	5	1.00
Math 97	21	4.18
Math 100	68	13.55
Math 103	59	11.75
Math 115	5	1.00
Math 135	52	10.36
Math 140	57	11.35
Math 203	0	0.00
Math 205	35	6.97
Math 206	1	0.20
Math 231	0	0.00
None	81	16.14
Total	804	160.16

Remarks

Since more than one class could be checked out by the same student, the “Percent” column is based on 502, the total number of respondents.

9b

All English classes taken at KCC

KCC English Classes	Total	Percent out of 502
PCC20	18	3.59
English 21	64	12.75
English 22	171	34.06
English 100	270	53.78
None	112	22.31
Total	635	126.49

Remarks

Since more than one class could be checked out by the same student, the “Percent” column is based on 502, the total number of respondents.

English 100 is the English course that was taken by most respondents (53.78%).

10

Employment status

Currently employed	Total	Percent	Percent w/o No Data
No	158	31.47	33.47
Yes	314	62.55	66.53
No data	30	5.98
Total	502	100.00	100.00

Error margin = +/- 4.26 %, with a confidence level of 0.95. We are 95% confident that there is a larger proportion of students currently employed than not employed.

If yes, hours per week

If yes, hours per week	Total	Percent	Percent w/o No Data
1-5 hours per week	15	2.99	3.79
11-15 hours per week	30	5.98	7.58
16-20 hours per week	58	11.55	14.65
21-25 hours per week	65	12.95	16.41
26-30 hours per week	40	7.97	10.10
31-35 hours per week	26	5.18	6.57
36-40 hours per week	29	5.78	7.32
40+ hours per week	30	5.98	7.58
6-10 hours per week	21	4.18	5.30
Not applicable	82	16.33	20.71
No data	106	21.12
Total	502	100.00	100.00

Remark

More than half the respondents (51.91%) who work spend between 16 and 30 hours per week at work.

11

People under your care

# people under yr care	Total	Percent	Percent w/o No Data
1	52	10.36	11.61
2	18	3.59	4.02
3	7	1.39	1.56
4	3	0.60	0.67
10+	5	1.00	1.12
Not applicable	363	72.31	81.03
No data	54	10.76
Total	502	100.00	100.00

Remark

A majority of respondents (81.03%) have no one under their care. A minority of students, (19%) have one or more persons under their care.

12

Computer location - primary

Where is comp you use?	Primary	Percent	Percent w/o No Data
Home	386	76.89	82.13
School	52	10.36	11.06
Friends	11	2.19	2.34
Other	8	1.59	1.70
Workplace	6	1.20	1.28
Library	5	1.00	1.06
Don't use computer	2	0.40	0.43
No data	32	6.37
Total	502	100.00	100.00

Remark

The overwhelming primary location of the respondents’ computer is home (82.13%).

Computer location - secondary

Where is comp you use?	Secondary	Percent	Percent w/o No Data
Home	60	11.95	14.42
School	228	45.42	54.81
Friends	47	9.36	11.30
Other	15	2.99	3.61
Workplace	36	7.17	8.65
Library	26	5.18	6.25
Don't use computer	4	0.80	0.96
No data	86	17.13
Total	502	100.00	100.00

Remark

The major secondary location of the respondents’ computer is school (54.81%).

Computer location

- tertiary

Where is comp you use?	Tertiary	Percent	Percent w/o No Data
Home	20	3.98	6.15
School	62	12.35	19.08
Friends	79	15.74	24.31
Other	50	9.96	15.38
Workplace	30	5.98	9.23
Library	68	13.55	20.92
Don't use computer	16	3.19	4.92
No data	177	35.26
Total	502	100.00	100.00

Remark

The tertiary most-frequent location of the respondents’ computer is friends (24.31%)

13

Attend an orientation session before registering

Attend		Total		Percent		Percent w/o No Data		Percent of Yes/No only
	No		235		46.81		49.68		52.34
	Yes		214		42.63		45.24		47.66
	Don't remember		13		2.59		2.75
	Not applicable		11		2.19		2.33
	No data		29		5.78
	Total		502		100.00		100.00		100.00

Remarks

Error margin = +/- 4.55 %, with a confidence level of 0.95. The difference in observed proportions of students who attended an orientation session and those who did not is within the margin of error.

If yes, was it useful?

Useful	Count	Percent of Yes/No only
No	64.00	31.37
Yes	140.00	68.63
Not applicable	9.00
No data	1.00
Total	214.00	100.00

Remarks

Error margin = +/- 6.37 %, with a confidence level of 0.95. We are 95% confident that, among those students who attended an orientation session, a larger proportion of students found the session useful than not useful.

If not, why not?

Why not	Total	Percent	Percent w/o No Data	Percent w/o No data & N.A.
Didn't know about it	109	21.71	29.95	50.70
Didn't need help	49	9.76	12.47	22.79
Schedule conflicts	39	7.77	4.61	18.14
Forgot	18	3.59	2.23	8.37
Not applicable	149	29.68	18.88
No data	138	27.49
Total	502	100.00	68.14	100.00

Remarks

More than half (50.70%) of those students who did not attend an orientation session did not know about it.

14

Source of advice for course sign-up

Source	Total	Percent	Percent w/o No Data
Self-advised	205	40.84	43.43
KCC counselor	181	36.06	38.35
Friends	25	4.98	5.30
Family members	19	3.78	4.03
Other students	17	3.39	3.60
Teachers	15	2.99	3.18
Other adults	10	1.99	2.12
No data	30	5.98
Grand Total	502	100.00	100.00

Remark

More than 43% of the students were self-advised

More than 38% received advice from a counselor.

Satisfaction with advice

Satisfied	Total	Percent	Percent w/o No Data
No	71	14.14	15.50
Yes	387	77.09	84.50
No data	44