SLS 670 (1): SECOND LANGUAGE QUANTITATIVE RESEARCH
Fall 2008
MW 9:30-10:45
Moore 107
Instructor: Thom Hudson
Office : Moore 401, 956-2795, tdh@hawaii.edu
Office Hours: MW 11:00-12:00 (sign up sheet on door) and by appointment
SLS 670 introduces basic design, measurement, analysis, and inference procedures used in second language quantitative research. The course addresses a range of topics critical to the accurate and ethical use of quantitative methods in L2 studies, including: (a) the purposes and roles of research; (b) literature review and the generation of research questions; (c) study design strategies for gathering, organizing, and analyzing quantitative data; (d) the importance of accuracy (and error) in developing and using measures and other data elicitation procedures; (e) critical reading of research reports; and (f) current concerns with the use of statistical significance testing, the role to be played by power analysis, effect sizes, and confidence intervals, and the nature of 'scientific research'.
Statistical techniques covered in this course include a variety of applications within the following broad categories:
description
comparison of means
comparison of frequencies
correlation and prediction
The course will address statistics from three perspectives:
principles (logic and use), practice (how to calculate and interpret),
and presentation (how to display and report). For each technique, the
use of computerized tools is demonstrated, and students are afforded practice
opportunities through analytic exercises. Time permitting, the course addresses
additional more advanced statistical techniques that students particularly
need to use or wish to know about, such as strategies for dealing with multivariate
analyses.
This syllabus provides a general description of the purpose, methods, content,
and requirements of the course. The specifics may be modified during the
semester in response to class needs.
Goals:
Students will become familiar with:
The purposes and types of second language quantitative research
Key components of research design and study reporting
Statistical inference
Threats to validity of interpretation of statistical studies
Statistical techniques and appropriate uses
Students will be able to:
Conduct effective literature reviews
Develop appropriate research questions and hypotheses
Design studies to answer specific research questions
Employ computer applications for statistical analysis
Display quantitative information
Interpret statistical significance correctly
Interpret effect sizes and the relationship to statistical significance
Review quantitative studies critically<
Requirements:
1. Full attendance and participation
2. Completion of all readings and homework assignments on time
3. Students should actively engage during: (a) lectures and demonstrations; (b) whole-class discussion; (c) small-group discussion.
Grading:
1. Attendance, engagement, and participation 10%
· completion of readings & thoughtful interaction with course content
· attendance and participation in all class sessions, in the form of individual contributions to class and small-group discussions (e.g., commenting, questioning, relating to own experience)
2. Accurate completion of homework assignments 10%
· learning during class demonstrations of statistical applications
· application of learning to practice exercises
3. Completion of major assignments
· Quantitative article critique presentation 10%
· Individual manuscript review 20%
· Research project paper and presentation 50%
Note: Guidelines and criteria for major assignments to be distributed in class. The research paper must be submitted as a word-processed text, spell-checked, double-spaced with minimum of 1-inch margins, and using academic English appropriate to the genre. Deadlines for assignments are set in order to help you organize and plan your time, and to allow me sufficient time for grading and providing feedback. Please contact me individually if you encounter major problems with these expectations.
Readings and resources
Readings: There is a reading packet at Professional Image (973-6599), on King Street across from Puck’s Alley. Also, additional readings and handouts will be provided throughout the semester.
The Readings come from several sources:
1) Draft Chapters of Hudson, T. Research methods in applied linguistics: Design and analysis
2) Selected chapters from: Hatch, E., & Lazaraton, A. (1991). The research manual: Design and statistics for applied linguists. Rowley, MA: Newbury House.
Electronic tools/resources: Much of the work in the course will require hands-on use of computer software. The two primary programs are Microsoft Excel and Statistical Package for the Social Sciences (SPSS). Many of you already have Excel. SPSS is available in the computer labs. However, there are student versions that you might take advantage of. There will be more discussion about options during the course.
Section 1: Purposes, designs, and measurement in L2 quantitative research
What preliminary concerns should be addressed from the outset of research planning?
What roles are played by quantitative research in applied linguistics?
What are the principle considerations in quantitative research design?
Which design options are available and typical for L2 research?
What roles are played by populations, samples, variables, and hypotheses?
What are the major threats or confounds in quantitative research?
How can accuracy be maximized and error minimized in the use of research 'measures'?
How can data representation be maximized in graphic display?
The following is a tentative week by week breakdown of the course schedule. However, there may be changes as the course develops.
Weeks 1-6
| Date |
Content |
Readings |
Assignments |
| August 25 & 27 |
Introduction, research prelims • Syllabus, technology • Ethics, IRB Why L2 quantitative research? • Roles of research in society • Types of research • Quantitative enquiry |
Hudson 1 |
Overview of the course |
| September 1 (Labor Day) 3 & 8 |
Research design basics • Literature review, synthesis • RQs & hypotheses • Variables & constructs |
Hudson 2 SPSS Introduction |
Research project assignments distributed Article critique assignments distributed |
| September 10, 15 & 17 |
Design details • 'Best' quantitative designs • Evaluating pitfalls • Intro to error • Measurement confounds • Complex designs |
Hudson 3 |
|
| September 22 & 24 |
Data collection • Descriptive statistics: review · Overview of concerns in quantitative data collection |
Hudson 4 |
|
| September 29 & October 1 |
Graphic Display of data SPSS Graphics |
Hudson 5 |
Section 2: Analysis and inference in L2 quantitative research
· How are basic quantitative data described, both statistically and visually?
· What is the meaning of statistical significance and how should it be interpreted?
· What are the key threats to validity of interpretations based on inferential statistics?
· Why is there debate surrounding the use of statistical significance testing?
· How are group mean values compared?
· How are frequencies of categorical observations compared?
· How are relationships among variables analyzed?
· How can predictions be made on the basis of multiple variables?
· What other analytic techniques are available for simultaneously comparing multiple variables under study?
Weeks 7-12
| Date |
Content |
Readings |
Assignments |
| October 6 & 8 |
Locating scales Scaling |
Hatch & Lazaraton: Chapter 7 Handouts |
|
| October 13 & 15 |
Comparing groups (means) • Simple designs • t-test • ANOVA |
Hatch & Lazaraton: Chapter 9 |
|
| October 20 & 22 |
Comparing groups (means) • Repeated measures |
Hatch & Lazaraton: Chapter 10 |
|
| October 27 & 29 |
Comparing three or more groups |
Hatch & Lazaraton: Chapter 11 |
|
| November 3 & 5 |
Repeated measures comparison of three or more groups Factorial Designs |
Hatch & Lazaraton: Chapter 12, Chapter 13 |
|
| November 10 & 12 |
Comparing frequencies & ranks Nonparametric approaches |
Hatch & Lazaraton: Chapter 14 |
Section 3: Summing up and wrapping up
· What is research synthesis and meta-analysis?
· What is correlational and regression analysis in SLS?
· What have students found in planning and/or conducting their own research?
· Which quantitative research issues or concerns have students encountered with their research?
Weeks 13-16
| Date |
Content |
Readings |
Assignments |
| November 17, 19 |
Correlation & regression |
Hatch & Lazaraton: Chapter 15 & 16 Urdan: Chapter 12 |
|
| Nov 24 & 26 December 1 & 3 |
Students' research |
Research presentations in class |
|
| December 8 & 10 |
Students' research, course wrap-up |
Research presentations in class Research paper due by 5:00 p.m. December 10 |
Final Project Due December
17, 2008
Some useful basic references for quantitative research methods and data analysis relevant to applied linguistics
Brown, J. D. (1988). Understanding research in second language learning: A teacher's guide to statistics and research design. Cambridge: Cambridge University Press.
Brown, J. D. (1992b). Statistics as a foreign language. Part 2: More things to look for in reading statistical language studies. TESOL Quarterly, 26, 629-664.
Brown, J. D. (2005). Testing in language programs: A comprehensive guide to English language assessment. New York: McGraw-Hill.
Brown, J. D. (2001). Using surveys in language programs. Cambridge: Cambridge University Press.
Brown, J. D., & Rodgers, T. S. (2002). Doing second language research. Oxford: Oxford University Press.
Chapelle, C., & Duff, P. (2003). Some guidelines for conducting quantitative and qualitative research in TESOL. TESOL Quarterly, 37(1), 157-178).
Dunn-Rankin, P. Knezek, G.A., Wallace, S. Zhang, S. (2004). Scaling methods. (2nd ed.) Mahwah, NJ: Lawrence Erlbaum.
Gass, S., Sorace, A., & Selinker, L. (1999). Second language learning data analysis (2"d ed.). Mahwah, NJ: Lawrence Erlbaum.
Hardy, M. & Bryman, A. (2004). Handbook of data analysis. Thousand Oaks: Sage Publications.
Johnson, D. M. (1992). Approaches to research in second language learning. New York: Longman. Mackey, A., & Gass, S. (2005). Second language research: Methodology and design. Mahwah, NJ: Erlbaum.
Mackey, A. & Gass, S. (2005). Second language research: Methodology and design. Mahwah, NJ: Lawrence Erlbaum Associates, Publishers.
Norris, J. M., & Ortega, L. (2003). Defining and measuring SLA. In C. Doughty, & M. H. Long, (Eds.), Handbook of second language acquisition (pp. 716-761). London: Blackwell.
Norris, J. M., & Ortega, L. (Eds.). Synthesizing research on language learning and teaching. Philadelphia: John Benjamins.
Porte, G. K. (2002). Appraising research in second language learning: A practical approach to critical analysis of quantitative research. Philadelphia: John Benjamins.
Scholfield, P. (1995). Quantifying language: A researcher's guide to gathering language data and reducing it to figures. Clevedon: Multilingual Matters.
Seliger, H., & Shohamy, E. (1989). Second language research methods. Oxford: Oxford University Press.
Tufte, Edward R. [1983] (2001). The visual display of quantitative information, 2nd Edition, Cheshire, CT: Graphics Press.
Tufte, Edward R. (1990). Envisioning information . Cheshire, CT: Graphics Press.
Tufte, Edward R. (1997). Visual explanations: Images and quantities, evidence and narrative Cheshire, CT: Graphics Press.
Tufte, Edward R. (2006). Beautiful evidence. Cheshire, CT: Graphics Press.>/p>
Woods, A., Fletcher, P., & Hughes, A. (1986). Statistics in language studies. Cambridge: Cambridge University Press.
Urdan, T. C. (2001). Statistics in plain English. Mahwah, NJ. LEA, Publishers.