Connectionist Language Modeling

Linguistics 431/631, Fall 2006

Moore Hall 228 and 153b (labs)

Ben Bergen

 

Overview

 

To model cognitive functions like language, perception, and memory, many scientists are turning to computational models that are based on the human brain. These connectionist models are gaining in popularity in Linguistics, Psychology, and Computer Science, as well as in other disciplines.

 

This course is an introduction to the connectionist modeling of language. The approach is extremely hands-on. Students will learn to use existing connectionist simulation programs to design models of human linguistic knowledge and functioning. The course is intended for students with little to no computer knowledge. NO PROGRAMMING EXPERIENCE IS REQUIRED.

 

This course is open to all advanced undergraduate students and all graduate students, in any discipline.

 

Coursework and grading

 

Most weeks consist of one regular class meeting and one computer lab meeting.

• The regular meetings discuss the theories behind and workings of connectionist models.
• In the computer lab meetings, students each work on computers to complete exercises pertaining to the week’s topic. Students are given assistance from the instructor and are also encouraged to work collaboratively with other students.
• Students are also given access to computers that can be used to work on exercises and on term projects outside of class hours.

 

The coursework is composed of computer lab exercises and a term project.

• Students are expected to attend all class meetings; participation is  10% of the course grade.
• Students are expected to complete all exercises, so that they can compile them into their own self-created manual on connectionist modeling, which they can use later on in their career. These exercises constitute 45% of the course grade.
• Each student picks a topic of interest and designs and runs a connectionist simulation relating to it for their term project. For example, a student might decide to design a connectionist model of the acquisition of phonological patterns in L1 and L2 speakers, or one of semantic categorization. The term project makes up 45% of the course grade.
• Students also present their term project work to the class at the end of the semester.

 

Logistics

 

Please take full advantage of my office hours, currently scheduled for Monday 10:30-11:30 and Wednesday 1:30-2:30. If you cannot make these times, please let me know and we can schedule another time to meet. I am also available by email at: bergen@hawaii.edu. Lecture notes, an up-to-date course schedule, and links to online versions of course readings will appear through the semester at: http://www2.hawaii.edu/~bergen/ling631/

Schedule

 

 

Readings

 

[Some papers require a login and password.]

 

R1       Overview of connectionism from Wikipedia: http://en.wikipedia.org/wiki/Connectionism

 

R2       Pages 1-10 of Kim Plunkett and Jeffrey Elman. 1996. Exercises in rethinking innateness. Cambridge, MA: MIT Press. http://www2.hawaii.edu/~bergen/ling631/papers/eri.pdf

 

R3       Get JavaNNS for your system: http://www-ra.informatik.uni-tuebingen.de/downloads/JavaNNS/

            The manual is included. Read sections 1-6

 

R4       Review section 6 of the JavaNNS manual

 

R5       Pages 10-18 of Kim Plunkett and Jeffrey Elman. 1996. Exercises in rethinking innateness. Cambridge, MA: MIT Press. http://www2.hawaii.edu/~bergen/ling631/papers/eri.pdf

 

R6       Sections 7 and 8 of the JavaNNS manual

 

R7       Rumelhart, D. and McClelland, J. (1986). On learning the past tenses of English verbs. In J. McClelland, D. Rumelhart, and the PDP Research Group (Eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition, volume 2. pp. 216-271.
http://www.cnbc.cmu.edu/~jlm/papers/PDP/Chapter18.pdf

 

R8       Pinker, S. and A. Prince (1988). On Language and Connectionism: Analysis of a Parallel Distributed Processing Model of Language Acquisition. Cognition, 28:73-- 193. http://www.ecs.soton.ac.uk/~harnad/Papers/Py104/pinker.conn.html

 

R9       MacWhinney, B., & Leinbach, J. (1991) Implementations are not conceptualizations: Revising the Verb Learning Model. Cognition, 29, 121-157. http://psyling.psy.cmu.edu/papers/neuralnets/cog91.pdf

 

R10     Dell, G., F. Chang, and Z. Griffin (1999). Connectionist Models of Language Production: Lexical Access and Grammatical Encoding. Cognitive Science 23 (4): 517-542. http://www2.hawaii.edu/~bergen/ling631/papers/delletal.pdf

 

R11     McClelland, J. L. and Elman, J. L. (1986). The TRACE Model of Speech Perception. Cognitive Psychology, 18, 1-86.
http://www.cnbc.cmu.edu/~jlm/papers/McClellandElman86.pdf

 

R12     Gaskell, M. and W. Marslen-Wilson (1999). Ambiguity, competition, and blending in spoken word recognition. Cognitive Science 23:4: 439-462 http://www2.hawaii.edu/~bergen/ling631/papers/gaskellmarslen.pdf

 

R13     Elman, J. L. (1991). Distributed representation, simple recurrent networks, and grammatical structure. Machine Learning, 7, 195–225. ftp://ftp.crl.ucsd.edu/pub/neuralnets/machine_learning.pdf

 

R14     Frank, R., D. Mathis and W. Badecker. 2004. The Acquisition of Anaphora by Simple Recurrent Networks. Ms.
http://www.cog.jhu.edu/faculty/frank/papers/SRN-note.pdf

 

R15     http://vv.carleton.ca/~neil/neural/neuron-a.html

http://faculty.washington.edu/chudler/cells.html

http://users.rcn.com/jkimball.ma.ultranet/BiologyPages/N/Neurons.html

 

R16     Gurney, K. (m.s.) Notes on An Introduction to Neural Networks  http://www.shef.ac.uk/psychology/gurney/notes/l1/l1.html