Linguistics 431/631: Connectionist language modeling

Ben Bergen

Meeting 1: Introduction

August 22, 2006

Introduction to connectionism

Connectionist models are models whose structure and functioning is based on properties of neural systems, i.e. the brain (see more at: http://www.aaai.org/AITopics/html/neural.html).

• Simple computational units (nodes or neurodes)
• Levels of activation for nodes.
• Numerous connections between nodes
• Different strengths of those connections
• A learning algorithm where strengths of connections are modified on the basis of data.
• Massively parallel architecture

Some properties of connectionism

• Rules are learned from data, not defined by an expert
• Many connectionist models are sub-symbolic – categories are emergent from data
• Connectionist models display graded responses
• They are often implemented in computers, which allows them to be tested in detail
• Many connectionist models degrade gracefully

Technological applications (see more at ftp://ftp.sas.com/pub/neural/FAQ7.html#A_applications)

• Speech recognition
• Speech synthesis
• Sonar mine detection
• Grading of produce
• Automobile steering
• Face recognition
• Chess
• and many others

This class focuses on uses of connectionism in the study of language

• Learning and use of phone and phoneme recognition and production
• Learning and use of word recognition and production
• Learning and use of morphological generalizations
• Learning and use of parts of speech and syntactic rules
• Use of semantic knowledge