Linguistics 431/631: Connectionist language modeling

Ben Bergen


Meeting 7: More complex morphology

October 3, 2006


MacWhinney & Leinbach


MacWhinney & Leinbach respond to all of Pinker and Prince's criticisms in a new connectionist model of the English past tense

      Phonological coding, e.g. bet

      6949 verb forms, presented according to their frequency in the Brown corpus



      After 24,000 epochs, produced past tenses with 97% accuracy, and resolved many of P&Ps concerns with R&Ms model

      But still had some problems

o      Did not memorize frequent irregulars early

o      Could not deal with homophones

o      Didnt do well with low frequency irregular compounds (like underwent)

o      Didnt do well with derived verbs, like flied

      The overall problem the model (like R&M) has no representation for lexical items




In other work, MacWhinney has described another model, ConComp, which is a connectionist model of determiners and nouns (in German) that includes lexical representations

      There are a number of structured connectionist models in which lexical items are represented

      ConComp has lexical representations be learned

      The task is learning to produce the right forms of German nouns and their determiners

o      Determiners

o      Nouns



      The model has three major parts

o      CatNet learns lexical categories, which are associations between particular articles and particular cases and numbers

o      ArtNet selects an appropriate article for a word with a particular phonology, case, number, and lexical category

o      StemNet inflects the noun