Lecture 10: Neural Network

1. Deep learning
2. Representation learning
3. Rule-based
   1. high explainability
4. Linguistic supervision
5. Semi-supervision
   1. have small set of data with label
   2. has large set of data without label
6. Recurrent-level supervision
7. Language structure

description lengths DL= size(lexicon) + size( encoding)

1. lex1
   1. do
   2. the kitty
   3. you
   4. like
   5. see
2. Lex2
   1. do
   2. you
   3. like
   4. see
   5. the
   6. kitty
3. How to evaluate the two lexicons?
   1. lex 1 have 5 words, lex 2 has 6 words
   2. Potential sequence
      1. lex1: 1 3 5 2, 5 2, 1 3 4 2
      2. lex2: 1 3 5 2 6, 5 2 6, 1 3 4 2 6

1. MDL: minimum description lengths
   1. unsupervised
   2. prosodic bootstrapping

Boltzmenn machine

Lexical space

relatedness vs. similarity

• use near neighbors: similarity
• use far neighbors: relatedness

ws-353 has similarity & relatedness

loss function:

 

project:

Part1: potential methods

• LDA
• readability
• syntactic analysis