From Learning to Machine Learning

# Learning: acquiring skills
observations -> learning    -> skill

# Machine Learning: acquiring skills
data         -> ML          -> skill
                               (improved performance measure)

• Some use scenarios
  • Cannot program the system manually
  • Cannot define the solution easily
  • Need rapid decisions
  • Need user-oriented in a massive scale
• Key essence
  • Exists some underlying pattern to be learned
  • But no programmable definition
  • There are data

Formalize the Learning Problem

• input: x ∈ X
• output: y ∈ Y
• target function: f: X -> Y
  • Unknown pattern to be learned
• data - training examples: D = {(x1, y1), (x2, y2), ...}
• Learning algorithm: A
• hypothesis - skill: g: X -> Y; g ∈ H = {hk}
  • Learned formula to be used
  • g hopefully ≈ f
  • H is a hypothesis set, A will pick the best h as g
• Learning model = A + H

    f
    |
{(xn, yn)} -> ML -> g

Machine Learning v.s. Data Mining

• Machine learning
  • Use data to compute hypothesisg that approximates target f
• Data mining
  • Use (huge) data to find property that is interesting
  • If interesting property same as hypothesis that approximates target
    • ML = DM
  • If interesting property related to hypothesis that approximates target
    • DM helps ML and vice versa
  • Traditional DM also focuses on efficient computation in large database
• Artificial Intelligence
  • Compute something that shows intelligent behavior
  • g ≈ f is something that shows intelligent behavior
    • ML realizes AI
• Statistics
  • Use data to make inference about an unknown process
  • g is an inference outcome, f is something unknown
    • Statistics is used to achieve ML
  • Traditional statistics also focuses on provable results with math assumptions and cares less about computation