• Human knowledge
  • Conscious knowledge
    • Can be expressed explicitly in words
      • Human expert writes down fuzzy IF-THEN rules
      • Use the rules to build a fuzzy system
  • Subconscious knowledge
    • Human expert knows what to do but unable to express in words or rules
      • Human expert demonstrate system operation
      • Collect input-output data
      • Generate fuzzy IF-THEN rules from the input-output pairs
      • May leave free parameters for optimizing or training the fuzzy system
  • Mixure
    • Human expert writes down fuzzy rules
      • Rules not fully describing the behavior of the expert due to overlooked details
      • Rules not matching the behavior of the expert due to subjective interpretation of the rules
• Designing fuzzy systems from input-output data
  • Learning from examples
  • Gradient descent training
  • Recursive least squares
  • Clustering

Learning from Examples

N-Dimensional Input

$$x = (x_1, ..., x_n)$$

Output

$$y \in V = [\alpha_y, \beta_y]$$

Input-Output Pair

$$x^k = (x_1^k, ..., x_n^k), y^k (k = 1, ..., N)$$

Procedure

1. Define input & output fuzzy sets
   1. Define $$N_i$$ fuzzy sets $$A_i^j (j = 1, ..., N_i)$$ for each input variable $$x_i$$
      1. Complete on $$[\alpha_i, \beta_i]$$
   2. Define $$N_y$$ fuzzy sets $$B^j (j = 1, ..., N_y)$$ for output $$y$$
      1. Complete on $$[\alpha_y, \beta_y]$$
   3. Maximum number of rules: $$\left{\prod^n_{i=1} N_i\right} N_y$$
2. Generate fuzzy rules from input-output data
   1. Identify the fuzzy set $$A^{j_k}_i$$ for input $$x^k_i$$ that maximizes membership function
   2. Identify the fuzzy set $$B^{j_k}$$ for output $$y^k$$ that maximizes membership function
   3. Fuzzy rule
      1. IF $$x_1$$ is $$A_1^{j_k}$$ and ... and $$x_n$$ is $$A_n^{j_k}$$ THEN $$y$$ is $$B^{j_k}$$
   4. Degree of reliability
      1. $$D(k) = \left{\prod^n{i=1}\mu{A_i^{j_k}}(x_i^k)\right} \mu_B^{j_k}(y^k)$$
3. Remove conflicting rules & generate rule base
   1. Choose the maximum $$D(k)$$

Lookup Table

Problems

• For 10 rules
  • 9 are consistent with reasonable reliabilities
  • 1 is inconsistent with the 9, but has higher reliability
    • This one may be generated by bad data, but would be retained

Clustering

• Number of rules in a fuzzy system
  • Too many rules
    • Unnecessary complexity
    • Reduced power of generalization
  • Too few rules
    • Insufficient to achieve the desired objective
• Solutions
  • Fix the number of rules before gradient descent training
  • Fix the input fuzzy sets in the table look-up scheme
  • Set the number of rules a parameter
    • Partition the input-output data into clusters of disjoint subsets
    • Cluster of points represents a single rule
    • Number of rules = number of clusters

Nearest Neighborhood Clustering Procedure

1. Start with the first input-output pair $$(x^1, y^1)$$, establish a cluster center at $$c^1 = x^1$$
   1. Count of number of data points in cluster: $$b^1 = 1$$
   2. Sum of all outputs in cluster: $$a^1 = y^1$$
2. For the k-th pair $$(x^k, y^k)$$, compute the distance of $$x^k$$ to each $$M$$ existing centers of clusters
   1. $$d^k = |x^k - c^{j_k}|$$
   2. If $$d^k \le r$$
      1. Add the point to cluster at $$c^{j_k}$$
      2. $$a^{j_k} += y^k$$
      3. $$b^{j_k} += 1$$
   3. Else
      1. $$M += 1$$
      2. Establish a new cluster at $$c^M = x^k$$
      3. $$a^M = y^k$$
      4. $$b^M = 1$$
3. Repeat until points exhausted
4. Construct fuzzy system based on the singleton fuzzifier, the product inference engine, and center-average defuzzification
   1. $$A^k$$: Gaussian membership function centered at $$c^k$$ with spread $$\sigma$$
      1. $$\sigma$$ chosen to compromise between the matching accuracy & generalizing power of the fuzzy system
   2. $$B^k$$: fuzzy set centered at $$\frac{a^k}{b^k}$$
   3. Rule base of $$M$$ rules
      1. IF $$x$$ is $$A^k$$ THEN $$y$$ is $$B^k$$
   4. Mapping
      1. $$\frac{\sum \frac{a^k}{b^k} e^{-(\frac{x-c^k}{\sigma}})^2}{e^{-(\frac{x-c^k}{\sigma}})^2}$$
      2. If k-th rule given weighting factor of $$b^k$$:
         1. $$\frac{\sum^M{k=1} a^k e^{-(\frac{x-c^k}{\sigma}})^2}{\sum^M{k=1} b^k e^{-(\frac{x-c^k}{\sigma}})^2}$$

Heuristic Reasoning and "Trial & Error"

• Choose state & control variables
  • Define domains & fuzzy sets for them
• Derive fuzzy IF-THEN rules
  • Relates the state & control variables using engineering heuristics e.g. verbalization of human expertise
• Simulate & fine-tune by trial & error