Classification and Prediction

• LDA - Linear Discriminant Analysis
• KNN - K Nearest Neighbours
• CC - Closest Centroid
• LDA - Artificial Neural Network
• SVM - Support Vector Machines
• RF - Random Forests

• Training set: 2 groups (or more) of samples to be used as classifier
  (obviously there should be enough replicates)
  
• Test set: samples which shall be predicted and characterized as: similar to class 1 or class 2

Analysis

Analysis & Prediction

Predict

In Log tab-sheet:

Classification graph:

• Red markers represent vectors assigned to class 1
• Blue markers represent vectors assigned to class 2
• Empty boxes = training vectors
• Solid diamonds = query vectors
• horizontal lines indicate custom defined confidence interval

• 20 traning sample were used (empty boxes)
• all 10 Class-2 samples (blue boxes) were correctly assigned to class in the crossvalidation (positive score)
• but 2 Class-2 samples with very low confidence (Score~0)
• from the 10 Class-1 training samples, (red-boxes
• one was wrongly assigned to Class-2 but with low confidence
• all 5 query samples (solid diamonds) were assigned to Class-2 (blue)
• but 2 at very low confidence (score~0)

LDA - Linear Discriminant Analysis

Discriminant analysis is a statistical technique to classify objects into mutually exclusive and exhaustive groups based on a set of measurable object's features.
Linear Discriminant Analysis (LDA) is used as dimensionality reduction technique for pattern-classification and machine learning applications - similar to Principal Component Analysis (PCA). The goal is to project a dataset onto a lower-dimensional space with good class-separability.

LDA tries to find the axes that maximize the separation between the classes, wheras PCA tries to find the axis which maximize the variance in the data.

LDA is a parametric method assumung unimodal Gaussian likelihoods which are linearly separable.


For significantly non-Gaussian distrbutions, LDA might not generate expected results:


LDA performs similar to Closest-Centroids.

---

CC - Closest Centroid

CC is an easy to understand straight forward method.

For each of the two traning sets (T1,T2) with multiple replicates for the two penotypes, compute the controids C1,C2.
I.e. component wise the mean for all features (genes) within the sets T1,T2 individually.
Depending of data type and assumend distribution use

• arithmetic mean - Gaussion distributed data (linear/log)
• median mean - non parametric (non Gaussian) disributed data (linear/log)
  outlier insensitive
• geometric mean - ratio data

For all new samples, compute the similarity (distance) to all centroids d_i.

Assign the new samples to that class where similarity is largest (distance is smallest).

Distances may be computed as simple geometric distances (Euclidean, Manhattan, Chebyshev, Canberrra, Gowan) or as correlation distances (Pearson, Spearman, Kendall's Tau).

It is also straightforward to normalize the distance of an individual sample to the distance between the centroids, thus allowing to score the quality of the classification for each new sample.

Centroids may be reused for other non-classified samples.

CC uses the information from ALL class members to assign the new samples independant of the internal distribution of training set members.

Thus, CC should give best results for homogenousliy, symmetric datasets - similar to LDA.

---

KNN - K Nearest Neighbours

KNN computes the distances of the new unclassified sample (X_s) to all members of the training sets.
Next we take the K most similar (smallest distance = neighbouring) vectors, and count their class assignments.
The new sample is assigned to the class with highest count number.

Let's look at a (constructed) example:
Traning set 1 and 2 with vectors a,b,c,d, ... each.
C₁, C₂ the centroids of the training sets (computed with arithmetic mean).
The Black, Green lines represent euclidean distance from the uncharacterized sample X_s to all training vectors:


Table of distances from X_s to all members of the training sets T_1i, T_2i;

Sample:	T1	T2
a	2.5	6
b	4	3
c	1	5
d	2.5	2
e	5	4
f	5	1
g	2	1
h	4.5	6
i	4	7
j	4

Lets rank the trainng set vectors by their distances to X_s:

Sample	T1c	T2f	T2g	T1g	T2d	T1d	T2b	...
Rank	1	2	3	4	5	6	7	...
Distance	1	1	1	2	2	2.5	3	...

For this example we choose K=5.
I.e. we take the 5 traning set vectors with smallest distance (T_1c, T_2f, T_2g, T_1g, T_2d) and count their class membership:

Class	Count
T1	2
T2	3

Thus we would assign the new sample to class 2 with KNN classification.

In opposite to Closest Centroid/LDA: there, we would assign the sample to class 1.

Obviously, KNN may be better suited to predict non-homogenious non symetric training data sets.

---

---

ANN - Artificial Neural Network

SVM - Support Vector Machines

RF - Random Forests

As input, the tree requires a vector containing information about the attributes of an Apple tree (e.g. age of the tree, strain: natural or variety, rich or poor soil). Starting with the root node of the decision-making rules of the tree be applied to the input vector. At each node one of the attributes of the input vector is queried. In the example, at the root node the age of the Apple tree. The answer decides: old=>no fruit, young=>proceed to next node. After a series of evaluated rules, you have the answer to the original question. Not always, all levels (nodes) of the decision tree must be traversed.

• Training: 2 or mopre groups groups (Class1, Class2, ...)
• Training + Predicition (Class1, Class2, ... + Query)
• Prediction with forest : 1 group (Query)
  NB: - Query data set should contain the same features in same order as the training set
  - Use the same parameters as used in the training

• same number and order of features
• same platform (e.g. transcriptome array)
• same data normalization / transformation.

Trouble shooting