![.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Dimension Reduction
MDS
Isomap
Reference
Classical Method
Mathematical Formula
Steps
Example
Mathematical Formula
Find spectral decomposition of B
B = V ΛV T
= (V Λ1/2
)(V Λ1/2
)T
= ZZT
Coordinator matrix Z can be assigned by
Z = V Λ1/2
where Λ is the diagonal matrix defined by Λii = λi the
eigenvalues that λ1 ≥ λ2 . . . λm ≥ 0, and V = [v1, v2, . . . , vm]
is matrix of eigenvectors.
Li, Cheng-Shiang Multidimension Scaling and Isomap](https://image.slidesharecdn.com/mds-slide-170214030121/85/Multidimension-Scaling-and-Isomap-7-320.jpg)
![.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Dimension Reduction
MDS
Isomap
Reference
Classical Method
Mathematical Formula
Steps
Example
Steps
Set up the proximity matrix D = [dij ].
Apply the double centering: B = −1
2 HDH using the centering
matrix H = I − 1
m 11
′
where m is the number of objects.
Determine the k largest eigenvalues λ1, λ2, ..., λk and
corresponding eigenvectors e1, e2, ..., ek of B
Then, Z = EkΛ
1/2
k , where Ek is the matrix of k eigenvectors
and Λk is the diagonal matrix of k eigenvalues of B.
Li, Cheng-Shiang Multidimension Scaling and Isomap](https://image.slidesharecdn.com/mds-slide-170214030121/85/Multidimension-Scaling-and-Isomap-8-320.jpg)
Multidimension Scaling and Isomap
- 1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dimension Reduction MDS Isomap Reference Multidimension Scaling and Isomap Li, Cheng-Shiang September 7, 2016 Li, Cheng-Shiang Multidimension Scaling and Isomap
- 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dimension Reduction MDS Isomap Reference Introduction Algorithm Dimension Reduction Curse of dimension. Choose a subset of important features. Feature selection and extraction. Selection: Find a subset of original variable. Extraction: Transform the data in high-dimensional space to a space lower of fewer dimensions. Li, Cheng-Shiang Multidimension Scaling and Isomap
- 3. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dimension Reduction MDS Isomap Reference Introduction Algorithm Algorithm PCA (Principal Components Analysis) LDA (Linear Discriminant Analysis) MDS (MultiDimension Scaling) Li, Cheng-Shiang Multidimension Scaling and Isomap
- 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dimension Reduction MDS Isomap Reference Classical Method Mathematical Formula Steps Example Classical MDS Source datum X as X = ⎡ ⎢ ⎣ X1 ... Xm ⎤ ⎥ ⎦ = ⎡ ⎢ ⎣ x11 · · · x1n ... ... xm1 · · · xmn ⎤ ⎥ ⎦ X is m × n matrix where m datum and n attributes Find datum Z as Z = ⎡ ⎢ ⎣ Z1 ... Xm ⎤ ⎥ ⎦ = ⎡ ⎢ ⎣ z11 · · · z1k ... ... zm1 · · · zmk ⎤ ⎥ ⎦ Z is m × k matrix where m datum and k attributes Li, Cheng-Shiang Multidimension Scaling and Isomap
- 5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dimension Reduction MDS Isomap Reference Classical Method Mathematical Formula Steps Example Classical MDS Concept Equation D ≈ P D and P are m × m Euclidean distance matrix of each datum of X and Z respectively. Dij is distance between xi and xj as well as Prs is distance between zr and zs. Dij = (xi − xj )(xi − xj )T , and Prs = (zr − zs)(zr − zs)T Li, Cheng-Shiang Multidimension Scaling and Isomap
- 6. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dimension Reduction MDS Isomap Reference Classical Method Mathematical Formula Steps Example Mathematical Formula Assume r zr = 0, the proximity matrix D can be transformed to inner product matrix B by B = − 1 2 HDH = ZZT where H = I − 1 m 11 ′ Li, Cheng-Shiang Multidimension Scaling and Isomap
- 7. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dimension Reduction MDS Isomap Reference Classical Method Mathematical Formula Steps Example Mathematical Formula Find spectral decomposition of B B = V ΛV T = (V Λ1/2 )(V Λ1/2 )T = ZZT Coordinator matrix Z can be assigned by Z = V Λ1/2 where Λ is the diagonal matrix defined by Λii = λi the eigenvalues that λ1 ≥ λ2 . . . λm ≥ 0, and V = [v1, v2, . . . , vm] is matrix of eigenvectors. Li, Cheng-Shiang Multidimension Scaling and Isomap
- 8. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dimension Reduction MDS Isomap Reference Classical Method Mathematical Formula Steps Example Steps Set up the proximity matrix D = [dij ]. Apply the double centering: B = −1 2 HDH using the centering matrix H = I − 1 m 11 ′ where m is the number of objects. Determine the k largest eigenvalues λ1, λ2, ..., λk and corresponding eigenvectors e1, e2, ..., ek of B Then, Z = EkΛ 1/2 k , where Ek is the matrix of k eigenvectors and Λk is the diagonal matrix of k eigenvalues of B. Li, Cheng-Shiang Multidimension Scaling and Isomap
- 9. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dimension Reduction MDS Isomap Reference Classical Method Mathematical Formula Steps Example Example Original Data −2 0 2 4 6 8 10 −2024681012 −8 −6 −4 −2 0 2 4 6 8 X Y Z −5 0 5 −4−2024 Classical MDS Coordinate 1 Coordinate2 Li, Cheng-Shiang Multidimension Scaling and Isomap
- 10. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dimension Reduction MDS Isomap Reference Isomap Algorithm Example Comparison Isomap - Isometric mapping Classical MDS maybe loss intrinsic structure of datums. Generate global distance of each datum from local information. Apply classical MDS algorithm. Li, Cheng-Shiang Multidimension Scaling and Isomap
- 11. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dimension Reduction MDS Isomap Reference Isomap Algorithm Example Comparison Algorithm Determine the neighbors of each point. All points in some fixed radius. K nearest neighbors. Construct a neighborhood graph. Each point is connected to other if it is a K nearest neighbor. Edge length equal to Euclidean distance. Compute shortest path between two nodes. Dijkstra’s algorithm Floyd–Warshall algorithm Compute lower-dimensional embedding. Multidimensional scaling(MDS) Li, Cheng-Shiang Multidimension Scaling and Isomap
- 12. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dimension Reduction MDS Isomap Reference Isomap Algorithm Example Comparison Example Li, Cheng-Shiang Multidimension Scaling and Isomap
- 13. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dimension Reduction MDS Isomap Reference Isomap Algorithm Example Comparison Comparison Figure: Comparison with PCA and LLE Li, Cheng-Shiang Multidimension Scaling and Isomap
- 14. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dimension Reduction MDS Isomap Reference Reference MDS - Wikipdeia Isomap - Author’s website Isomap - Wikipedia http://ciera.northwestern.edu/Education/REU/2015/ Thorsen/ Li, Cheng-Shiang Multidimension Scaling and Isomap