{"id":2148,"date":"2025-09-07T20:05:54","date_gmt":"2025-09-07T19:05:54","guid":{"rendered":"https:\/\/cyberenlightener.com\/?page_id=2148"},"modified":"2025-10-04T10:16:53","modified_gmt":"2025-10-04T09:16:53","slug":"unsupervised-learning","status":"publish","type":"page","link":"https:\/\/cyberenlightener.com\/?page_id=2148","title":{"rendered":"Unsupervised learning"},"content":{"rendered":"\n<h2 class=\"wp-block-heading\"><strong>K-MEANS CLUSTERING<\/strong><\/h2>\n\n\n\n<p><strong>K-Means<\/strong>\u00a0clustering is a powerful and efficient algorithm for grouping data points into clusters based on similarity. Despite its simplicity, it is widely used in various fields due to its scalability, ease of use, and effectiveness in uncovering patterns in large datasets. However, it is important to consider its limitations, such as sensitivity to initialization and the need for a predefined number of clusters. Despite these challenges, K-Means remains one of the most popular clustering algorithms in data analysis and machine learning.<\/p>\n\n\n\n<p><strong><a href=\"https:\/\/medium.com\/@karna.sujan52\/k-means-algorithm-solved-numerical-3c94d25076e8\">Excercise-1<\/a><\/strong><\/p>\n\n\n\n<p>Implement the K-Means algorithm with K = 2 on the data points (185, 72), (170, 56), (168, 60), (179, 68), (182, 72), and (188, 77) for two iterations, and display the resulting clusters. Initially, select the first two data points as the initial centroids.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"598\" height=\"301\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/table-1.jpg\" alt=\"\" class=\"wp-image-2320\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/table-1.jpg 598w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/table-1-300x151.jpg 300w\" sizes=\"auto, (max-width: 598px) 100vw, 598px\" \/><\/figure>\n\n\n\n<p>In the\u00a0<strong>initial step,\u00a0<\/strong>we determine the similarity between data points using the Euclidean distance metric.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"822\" height=\"753\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/3_first.webp\" alt=\"\" class=\"wp-image-2321\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/3_first.webp 822w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/3_first-300x275.webp 300w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/3_first-768x704.webp 768w\" sizes=\"auto, (max-width: 822px) 100vw, 822px\" \/><\/figure>\n\n\n\n<p>In tabular-form it can be represented as,<\/p>\n\n\n\n<figure class=\"wp-block-image size-full is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"544\" height=\"173\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/3_2nd.webp\" alt=\"\" class=\"wp-image-2322\" style=\"width:1029px;height:auto\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/3_2nd.webp 544w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/3_2nd-300x95.webp 300w\" sizes=\"auto, (max-width: 544px) 100vw, 544px\" \/><\/figure>\n\n\n\n<p>The result after first iteration.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"608\" height=\"313\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/4th.png\" alt=\"\" class=\"wp-image-2323\" style=\"width:896px;height:auto\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/4th.png 608w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/4th-300x154.png 300w\" sizes=\"auto, (max-width: 608px) 100vw, 608px\" \/><\/figure>\n\n\n\n<p>In the second iteration, calculating centroids again,<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"200\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/5th-1-1024x200.webp\" alt=\"\" class=\"wp-image-2325\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/5th-1-1024x200.webp 1024w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/5th-1-300x59.webp 300w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/5th-1-768x150.webp 768w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/5th-1.webp 1047w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p>Calculating distances again,<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"659\" height=\"807\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/6th.webp\" alt=\"\" class=\"wp-image-2326\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/6th.webp 659w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/6th-245x300.webp 245w\" sizes=\"auto, (max-width: 659px) 100vw, 659px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"632\" height=\"174\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/7th.jpg\" alt=\"\" class=\"wp-image-2328\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/7th.jpg 632w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/7th-300x83.jpg 300w\" sizes=\"auto, (max-width: 632px) 100vw, 632px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"590\" height=\"260\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/8TH.png\" alt=\"\" class=\"wp-image-2327\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/8TH.png 590w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/8TH-300x132.png 300w\" sizes=\"auto, (max-width: 590px) 100vw, 590px\" \/><\/figure>\n\n\n\n<p>As two iterations have already been completed as required by the problem, the numerical process concludes here. Since the clustering remains unchanged after the second iteration, the process will be terminated, even if the question does not explicitly state to do so.<\/p>\n\n\n<a class=\"wp-block-read-more\" href=\"https:\/\/cyberenlightener.com\/?page_id=2148\" target=\"_self\">Read more<span class=\"screen-reader-text\">: Unsupervised learning<\/span><\/a>\n\n\n<h2 class=\"wp-block-heading\"><strong>K-MODES CLUSTERING<\/strong><\/h2>\n\n\n\n<div data-wp-interactive=\"core\/file\" class=\"wp-block-file\"><object data-wp-bind--hidden=\"!state.hasPdfPreview\" hidden class=\"wp-block-file__embed\" data=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/K-Modes-clustering-tutorial-2-1.pdf\" type=\"application\/pdf\" style=\"width:100%;height:600px\" aria-label=\"Embed of K-Modes clustering tutorial (2).\"><\/object><a id=\"wp-block-file--media-aa8d7e9b-bccd-4136-ac64-6b9b3a314896\" href=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/K-Modes-clustering-tutorial-2-1.pdf\">K-Modes clustering tutorial (2)<\/a><a href=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/K-Modes-clustering-tutorial-2-1.pdf\" class=\"wp-block-file__button wp-element-button\" download aria-describedby=\"wp-block-file--media-aa8d7e9b-bccd-4136-ac64-6b9b3a314896\">Download<\/a><\/div>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Hierarchical Clustering<\/strong><\/h2>\n\n\n\n<div data-wp-interactive=\"core\/file\" class=\"wp-block-file\"><object data-wp-bind--hidden=\"!state.hasPdfPreview\" hidden class=\"wp-block-file__embed\" data=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/hierarchical_clustering.pdf\" type=\"application\/pdf\" style=\"width:100%;height:600px\" aria-label=\"Embed of hierarchical_clustering.\"><\/object><a id=\"wp-block-file--media-93bd4b0d-9ff8-4727-9fcf-44c3c08a1971\" href=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/hierarchical_clustering.pdf\">hierarchical_clustering<\/a><a href=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/hierarchical_clustering.pdf\" class=\"wp-block-file__button wp-element-button\" download aria-describedby=\"wp-block-file--media-93bd4b0d-9ff8-4727-9fcf-44c3c08a1971\">Download<\/a><\/div>\n\n\n\n<div data-wp-interactive=\"core\/file\" class=\"wp-block-file\"><object data-wp-bind--hidden=\"!state.hasPdfPreview\" hidden class=\"wp-block-file__embed\" data=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/Dendogram-Hierarchical_CLustering.pdf\" type=\"application\/pdf\" style=\"width:100%;height:600px\" aria-label=\"Embed of Dendogram-Hierarchical_CLustering.\"><\/object><a id=\"wp-block-file--media-20a5cc70-ff2c-4015-9964-825fb6b36f0b\" href=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/Dendogram-Hierarchical_CLustering.pdf\">Dendogram-Hierarchical_CLustering<\/a><a href=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/Dendogram-Hierarchical_CLustering.pdf\" class=\"wp-block-file__button wp-element-button\" download aria-describedby=\"wp-block-file--media-20a5cc70-ff2c-4015-9964-825fb6b36f0b\">Download<\/a><\/div>\n\n\n\n<div data-wp-interactive=\"core\/file\" class=\"wp-block-file\"><object data-wp-bind--hidden=\"!state.hasPdfPreview\" hidden class=\"wp-block-file__embed\" data=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/Solved_example.pdf\" type=\"application\/pdf\" style=\"width:100%;height:600px\" aria-label=\"Embed of Solved_example.\"><\/object><a id=\"wp-block-file--media-659df37a-e07d-475c-b99a-8adf55a28a56\" href=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/Solved_example.pdf\">Solved_example<\/a><a href=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/Solved_example.pdf\" class=\"wp-block-file__button wp-element-button\" download aria-describedby=\"wp-block-file--media-659df37a-e07d-475c-b99a-8adf55a28a56\">Download<\/a><\/div>\n\n\n\n<div data-wp-interactive=\"core\/file\" class=\"wp-block-file\"><object data-wp-bind--hidden=\"!state.hasPdfPreview\" hidden class=\"wp-block-file__embed\" data=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/DBSCAN.pdf\" type=\"application\/pdf\" style=\"width:100%;height:600px\" aria-label=\"Embed of DBSCAN.\"><\/object><a id=\"wp-block-file--media-a78f622f-c3ae-456a-ac1e-44b7ba88eb90\" href=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/DBSCAN.pdf\">DBSCAN<\/a><a href=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/10\/DBSCAN.pdf\" class=\"wp-block-file__button wp-element-button\" download aria-describedby=\"wp-block-file--media-a78f622f-c3ae-456a-ac1e-44b7ba88eb90\">Download<\/a><\/div>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Expectation-Maximization (EM) Algorithm<\/strong><\/h2>\n\n\n\n<p>The <strong>Expectation-Maximization (EM) algorithm<\/strong> is a powerful iterative method to estimate parameters of probabilistic models when data is <strong>incomplete, uncertain, or involves hidden variables<\/strong>.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Idea:<\/strong> Direct maximization of likelihood is hard due to missing\/hidden data. EM solves this by alternating between two intuitive steps:\n<ol class=\"wp-block-list\">\n<li><strong>E-step (Expectation):<\/strong> Compute the expected value of hidden variables using current parameters.<\/li>\n\n\n\n<li><strong>M-step (Maximization):<\/strong> Update parameters by maximizing the likelihood with these expectations.<\/li>\n<\/ol>\n<\/li>\n<\/ul>\n\n\n\n<p>This <strong>repeat\u2013refine process<\/strong> guarantees non-decreasing likelihood and converges to a local optimum. EM underlies key methods like <strong>Gaussian Mixture Models, Hidden Markov Models, and clustering with incomplete data<\/strong>, making it a cornerstone of statistical learning.<\/p>\n\n\n\n<p><strong>Other applications : Image Processing<\/strong>: Segmentation, denoising;<strong>Missing Data Problems<\/strong>: Robust parameter estimation;<strong>Anomaly Detection<\/strong>: Fraud, outliers.,<strong>Medical &amp; Bioinformatics<\/strong>: Gene expression, disease models;<strong>Recommender Systems<\/strong>: Latent factor models.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"683\" height=\"1024\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/Roy_EM2025-683x1024.png\" alt=\"\" class=\"wp-image-2166\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/Roy_EM2025-683x1024.png 683w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/Roy_EM2025-200x300.png 200w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/Roy_EM2025-768x1152.png 768w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/Roy_EM2025.png 1024w\" sizes=\"auto, (max-width: 683px) 100vw, 683px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"765\" height=\"1024\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1-3-765x1024.jpeg\" alt=\"\" class=\"wp-image-2183\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1-3-765x1024.jpeg 765w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1-3-224x300.jpeg 224w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1-3-768x1028.jpeg 768w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1-3.jpeg 956w\" sizes=\"auto, (max-width: 765px) 100vw, 765px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"739\" height=\"1024\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/2-3-739x1024.jpeg\" alt=\"\" class=\"wp-image-2181\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/2-3-739x1024.jpeg 739w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/2-3-217x300.jpeg 217w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/2-3-768x1064.jpeg 768w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/2-3.jpeg 924w\" sizes=\"auto, (max-width: 739px) 100vw, 739px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"767\" height=\"1024\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/3-1-767x1024.jpeg\" alt=\"\" class=\"wp-image-2185\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/3-1-767x1024.jpeg 767w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/3-1-225x300.jpeg 225w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/3-1-768x1025.jpeg 768w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/3-1.jpeg 959w\" sizes=\"auto, (max-width: 767px) 100vw, 767px\" \/><\/figure>\n\n\n\n<p>Mathematical Prerequisites:<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"657\" height=\"105\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/likelyhood.png\" alt=\"\" class=\"wp-image-2172\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/likelyhood.png 657w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/likelyhood-300x48.png 300w\" sizes=\"auto, (max-width: 657px) 100vw, 657px\" \/><\/figure>\n\n\n\n<p><strong>MCQs on EM<\/strong><\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>What is the main purpose of the EM algorithm?<\/strong><br>A) Solve linear equations<br>B) Maximize likelihood with incomplete data<br>C) Minimize squared error<br>D) Perform gradient descent<br><strong>Answer:<\/strong> B) Maximize likelihood with incomplete data<\/li>\n\n\n\n<li><strong>Which two main steps does EM consist of?<\/strong><br>A) Encode and Decode<br>B) Expectation and Maximization<br>C) Sampling and Updating<br>D) Forward and Backward<br><strong>Answer:<\/strong> B) Expectation and Maximization<\/li>\n\n\n\n<li><strong>EM is typically used when:<\/strong><br>A) All data is observed<br>B) Data is incomplete or has latent variables<br>C) The dataset is small<br>D) The data is categorical only<br><strong>Answer:<\/strong> B) Data is incomplete or has latent variables<\/li>\n\n\n\n<li><strong>The EM algorithm guarantees:<\/strong><br>A) Convergence to global maximum likelihood<br>B) Convergence to a local maximum of likelihood<br>C) Exact posterior distributions<br>D) Linear convergence speed<br><strong>Answer:<\/strong> B) Convergence to a local maximum of likelihood<\/li>\n<\/ol>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h3 class=\"wp-block-heading\"><\/h3>\n\n\n\n<ol start=\"5\" class=\"wp-block-list\">\n<li><strong>In the Gaussian Mixture Model (GMM), what does the E-step compute?<\/strong><br>A) Update the means of clusters<br>B) Update the covariance matrices<br>C) Compute responsibilities (probabilities of points belonging to clusters)<br>D) Compute the gradient of the likelihood<br><strong>Answer:<\/strong> C) Compute responsibilities (probabilities of points belonging to clusters)<\/li>\n\n\n\n<li><strong>Which of the following is a limitation of EM?<\/strong><br>A) Requires derivative computations<br>B) Sensitive to initial parameter values<br>C) Can only handle two clusters<br>D) Works only with discrete data<br><strong>Answer:<\/strong> B) Sensitive to initial parameter values<\/li>\n\n\n\n<li><strong>After how many iterations does EM converge?<\/strong><br>A) Always in one iteration<br>B) Depends on the convergence criterion (e.g., change in likelihood)<br>C) Always after 10 iterations<br>D) EM never converges<br><strong>Answer:<\/strong> B) Depends on the convergence criterion (e.g., change in likelihood)<\/li>\n\n\n\n<li><strong>Which of these is true about the log-likelihood in EM?<\/strong><br>A) It decreases in each iteration<br>B) It increases or stays the same in each iteration<br>C) It oscillates randomly<br>D) It remains constant<br><strong>Answer:<\/strong> B) It increases or stays the same in each iteration<\/li>\n<\/ol>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<ol start=\"9\" class=\"wp-block-list\">\n<li><strong>Consider EM applied to a mixture of two Gaussians with identical variances. If the initial means are very close, which problem can occur?<\/strong><br>A) The algorithm will converge to the true global maximum<br>B) The algorithm may converge to a degenerate solution or local maximum<br>C) The E-step cannot be computed<br>D) The covariance will become negative<br><strong>Answer:<\/strong> B) The algorithm may converge to a degenerate solution or local maximum<\/li>\n\n\n\n<li><strong>Which mathematical property of the EM algorithm ensures the likelihood never decreases?<\/strong><br>A) Jensen\u2019s inequality<br>B) Cauchy-Schwarz inequality<br>C) Bayes\u2019 theorem<br>D) Law of Large Numbers<br><strong>Answer:<\/strong> A) Jensen\u2019s inequality<\/li>\n<\/ol>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Self Organizing Maps(SOM)<\/strong><\/h2>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"802\" height=\"1024\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1_1-1-802x1024.jpeg\" alt=\"\" class=\"wp-image-2216\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1_1-1-802x1024.jpeg 802w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1_1-1-235x300.jpeg 235w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1_1-1-768x981.jpeg 768w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1_1-1.jpeg 1002w\" sizes=\"auto, (max-width: 802px) 100vw, 802px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-full is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"622\" height=\"366\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/SOM2.png\" alt=\"\" class=\"wp-image-2194\" style=\"width:1200px;height:auto\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/SOM2.png 622w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/SOM2-300x177.png 300w\" sizes=\"auto, (max-width: 622px) 100vw, 622px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"760\" height=\"1024\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1_2-760x1024.jpeg\" alt=\"\" class=\"wp-image-2191\" style=\"width:1188px;height:auto\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1_2-760x1024.jpeg 760w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1_2-223x300.jpeg 223w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1_2-768x1035.jpeg 768w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1_2.jpeg 950w\" sizes=\"auto, (max-width: 760px) 100vw, 760px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"774\" height=\"1024\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1_3-774x1024.jpeg\" alt=\"\" class=\"wp-image-2192\" style=\"width:1181px;height:auto\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1_3-774x1024.jpeg 774w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1_3-227x300.jpeg 227w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1_3-768x1017.jpeg 768w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1_3.jpeg 967w\" sizes=\"auto, (max-width: 774px) 100vw, 774px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"769\" height=\"1024\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1_4-769x1024.jpeg\" alt=\"\" class=\"wp-image-2193\" style=\"width:1197px;height:auto\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1_4-769x1024.jpeg 769w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1_4-225x300.jpeg 225w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1_4-768x1023.jpeg 768w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/1_4.jpeg 961w\" sizes=\"auto, (max-width: 769px) 100vw, 769px\" \/><\/figure>\n\n\n\n<p><strong>POINTS TO REMEMBER<\/strong>&#8211;&gt;<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Unsupervised neural network using competitive, neighborhood-based weight adaptation.<\/li>\n\n\n\n<li>Preserves topological relationships: nearby neurons map nearby inputs.<\/li>\n\n\n\n<li>Dimensionality reduction alternative to PCA, nonlinear manifold preserving.<\/li>\n\n\n\n<li>Converges via decreasing learning rate and shrinking neighborhood radius.<\/li>\n\n\n\n<li>Applications: clustering, visualization, anomaly detection, feature extraction.<\/li>\n\n\n\n<li>SOM scales poorly; parallel GPU\/mini-batch training mitigates complexity.<\/li>\n\n\n\n<li>Topology choice (hexagonal vs rectangular grid) impacts neighborhood smoothing.<\/li>\n\n\n\n<li>Initialization (PCA-based vs random) strongly influences convergence stability.<\/li>\n\n\n\n<li>Quantization error and topographic error are key SOM evaluation metrics.<\/li>\n\n\n\n<li>Used for high-dimensional clustering in genomics, NLP embeddings, finance.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>PCA<\/strong><\/h2>\n\n\n\n<p><strong>PCA (Principal Component Analysis) is a statistical<\/strong> method used to reduce the number of variables (dimensions) in a large dataset while retaining most of the original data\u2019s variation and information. It does this by converting the data into new, independent variables called principal components, which help simplify complex data, improve visualizations, boost machine learning model performance, and make datasets easier to analyze and interpret.<\/p>\n\n\n\n<div data-wp-interactive=\"core\/file\" class=\"wp-block-file\"><object data-wp-bind--hidden=\"!state.hasPdfPreview\" hidden class=\"wp-block-file__embed\" data=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/pca_4thAug.pdf\" type=\"application\/pdf\" style=\"width:100%;height:600px\" aria-label=\"Embed of pca_4thAug.\"><\/object><a id=\"wp-block-file--media-472615dd-c91f-430a-8620-7bc9a3127564\" href=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/pca_4thAug.pdf\">pca_4thAug<\/a><a href=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/pca_4thAug.pdf\" class=\"wp-block-file__button wp-element-button\" download aria-describedby=\"wp-block-file--media-472615dd-c91f-430a-8620-7bc9a3127564\">Download<\/a><\/div>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>PCA-KERNEL<\/strong><\/h2>\n\n\n\n<div data-wp-interactive=\"core\/file\" class=\"wp-block-file\"><object data-wp-bind--hidden=\"!state.hasPdfPreview\" hidden class=\"wp-block-file__embed\" data=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/PCA_KERNEL-1.pdf\" type=\"application\/pdf\" style=\"width:100%;height:600px\" aria-label=\"Embed of PCA_KERNEL.\"><\/object><a id=\"wp-block-file--media-83d49c79-b3b9-4a0a-a032-d023affb413c\" href=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/PCA_KERNEL-1.pdf\">PCA_KERNEL<\/a><a href=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/PCA_KERNEL-1.pdf\" class=\"wp-block-file__button wp-element-button\" download aria-describedby=\"wp-block-file--media-83d49c79-b3b9-4a0a-a032-d023affb413c\">Download<\/a><\/div>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"469\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/pcakernelssroy-1-1024x469.png\" alt=\"\" class=\"wp-image-2290\" style=\"width:1200px;height:auto\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/pcakernelssroy-1-1024x469.png 1024w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/pcakernelssroy-1-300x137.png 300w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/pcakernelssroy-1-768x352.png 768w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/pcakernelssroy-1.png 1360w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>t-SNE (t-distributed Stochastic Neighbor Embedding)<\/strong><\/h2>\n\n\n\n<p><strong>t-SNE<\/strong> (t-distributed Stochastic Neighbor Embedding), introduced by <strong>Laurens van der Maaten and Geoffrey Hinton (2008)<\/strong>, is a <strong>non-linear dimensionality reduction<\/strong> technique, mainly used for <strong>visualizing high-dimensional data<\/strong> in 2D or 3D.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"644\" height=\"1024\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tsne1-2-644x1024.jpeg\" alt=\"\" class=\"wp-image-2219\" style=\"width:873px;height:auto\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tsne1-2-644x1024.jpeg 644w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tsne1-2-189x300.jpeg 189w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tsne1-2-768x1221.jpeg 768w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tsne1-2.jpeg 805w\" sizes=\"auto, (max-width: 644px) 100vw, 644px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"710\" height=\"1024\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tsne21-3-710x1024.jpeg\" alt=\"\" class=\"wp-image-2233\" style=\"width:885px;height:auto\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tsne21-3-710x1024.jpeg 710w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tsne21-3-208x300.jpeg 208w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tsne21-3-768x1107.jpeg 768w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tsne21-3.jpeg 888w\" sizes=\"auto, (max-width: 710px) 100vw, 710px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"722\" height=\"1024\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tse3-3-722x1024.jpeg\" alt=\"\" class=\"wp-image-2220\" style=\"width:884px;height:auto\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tse3-3-722x1024.jpeg 722w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tse3-3-212x300.jpeg 212w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tse3-3-768x1089.jpeg 768w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tse3-3.jpeg 903w\" sizes=\"auto, (max-width: 722px) 100vw, 722px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"689\" height=\"1024\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tsne4-4-689x1024.jpeg\" alt=\"\" class=\"wp-image-2225\" style=\"width:889px;height:auto\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tsne4-4-689x1024.jpeg 689w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tsne4-4-202x300.jpeg 202w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tsne4-4-768x1142.jpeg 768w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tsne4-4.jpeg 861w\" sizes=\"auto, (max-width: 689px) 100vw, 689px\" \/><\/figure>\n\n\n\n<figure class=\"wp-block-image size-large is-resized\"><img loading=\"lazy\" decoding=\"async\" width=\"712\" height=\"1024\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tsne5-2-712x1024.jpeg\" alt=\"\" class=\"wp-image-2223\" style=\"width:888px;height:auto\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tsne5-2-712x1024.jpeg 712w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tsne5-2-209x300.jpeg 209w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tsne5-2-768x1105.jpeg 768w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/tsne5-2.jpeg 890w\" sizes=\"auto, (max-width: 712px) 100vw, 712px\" \/><\/figure>\n\n\n\n<p><strong>Important points<\/strong>-&gt;<\/p>\n\n\n\n<p><strong>How does perplexity influence structure?<\/strong><br>\u2192 Balances local detail vs. global cluster relationships.<\/p>\n\n\n\n<p><strong>Why Student-t distribution instead of Gaussian?<\/strong><br>\u2192 Heavy tails prevent crowding in low dimensions.<\/p>\n\n\n\n<p><strong>Why asymmetric KL-divergence?<\/strong><br>\u2192 Emphasizes preserving local neighborhoods more than global structure.<\/p>\n\n\n\n<p><strong>t-SNE vs. UMAP trade-offs?<\/strong><br>\u2192 UMAP faster, preserves global structure better than t-SNE.<\/p>\n\n\n\n<p><strong>Why not for downstream ML tasks?<\/strong><br>\u2192 Embeddings distort distances; unsuitable for predictive models.<\/p>\n\n\n\n<p><strong>Effect of initialization?<\/strong><br>\u2192 PCA gives stable maps; random can cause variability.<\/p>\n\n\n\n<p><strong>Scaling to millions of points?<\/strong><br>\u2192 Use Barnes-Hut or FFT approximations for efficiency.<\/p>\n\n\n\n<p><strong>Limitations of interpretability?<\/strong><br>\u2192 Global distances meaningless; only local clusters reliable.<\/p>\n\n\n\n<p><strong>Relation to manifold learning?<\/strong><br>\u2192 Captures local manifold structure via probabilistic similarities.<\/p>\n\n\n\n<p><strong>Misleading conclusions risk?<\/strong><br>\u2192 Wrong perplexity or iterations produce artificial clusters.<\/p>\n\n\n\n<p><strong>Kullback\u2013Leibler (KL) divergence<\/strong>&#8211;&gt;<\/p>\n\n\n\n<p><strong>Kullback\u2013Leibler (KL) divergence<\/strong> is an <strong>information-theoretic measure<\/strong> of how a probability distribution PPP diverges from a reference distribution Q. It is <strong>asymmetric<\/strong>, always non-negative, and equals zero only if P=Q. Widely used in <strong>t-SNE, variational inference, and generative models<\/strong>, KL-divergence quantifies <strong>information loss<\/strong> when approximating P with Q.<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"316\" height=\"69\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/KL.png\" alt=\"\" class=\"wp-image-2212\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/KL.png 316w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/KL-300x66.png 300w\" sizes=\"auto, (max-width: 316px) 100vw, 316px\" \/><\/figure>\n\n\n\n<div data-wp-interactive=\"core\/file\" class=\"wp-block-file\"><object data-wp-bind--hidden=\"!state.hasPdfPreview\" hidden class=\"wp-block-file__embed\" data=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/t-SNE-EX-1ROY.pdf\" type=\"application\/pdf\" style=\"width:100%;height:600px\" aria-label=\"Embed of t-SNE-EX-1ROY.\"><\/object><a id=\"wp-block-file--media-cccabbdc-6db9-450e-8da7-8cd33b80c708\" href=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/t-SNE-EX-1ROY.pdf\">t-SNE-EX-1ROY<\/a><a href=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/t-SNE-EX-1ROY.pdf\" class=\"wp-block-file__button wp-element-button\" download aria-describedby=\"wp-block-file--media-cccabbdc-6db9-450e-8da7-8cd33b80c708\">Download<\/a><\/div>\n\n\n\n<p>MATRIX DATA CALCULATION.*<\/p>\n\n\n\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"605\" height=\"495\" src=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/image.png\" alt=\"\" class=\"wp-image-2214\" srcset=\"https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/image.png 605w, https:\/\/cyberenlightener.com\/wp-content\/uploads\/2025\/09\/image-300x245.png 300w\" sizes=\"auto, (max-width: 605px) 100vw, 605px\" \/><\/figure>\n","protected":false},"excerpt":{"rendered":"<p>K-MEANS CLUSTERING K-Means\u00a0clustering is a powerful and efficient algorithm for grouping data points into clusters based on similarity. Despite its simplicity, it is widely used in various fields due to its scalability, ease of use, and effectiveness in uncovering patterns in large datasets. However, it is important to consider its limitations, such as sensitivity to [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"site-sidebar-layout":"default","site-content-layout":"","ast-site-content-layout":"default","site-content-style":"default","site-sidebar-style":"default","ast-global-header-display":"","ast-banner-title-visibility":"","ast-main-header-display":"","ast-hfb-above-header-display":"","ast-hfb-below-header-display":"","ast-hfb-mobile-header-display":"","site-post-title":"","ast-breadcrumbs-content":"","ast-featured-img":"","footer-sml-layout":"","ast-disable-related-posts":"","theme-transparent-header-meta":"","adv-header-id-meta":"","stick-header-meta":"","header-above-stick-meta":"","header-main-stick-meta":"","header-below-stick-meta":"","astra-migrate-meta-layouts":"default","ast-page-background-enabled":"default","ast-page-background-meta":{"desktop":{"background-color":"var(--ast-global-color-4)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"ast-content-background-meta":{"desktop":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"tablet":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""},"mobile":{"background-color":"var(--ast-global-color-5)","background-image":"","background-repeat":"repeat","background-position":"center center","background-size":"auto","background-attachment":"scroll","background-type":"","background-media":"","overlay-type":"","overlay-color":"","overlay-opacity":"","overlay-gradient":""}},"footnotes":""},"class_list":["post-2148","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/cyberenlightener.com\/index.php?rest_route=\/wp\/v2\/pages\/2148","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cyberenlightener.com\/index.php?rest_route=\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/cyberenlightener.com\/index.php?rest_route=\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/cyberenlightener.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/cyberenlightener.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2148"}],"version-history":[{"count":21,"href":"https:\/\/cyberenlightener.com\/index.php?rest_route=\/wp\/v2\/pages\/2148\/revisions"}],"predecessor-version":[{"id":2329,"href":"https:\/\/cyberenlightener.com\/index.php?rest_route=\/wp\/v2\/pages\/2148\/revisions\/2329"}],"wp:attachment":[{"href":"https:\/\/cyberenlightener.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2148"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}