Generate a random dataset (1110 rows, 3 columns). The last column of dataset should be an integer and also the cluster number. Treat this column as unknown value. You are going to cluster the dataset using only first two columns and test your results by verifying the total number of clusters.
Use Kmeans to determine the # of clusters. Tabulate the # of clusters from 1 – 40 and total within-cluster variance. Plot the scree plot. Using hierarchal clustering, calculate the pairwise distance. Create various dendrograms using complete and average linkage. Cut dendogram into groups of 5-7. Which is the most appropriate # of groups? Explain results of each question listed.
Output: Python coding and commenting to be included Jupyter notebook file (format: .ipynb)
Other details: Using the randomly generated 1100 by 3 data set, you are going to practice different clustering methods to learn how each method works, to observe how each provides the output graphically, and how to make a decision based from the output.