What is the Curse of Dimensionality?

In machine learning dimensionality simply refers to the number of features(I.e. input variables in the datasets). when the number of features is very large relative to the number of observations in the dataset certain algorithms struggle to train efficient models.

Let me put it this way imagine that you have a straight line of hundred meters long and you dropped a coin as Euro somewhere on it wouldn’t be too hard to find. You take some steps along the line and it takes you a couple of minutes. Now let’s say you have a square of one hundred meters and you dropped a…