So far, we have built tables for Q-values or V-values (tabular TD learning) which assumes discrete spaces. However, input spaces are in reality mostly continuous. Even where they look discrete such as in games as BackGammon, the number of states is so huge, that we cannot fill a table. Hence we need to model the input with a neural network or radial basis function network. How this is done is explained in this video.