Regression is a statistical method used to build such predictive models based on historical data. The input data for regression must include both the predictor variables (the attributes that are used as input from which to generate a predicted outcome) and the predicted variable (the output that is expected to occur based upon the input predictor variables).
As an example of this process, imagine that you wanted to predict the water level of lakes in your area. To do so, you collect metrics on rainfall and lake water levels for all of the lakes in your area. magine further that you were able to get this same data collected over the course of several years. Using this data we could develop a model that would allow us to predict what the water level in the lake will be based upon the amount of rainfall that is received.
Regression is a process used to develop a model that rationalizes the data and identifies predictive trends within the data. There are a number of different types of regression models that can be used to predict different types of relationships. Some of these relationships, when plotted, are linear; others have a curved shape. We will walk through an example of the simplest form of regression, which is linear regression, which is used when a scatterplot of the data has roughly a linear shape. Going back to our lake level example Let us assume that we have all of the data as follows.
If we look at this graph, one of the things that we realize is that the plotted values begin to form a pattern. In fact, we could draw a straight line that follows the shape of the data points that have been plotted. In figure (x), we added a line to the graph to indicate the general shape of the points. This line is called a regression line. Essentially, this line provides us with a way to use the data to create a prediction model. For example, if we look at the graph, we can see that, in general, when we have 15 inches of rainfall, the lake levels will tend to be 2 inches lower. We can also see that when we have 20–25 inches of rainfall, the levels of water in the lakes tend to be 3 inches higher.
In the figure, we just kind of "drew" a line on the data, but the regression algorithm employs a process to "fit" the line to the data. The first thing that we need to understand about fitting a regression line is the concept of error. When we look at Figure (x), we can see that many of the data points do not touch the line. The line represents the predicted value of Y for any input value of X. The dots that do not touch the line indicate that the actual data doesn’t match the prediction.
Linear regression is clearly not going to be an accurate solution to predict these values. What is needed is the ability to have a curved line that models the behavior of the data to increase up to a point and then decrease. Fortunately, we have such a solution in curvilinear regression, which uses splines to model data that is not linear yet has a strong relationship between the data attributes. In curvilinear regression, we no longer have an algorithm to fit a "line," but rather an algorithm to fit a "spline," which is accordingly called "spline fitting."
The reason that we square the value is because the values that are above the red line will have a ‘positive’ value and the points below the line will have a ‘negative’ value. When we square them, all of the distances become positive. The algorithm must then minimize the value when all of these ‘errors’ have been added together.
Supervised learning for prediction The lake's water level is an example of a linear regression. We call it linear regression because it fits a straight regression line to the data, and this line is expressed in the form:
y = mx + b
If you recall your basic algebra, this is essentially a function where for any input value of x, there is an output value of y. The m in the equation is called the slope. The slope defines the rise over run or the change in the value of y for any value of x. The b in the equation specifies the y intercept, or the point where the line intercepts the y axis. The y intercept is typically the point where the value of x is zero.
Curvilinear regression There are other forms of regression that are often important and needed. For example, consider the following data set (Figure x). The data in this graph are clearly not linear. We see the value in Y rise with the values in X up to a point around 4 on the X axis, and then they begin to decline.
