Linear regression is a method for determining the relationship between two or more related numeric variables (population characteristics) within a set of data. It is the inferential statistics method that corresponds to the use of a scatterplot for Exploratory Data Analysis.
An example of a linear regression is given in the figure above. The basic idea is that a line is fit to the data in such a way as to minimise, overall, the distances between the data points and the line. That line then represents the basic relationship between the two population characteristics graphed in the scatterplot - in this case, levels of crime and spending on police.
A statistical test can be done to determine if the slope of that line is significantly different than zero. If it is determined to be positive - that is, the line slopes upward to the right - then the two variables are said to be positively associated. If that slope is determined to be negative - that is, the line slopes downward to the right - then the two variables are said to be negatively associated. If the statistical test does not find the slope of the line to be significantly different than zero, then the two variables are not associated.
More technical information on linear regression can be found at: