Definition of best straight line

Answer 1

In statistics, if you are given a set of observations for two variable, X and Y, and if you can assume that the relationship between the two variables is linear, the [straight] line of best fit is a line such that it minimises the sum of squares of the vertical distances between the line and the points representing the observations.

That last was a bit of a mouthful so let us take it bit by bit.

The line shows what the value of Y should be for each value of X. Therefore, the vertical distance between the point representing an observation and the line as the difference between the observed value and what it should have been under the assumption of a linear relationship.

The aim is to find a line that will reduce all these differences or errors - or some measure of their sum. If you then assume that an underestimate is just as bad as an equivalent overestimate, you don't simply want to minimise the sum but either the absolute value of the sum or the sum of the squares of the differences. The latter, for a variety of statistical reasons, is the preferred option.