Linear regression is a statistical method that establishes a linear relationship between independent and dependent variables for predictive analysis, widely used in data science and machine learning. It is easy to implement, interpretable, scalable, and optimal for online settings, forming a 'best fit' line through data points to predict outcomes. The mathematical representation includes simple and multiple regression equations, which can be illustrated through programming with tools like Python.