Introduction to Machine Learning

Learning from observations

A lot of machine learning can look like an enormous collection of algorithms and formulas when you first encounter it. I don't think that's the most interesting way to understand it. Beneath those methods is a shared difficulty: the observations available to us can usually be explained in many different ways. Fitting them is only part of the problem, because the explanation we choose also determines what we expect to happen in situations we haven't observed.

To make that choice, we bring assumptions about the world into our models. Even fitting a straight line expresses a preference for one kind of explanation over countless others that might agree with the data. This is the beginning of inductive bias, and following it through more elaborate models gives the subject a sense of continuity. As the mathematics develops, it helps us make those assumptions precise enough to examine, question, and sometimes revise.

Recitations

Connections to research and practice

Part of the pleasure of learning these foundations is recognizing them in work that initially seems far beyond an introductory course. In linear regression, choosing the features means deciding which aspects of an observation the model can use. Deep learning takes that decision further by allowing representations themselves to be learned, which brings us back to a familiar concern: what in the learning problem encourages a useful representation to emerge?

Language offers a way into that question. A model asked to recover a missing word has to use the surrounding sentence, so ordinary text can supply both the evidence and the target for learning. This is one route into self-supervised learning, where the interest extends beyond whether the missing word was guessed correctly to what the model learned in order to guess it. Applying a similar idea to biological sequences opens a connection to protein modeling and genomics: could the patterns learned from sequences help answer questions about their structure or function? The hope is to make unfamiliar questions feel within reach.