MIT6036L01g

okay so here's our enterprise then we have a data set we're interested for right now on finding hypothesis that we'll have good training center okay so that's gonna be the job that we try to do and so that's the job of a learning algorithm so now let me write what a learning algorithm is right so this is a hypothesis that's not a learning algorithm a learning algorithm is a different kind of a creature so a learning algorithm you can think of it as a box what comes in is a data set and what comes out is a hypothesis right and you know you can think of this as a learning algorithm and maybe you can think of it as potentially being parameterised by a class of possible hypotheses it could generate so learning algorithm that's the computer program that we're gonna write right we're gonna we're not gonna write down hypotheses directly we're gonna write down algorithms that consume datasets and give out hypotheses so that's that's a learning algorithm consume a dataset give out a hypothesis and what we'd like to do the way we would like to frame one way to so the question is how do we come up with learning algorithms actually so this is a good good thing how do we come over learning algorithms so there's kind of two strategies and one is to be a clever human which many of us are and the other one is to use optimization methods so the first algorithm that we're going to talk about a couple algorithms today actually one is Dom Oh you can also come up with that algorithm so you can be just you could be dumb so that's good so we're gonna today okay awesome today we're gonna first of all think about an algorithm that's dumb my favorite thing would give it a problem is to come up with a really dumb algorithm it's a good idea it helps you at least be sure that you understand your problem even if you don't know the answer yet so we're gonna pull the dumb algorithm and then we're going to look at an algorithm are very clever human invented just kind of directly but after that basically all of the ways that we think about solving machine learning problems coming up with machine learning algorithms is going to be actually to use optimization methods right so what we're gonna do is write down an objective function like the training error come up with a hypothesis class and then use computational optimization methods to find to try to find the best element of the hypothesis class according to our objective function so that's really that kind of this the standard method and it's what that's where I feel like there's some kind of like a boundary a machine learning person has to be partly a kind of a statistical analysis and generalization person and partly a computer scientist right and it's up and to the point that you frame it as an optimization problem that you're really kind of worrying about the the statistical and generalization aspects and once you frame your questions as an optimization problem it becomes a problem in numerical algorithms and so and sort of then your role changes a little bit and you become an optimization person now of course that boundary is not rigid and generally the best practitioners of machine learning are mindful of the kinds of optimization problems we know how to solve and they work at trying to formulate their machine learning problem as an optimization that we have good methods for or they invent new methods to solve those problems that they come up but that's a kind of a way to think about the the overall problem that we try to address