The launch of OpenAI’s o1 series of models heralds a new approach to pursuing AI - the one that is more reliant on the test-time compute, and not completely constrained by the capabilities of the underlying foundational models.
I am not an AI expert, just studied some math, I kinda frown, but still have an open mind. I have some questions/doubts too thought. I am not going to divert into what applications and versions of AI apps can do what, again, I am not an expert in that field. However "the quality of the base models still matters" where would one base that on, how does one determine the quality of a basemodel unbiased, or even absolutely? Fundamentally, 'the' base models, a mathematical model if you wish, determining the constraints, limitations of what is possible, and the subjective choice of such model (probably still a statistical/stochastical approach) very much imposes limitations on a "foundational model" whatever that definition is/means. In my opinion, one would be very much constraint simply because of logical issues/hurdles, and severe limitations of such models chosen. One simply can not overcome basic fundamental constraints in math, logic and set theory.
On the topic of scale, I wonder if anyone really understands scale, let alone being able to determine 'laws' about scale. Over centuries, mathematicians have been driven to insanity, even to the point of becoming suicidal, just contemplating scale. So proposing a "law of scale", seems a little far fetched, ignorant even. But hey .....
"The AI trend is not slowing down, but each subsequent leap forward requires at least an order of magnitude ...." well ... that exactly has always been the problem in the last 5,6,7 iterations, in the last several decades, of AI. If we just had more speed, ram, storage, power, more sophisticated languages/models (which is a math thing), we could get there. It seems though that every step forward creates a new order of magnitude of issues, that doesnt seem like something that is helpful and can be overcome.
It is like this thing we call conjectures in math, it seems so easy to understand and within grasp, we just didn't get it quite yet. (The math way of biting of more than we can chew).
I am not an AI expert, just studied some math, I kinda frown, but still have an open mind. I have some questions/doubts too thought. I am not going to divert into what applications and versions of AI apps can do what, again, I am not an expert in that field. However "the quality of the base models still matters" where would one base that on, how does one determine the quality of a basemodel unbiased, or even absolutely? Fundamentally, 'the' base models, a mathematical model if you wish, determining the constraints, limitations of what is possible, and the subjective choice of such model (probably still a statistical/stochastical approach) very much imposes limitations on a "foundational model" whatever that definition is/means. In my opinion, one would be very much constraint simply because of logical issues/hurdles, and severe limitations of such models chosen. One simply can not overcome basic fundamental constraints in math, logic and set theory.
On the topic of scale, I wonder if anyone really understands scale, let alone being able to determine 'laws' about scale. Over centuries, mathematicians have been driven to insanity, even to the point of becoming suicidal, just contemplating scale. So proposing a "law of scale", seems a little far fetched, ignorant even. But hey .....
"The AI trend is not slowing down, but each subsequent leap forward requires at least an order of magnitude ...." well ... that exactly has always been the problem in the last 5,6,7 iterations, in the last several decades, of AI. If we just had more speed, ram, storage, power, more sophisticated languages/models (which is a math thing), we could get there. It seems though that every step forward creates a new order of magnitude of issues, that doesnt seem like something that is helpful and can be overcome.
It is like this thing we call conjectures in math, it seems so easy to understand and within grasp, we just didn't get it quite yet. (The math way of biting of more than we can chew).