piyabati

piyabati t1_iybj44z wrote

Data scarcity is a problem of methods not data.

Starting about a decade ago, cheap hardware made it possible to run vast datasets, allowing for models with more degrees of freedom. These models in turn led to the demand for massive amounts of human labeled data. It's questionable whether all this vast amount of crunching has led to an improved understanding of the world, although we now have machines that can mimic humans a lot better than they used to. The whole exercise of iterating over increasingly bigger models and bigger data, without any increase in fundamental scientific understanding, feels as pointless as bitcoin mining.

What is holding back AI/ML is to continue to define intelligence the way Turing did back in 1950 (making machines that can pass as human), and chasing big data, especially human labeled data and its attendant subjectivity and pointlessness. Essentially, we are getting hung up on local minima in the search for intelligence.

7