The paper “SmileyNet – Towards the Prediction of the Lottery by Reading Tea Leaves with AI” [1] is sometimes misunderstood as a bashing of data-driven AI, respectively of deep learning. Well, it is not. In contrary…
SmileyNet is meant to help to educate students and junior researchers. There are excellent scientific contributions that lay the foundations of data-driven AI. And there is a large potential of applying it in various domains. But there are also some (ab)use-patterns that keep on popping up over and over again. It is too easy and too tempting to use ready-to-go software packages and datasets to produce “results”, i.e., some numbers or graphs that are in some conditions slightly higher than alternative methods. While it is legitimate to go for low-hanging fruit, a substantial amount of this “research” goes for fruit that has fallen of the tree and that is already rotten.
More concretely, it is legitimate and even productive to go for the low hanging fruit of applying data-driven methods to new domains and applications where a clear advancement to the state of the art can be made. Low hanging fruit refers in this case to the fact that no majorly novel concepts and methods need to be developed. Instead, it is “only” a matter of getting the right data as well as of selecting and applying the right methods. While it can lead to meaningful results in a fast way, it also requires a proper understanding of the application domain and of the data-driven methods that may be suited for the problem.
But the rotten fruit of a mindless application of data-driven AI to whatever datasets are available is unfortunately tremendously tempting. Instead of spending time and effort on analysing the underlying problem, “results” can be easily generated by using a software library on data that is simply converted into the right format. Most of the time and energy is then devoted to get the “results” published somewhere. You might ask, so what?
First, it is a strain on the peer-reviewing system. There is a saturation of the performance level for many applications. Hence, minor modifications of existing methods get sold as “novel” and minor “improvements” in some (random) conditions get sold as performance gains. Given the easiness to generate a plethora of papers, partially as ChatGPT & co can be abused for this as well, the burden on reviewers increases and there is a reasonable chance for rotten fruits to make it through the review noise level.
Second, and imho more importantly, the temptation of the rotten fruit can lead students and junior researchers on the wrong path. Some get the illusion that they do not need to understand the phenomena that are underlying the investigated problems anymore – as data-driven methods will be the panacea for problems that previously required hard studies. This holds especially for areas like my own, i.e., robotics, where many different fields of mathematics, physics, and engineering play a role. Furthermore, “classical” methods are often enough a valid alternative, and they should at least be known and understood. It is shocking to see how often comparisons in papers are only done among different – sometimes even conceptually very similar – data-driven methods without even mentioning state-of-the-art methods from outside of “AI”, which are often enough known to perform very well on the problem without the need of training.
Long story short. Data-driven AI is a fascinating field with some exceptional academic contributions. Its methods are useful tools worth considering in many application scenarios in many domains. But this requires a proper understanding of (a) the problem to be able to find or generate the right data, (b) the AI methods that are applicable, and (c) the state of the art including “classical” methods beyond “AI” suited for the problem.
[1] A. Birk, “SmileyNet – Towards the Prediction of the Lottery by Reading Tea Leaves with AI,” arXiv, 2024. https://doi.org/10.48550/arXiv.2407.21385