Why do so many companies, presumably on the basis of rational decisions, limit themselves simply to exploring the potential of machine learning, and even after undertaking large investments, hiring data scientists and investing resources, time and money, fail to take things to the next level?
Quite simply, an inbuilt experimental mindset. For years, we have decided that machine learning, which is really a discipline dating back many decades that simply stopped progressing for a while until technology caught up, required teams of data scientists armed with programming languages such as Python, R who would develop ad hoc tools to carry out the complex analysis necessary to design and educate those mythical algorithms. The whole thing was seen as an experiment. Even today, it doesn’t matter who you consult, whether it’s the extremely popular course by Andrew Ng, or “Machine learning for average humans” or even “Absolute beginning into machine learning,” you’ll be told you need to learn to program and then relearn statistics as though we were starting from scratch, when in fact the tools have been around for years.
Would anyone think of hiring software engineers to develop a tool to keep their company’s accounts? Of course not. Instead, businesses choose an accounting program and use it. The only difference between accounting and machine learning is the raw material they use: in general, the accounting data we feed into our accounts is readily available, are calculated in a reasonably standardized manner and generate no doubts about their origin. And yet the data we feed our machine learning analysis is often more difficult to locate or prepare. What’s really going on is that we have a data culture problem and so we need to inculcate our workforces about the importance of data, of reinterpreting our value chain in order to obtain data we previously ignored. If we have the data, analyzing it through machine learning should simply be a matter of using the right tools for the job. If instead of simply using those tools we instead spend our time trying to invent them, our projects will never get off the ground.
If the advice you’re given is that launching a machine learning project in your company will require you to hire one or more data scientists and write down millions of programs in Python or R, stop and rethink the whole thing with people who really know what they are talking about, otherwise you will end up trying to reinvent the wheel and failing miserably, because you won’t have the right tools for the job. The chances of such experiments actually going into production, which is the only valid metric for evaluating them, are as rare as the 13% mentioned above. In other words, you are 87% likely to waste your time, effort and money. It’s a losing game.
Machine learning has long since passed the experimental phase, is now MLaaS (Machine Learning as a Service) and is quickly entering the commodity phase. Please, bear that in mind: if those who want to start a machine learning project in your company ignore that and try to return to the experimental phase, ignore them, or better still, ask them if they think you need to bring in engineers to develop a spreadsheet. The real point here is that somewhere, one of your competitors is already using standard tools for doing these things, and they are moving much faster than your company.
Don’t be fooled: applying machine learning is not easy: we’re talking about complex analytical procedures in which the phases of defining objectives and data collection and transformation will take a very high percentage of the project effort. They are not projects to be taken lightly. But neither are they overly complex, nor do they require experts to build experimental analytical tools, because those analytical tools have been around for a long, long time.
If we abandon this absurd experimental mentality toward machine learning, which is a by-product of ignorance and fear, we will make much faster progress.