Source – forbes.com
Machine learning (ML) is touted as the most critical skill of current times. Artificial intelligence (AI), an application of ML, is becoming pervasive. From autonomous vehicles to self-tuned databases, AI and ML are found everywhere. Industry analysts often refer to AI-driven automation as the job killer. Almost every domain and industry vertical are getting impacted by AI and ML. Platform companies with massive investments in AI research are shipping new tools and frameworks at a rapid pace.
All the above factors have put an average developer under pressure to acquire machine learning skills. There is a sudden rush to learn the tools and technologies related to ML. The number of self-paced courses and MOOCs has doubled in 2017. In emerging markets like India, there are many technical training institutes promising to transform programmers into data scientists.
Despite the interest and a sense of urgency, developers are struggling to learn the essential skills required to master ML.
Here are some of the challenges that developers need to overcome before mastering machine learning.
The Math Connection
Let’s admit it – Most of us are scared of math. Software development did not enforce the direct application of math. The availability of reusable math libraries and functions relieved developers from doing math the hard way. An average programmer doesn’t get to deal with mathematics on a day to day basis. Only a few gifted developers have the natural intuition to math.
To master ML, mathematics is mandatory. Linear algebra, statistics and probability form the foundation of machine learning. If you are a developer with serious plans to join the ML bandwagon, it’s time to brush up your high school math. It’s certainly a worthy investment.
The Need to Analyze Data
Apart from math, data analysis is the essential skill for machine learning. The ability to crunch data to derive useful insights and patterns form the foundation of ML. Like math, not every developer has the knack to play with data. Loading a large dataset, cleansing it to fill missing data, slicing and dicing the dataset to find patterns and correlation are the critical steps in data analysis.
Even if you are not a person who can instantly parse histograms, bar charts, line charts and pie charts, you need to appreciate the power of visualization. Majority of data science deals with data preparation and analysis. Start spending time with Microsoft Excel to understand Pivot Tables and various visualization techniques available as charts.
The Debate of Python vs. R vs. Julia
Developers are often caught in the debate of using Python vs. R vs. Julia for developing ML models. The choice of a language is religious and best left to an individual. But if you are a beginner willing to learn one of these languages from scratch, it may get confusing.
Python seems to be winning the battle as the preferred language of ML. The availability of libraries and open source tools make it an ideal choice for developing ML models. Though R is preferred by traditional statisticians, Python is recommended for most of the developers.
Languages like Julia are gaining popularity, but it is Python that has the best data science ecosystem.
The Fragmentation of Frameworks
Even if you are a gifted math wizard with fantastic programming skills, one of the most significant challenges is choosing the right ML framework. Today, developers will have to choose from a variety of frameworks and libraries to build ML models. There are Python modules like NumPy, Pandas, Seaborn, Scikit-Learn followed by open source toolkits such as Apache MXNet, Caffe2, Keras, Microsoft Cognitive Toolkit, TensorFlow and PyTorch. It’s often confusing to a developer on choosing the right module and toolkit.
If you are a Python developer, start with Scikit-Learn to build basic models before exploring advanced toolkits such as a Caffe2 and Keras. Most of these open source tools are meant for deep learning, which is an advanced technique of machine learning. The combination of Python and Scikit-Learn provides enough abstraction for developers to get started with the ML journey.
Multiple Approaches to Solve the Same Problem
After learning how to use the tools and modules, developers grapple with the confusion of choosing a specific algorithm to solve an ML problem. Machine learning comes with a set of predefined recipes called algorithms that are best suited for solving a particular problem.
For example, choosing between Logistic Regression and K-Nearest Neighbor algorithm can be confusing to a beginner. Like most of the branches in computer science, ML offers multiple techniques to solve the same problem.
Developers should learn the core concepts related to the algorithms and use their intuition in applying it to a given problem. In many cases, multiple algorithms are used to evaluate the precision and accuracy of a model before settling for one.
Lack of Development and Debugging Tools
The advancement in integrated development environments (IDE) enabled programmers to focus on the business problem than dealing with the configuration of the environment. Tools such as Eclipse, Microsoft Visual Studio and IntelliJ IDEA deliver out-of-the-box development and debugging experience to developers. Programmers can quickly set a breakpoint in the for loop to visualize the state of a variable that changes with every iteration. The developer experience delivered by the tools accelerated the process of shipping software.
Unfortunately, existing developer tools are not ready for machine learning. Developers switch to an entirely different toolchain for developing ML models. Though tools such as Jupyter Notebooks are robust and mature, they are fundamentally different from traditional developer tools.
Debugging an ML model is extremely hard when compared to a traditional program. Stepping through the code written to create a deep learning network is very complicated.
IDE vendors such as Microsoft are working towards making the tooling experience seamless for ML developers. But it pays off to master Jupyter Notebooks to develop interactive Python applications.
Too Many Learning Resources
The number of self-paced courses and massive open online courses (MOOC) exploded in the recent past. There are dozens of courses available for developers to learn data science and machine learning.
But the choice of these courses leads to confusion. Given how vast the ML domain is, no course is complete. The tools and frameworks are rapidly evolving making these courses outdated.
Social media and blogosphere are full of articles, tutorials and guides related to ML. The challenge with this is that most of them are incomplete leaving the essential part to the imagination of the developer.
It’s better to choose a single course or a guide to master a concept than referring to multiple resources. The overlapping and conflicting content is overwhelming and even misleading.