Source: towardsdatascience.com
Software engineering has blown up to encompass more than 1million employees in the US as of 2018 and is not forecasted to slow in growth. Next to come is the machine learning engineer, who takes an automation or decision making problem and applies cutting edge tools to it.
With the pervasive nature of machine learning (particularly deep learning) across industry, more engineers deploy these tools on a day to day basis. The list of tools that use deep learning that make companies huge profit margins is effectively endless: search recommendation, speech-to-text, voice assistants, facial recognition, advertisements, and more.
How does implementing these models differ from the roles of building vast distributed software-systems? The mindset is similar, but the specializations are different.
Software Engineering —building a data network
The data flow is the key to any at-scale software project. Engineers must choose the right algorithm to deploy on devices locally, what languages to develop in (and what language they compile into), and how many levels in the software stack.
The software engineer ultimately works in the space of language, data structures, and algorithms.
- Language: The development and test languages are the work environment of software engineers. They develop an intimate understanding of capabilities for different languages, and the tradeoffs scale drastically. Python is a favorite because the downstream decisions become so much more fluid (I agree with Python).
- Data structures: Different data structures determine which computer operations are fast — do we want fast access of data (hash table)? Fast post processing with the learning tool (Tensor)? Or something else? Different languages have different properties to leverage, and the best software engineers are fluent in these like a foreign language.
- Algorithms: Standard algorithms are the foundation of technology interviews — sort, search, and so on — because they do matter at scale. “Big O” notation is a quirky tool for learning, but the ideas translate massively when working on deployed systems.
Loving the complexity of one’s own system so that you can create more in it and show off metrics falls short when others try to use it is the downfall of a super-capable engineer. Simplicity is king because it scales and enables collaboration at the company scale.
Good software engineering ultimately will make the task of machine learning easier. The data will be more available and more uniform for distillation into products and value.
Machine Learning Engineering— building a knowledge network
Learning engineers are distilling logged knowledge (data) and creating decision boundaries. The decision boundaries are frequently nonlinear, and frequently difficult to interpret (such as a trading agent or a robot planner), but they are decision boundaries informed by data.
Machine learning engineers think in the space of Models, Deployment, and Impact.
- Models: When should I use a deep model or a Bayesian approximation? Knowing which systems generalize better, can be fine-tuned on-device, and are interpretable is the key for machine learning engineers. Also, the expertise over models is what makes ML PhD’s such valuable hires for technology companies.
- Deployment: Many companies have defined their niche in this area. Device scale artificial intelligence is the current push for consumer electronic companies (ahem, Apple) and model efficiency dominates costs of the digital goliaths. (Facebook, Google, etc). Tesla dominates the automative automation market with unmatched cloud car updates. Next is how individual engineers contribute — more specific models for specific tasks will add up in our lives, and the efficiency of models will change internet speeds and battery life.
- Impact: Ethics. Does the model I am deploying benefit a subgroup at the cost of another? This is something ML engineers need in their repertoire because the dataset you choose and train on will be reflected in your product. Consider if a dataset is collected from a sample of 100 pre-alpha users, how will that translate when it touches millions of unknowing eyes? Data transparency is behind and individuals need to be accountable.
When surveying options for implementation, other machine learners want to be able to extract and mirror useful code in a modular fashion, enabling rapid evolution. I’ve tried to utilize multiple state-of-the-art projects that were trapped in too many layers internally to take the next step and make them impactful at scale in the real world — which is why simplicity is king.
The Theme — Digital is Cheap, Simplicity is King
Both the engineering roles leverage the face that iterating in the digital domain is cheap and fast — every marginal user adds high value at a low cost. With this, the simplest methods tend to dominate because they can be so pervasive — simple methods have better generalization in learning and better interfacing in software.
The best engineering students don’t optimize within the box given to them, they look for cracks that’ll totally change the nature of the game. In software engineering, that is in the form of using new tools and data structures, in machine learning engineering that’ll be in tweaking a new model type or how it is deployed. I suspect as software engineering becomes increasingly automated, machine learning engineers will drive the best companies.
This post was inspired by a conversation on the Artificial Intelligence Podcast, with Lex Friedman hosting Andrew Ng, when discussing the impact that Massive Online Open Courses are having, how computer science is taught, and how the big tech companies dominate markets.