- Nick Martin, Editorial Director
Machine learning isn't a new concept, and you don't need to tell that to Monte Zweben. He's been involved in artificial intelligence research for 30 years and calls himself an "old school AI person." However, recent developments and a flood of new companies offering machine learning-powered applications have made the technology more accessible than ever. Zweben has previously worked as co-manager of NASA's principal artificial intelligence laboratory and is now CEO of Splice Machine, a SQL-on-Hadoop database company in San Francisco, working on a machine learning platform.
Modern Infrastructure's Nick Martin talked with Zweben about the evolution of machine learning and how its adoption will affect businesses and IT professionals.
How has machine learning changed over the last 30 years?
Monte Zweben: The algorithms have not changed much. If you look at the algorithms embedded in Spark or utilized in Google's TensorFlow, these algorithms have been around since the 1990s or even the late 1980s. What's changed is the democratization of big data. These algorithms all do some sort of predictive classification or clustering. That means the signal they get comes from the data. In other words, the data has to have something that can group together examples that they're training the platform on. The data is critical. Until recently -- the last five or 10 years -- only certain companies, schools or governments would be able to handle the amount of data necessary to make these algorithms effective.
There weren't data platforms available that were affordable and easy to use. Suddenly, the big data architecture exploded. It started with Google's MapReduce paper and morphed into open source on Hadoop. It matured with initiatives coming out of Berkeley's AMPLab in Spark and it accelerated with cloud services like TensorFlow. Now, anybody can take a petabyte of data, build a machine learning model and deploy it. What we're trying to do is take advantage of that and build an end-to-end [machine learning platform to handle the data for the user]. You need a great engine to do machine learning, to manipulate data, prepare it and experiment with it. But the next big transformation is putting this into the seamless day-to-day operations of a company.
How can IT operations professionals get involved in and add value as companies look to adopt machine learning?
Zweben: In the past, this process has largely been about [people who had] Ph.Ds. in data science or mathematics injecting their techniques and building greenfield opportunities. I think now the world is inverting and data science and machine learning are coming out of the lab and moving into the factory.
So, [IT professionals] can inject themselves in a few ways. First, it used to be that you needed to code all of this [machine learning platform] from scratch in languages like Python or Java using a number of different data engines, and it was all done bespoke. That process really focused on developers. Now there are new data platforms that can leverage skill sets of the traditional infrastructure community, and that is behind what's convinced the founders of Splice Machine to get into the business. We wanted to democratize that computing efficiency and make it available to the DevOps community and the traditional IT pros who know how their existing infrastructure operates. You can take today's existing applications that may be running on Oracle or Teradata or IBM, and you now have the opportunity to add machine learning.
Now you can take your platform that was running on a huge engineered system and put it onto a scale-out architecture potentially running in the cloud and getting 10 times the performance at a tenth of the cost. More importantly, since it's running on this new architecture, it has access to all the new machine learning capabilities available on that architecture. You can turn antiquated applications into a modern artificial intelligence-powered application with a very limited amount of work and with a limited skill set in the underlying math.
The other interesting thing about machine learning that is happening is, as it becomes accessible to the DevOps community and traditional application developers, we'll see them begin to incorporate machine learning as a tool. Just like a programming language is a tool, there are now machine learning components that your average developer will be able to incorporate into their apps. You can change your entire traditional application just by incorporating a little machine learning.
Things will never be the same with machine learning in apps
Where IT pros can expect to find AI on the job
Data specialists discuss machine learning platforms
What you need to know to evaluate machine learning platforms