Searching for the Master Algorithm

Knowledge is power.

It may sound trite, but humanity has come to dominate the world using this tool alone. Humans lack natural weapons, have no natural protection from the elements, and enter life as helpless infants. But our unique brains allow us to acquire, use, and communicate knowledge, and this advantage alone has allowed us to create the intricate social and technological reality we now inhabit.

Our brains evolved to process, store, retrieve, and integrate sensory data into working knowledge that allows us to navigate reality. Until recently, humans were the only significant force that could translate raw data into accurate, actionable knowledge.

But a new force has recently entered this arena: the computer.

Still in its infancy, Machine Learning is a set of techniques designed automate learning by using computers to transform data into knowledge. Because computers can process data more quickly than human nervous systems, and can be scaled to enormous size, they can parse enormous amounts of data and discover new knowledge much faster than their human counterparts. Pedro Domingo presents these concepts in his “Searching for the Master Algorithm” lecture, in which he explores the emerging techniques that allow computers to make sense of data and generate new knowledge.

The Internet of Things (IoT) is one major way that these Machine Learning models are impacting our world. Machine Learning and IoT are creating a new networked intelligence that can understand events in the real world, form an understanding of how those events are organized, and respond appropriately.

Machine Learning and IoT offer limitless possibilities for innovation in business. They allow companies to offer previously unimaginable products, automate dizzyingly complex processes, identify and prevent waste, and achieve a deep understanding of customer behavior.

Understanding these new capabilities will be essential to navigating the business environment of the future.

The Five Tribes of Machine Learning

Pedro Domingo identifies five tribes of machine learning that use their own master algorithm to make sense of data. Each tribe has a unique origin in the human search for knowledge, and takes its own approach to transforming data into useful knowledge.

Reference: Pedro Domingos, The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World

We’ll focus on the three tribes that have the immediate application to IoT & Machine Learning scenarios in business: Symbolists, Connectionists, and Bayesians.

Symbolists

Symbolist algorithms use inverse deduction to solve problems and generate new knowledge.

Deduction is the process of deriving specific facts from general principles.

Deduction:

1. General Principle: Humans are mortal
2. General Principle: Socrates is a human3. Specific Fact: Socrates is mortal

Inverse deduction, or Induction, does the opposite: it derives general principles from a set of specific facts. By observing thousands of specific facts in which humans die, a symbolist algorithm might derive the general principle “Humans are mortal.”

Inverse Deduction (Induction):

1. Specific Fact: Socrates is human
2. Specific Fact: Socrates is mortal
3. Specific Fact: Plato is human
4. Specific Fact: Plato is mortal
5. Specific Fact: Aristotle is human
6. Specific Fact: Aristotle is mortal
[…]
7. General Principle: Humans are mortal

This technique is common in commercial IoT & Machine Learning applications.

For example, a manufacturing operation might feed operational data from its machines and process monitoring systems into a machine learning model, which would parse thousands of events to determine the general patterns that predict defects. The Machine Learning model derives the general truth of what causes a defect from the specific facts of sensor values and defect markers.

These types of models can drive immense business value, allowing companies to make sense out of data too large and complex for humans to parse. Symbolic models also become more accurate over time, as more and more facts can be parsed, producing an increasingly accurate model over time.

Connectionists

Another principle methodology used in advanced analytics and machine learning is Connectionism, which builds models that emulate the human brain.

Based in neuroscience, this method involves creating a network of simulated neurons, each of which receives input from other neurons, makes an evaluation and either fires or stays silent. As described in Kurzweil’s “pattern recognition theory of mind” (How to Create a Mind), each neuron is responsible for a discrete, binary evaluation i.e. [“this object is an apple”], based on evaluations from many lower-level neurons in a nested hierarchy i.e. [“this is red”, “this is round”, “this is shiny”, “this is a fruit”], each of which has a different weight in influencing the evaluation.

Reference: Pedro Domingos, The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World

As the system operates, successful connections are strengthened over time, while unsuccessful connections are weakened and trimmed, resulting in adaptations over time. According to Hebbian theory, this is the same basic process by which our brains learn from experience.

In Connectionist models, systematic learning in neural networks involves adjusting the weights of input connections based on experimental results, an algorithm called “Backpropagation”.

Tools like Azure Machine Learning allow us to create neural nets that operate in a similar manner and can be applied to commercial applications. Neural networks excel at pattern recognition tasks such as facial and optical character recognition, and have been applied in a wide variety of contexts.

Bayesians:

Bayesian analysis start from the simple premise that all knowledge is uncertain, and Bayesian models focus on quantifying and reducing uncertainty. Bayesians view probabilities as subjective beliefs about the likelihood of a future event, subject to modification as new evidence comes to light.

Bayesian models begin with a hypothesis (i.e. “the product is defective”), and assign a Prior – this is how much you believe in the hypothesis before you collect any evidence.

As you collect data, you update the prior by using probabilistic inference to assess how probable the hypothesis is given the observed data (i.e. “how likely is this product to be defective, given the measured sensor values”).

Bayesian models develop over time as more evidence is collected, and Posteriors from prior experiments become Prior inputs to later models.

Cloud technologies like Azure Machine Learning support the heavy computation needed to run and evaluate Bayesian models repeatedly, allowing them to become more accurate as they consume more and more data through successive iteration.

Algorithms in Motion: Applying IoT & Machine Learning

Understanding how Machine Learning algorithms work is essential to finding the right model for your business problem, and informs every aspect of the data science process and every component of a production solution.

Powerful technologies like Machine Learning & IoT require hard thinking about the nature of a problem, incisive analysis of the available data, and the right algorithm to bring machine intelligence to bear.

Platforms like Microsoft Azure make deploying Machine Learning & IoT solutions faster and easier than ever, but successful solutions are ultimately built on the human intelligence behind the technology.

Next week, the IoT feature blog will focus on the commoditization of machine learning.