Deep learning recently returned to the headlines when Google's AlphaGo program crushed Lee Sedol, one of the highest-ranking Go players in the word. Google has invested heavily in deep learning and AlphaGo is just their latest deep learning project to make the news. Google’s search engine, voice recognition system and self-driving cars all rely heavily on deep learning. They’ve used deep learning networks to build a program that picks out an attractive still from a YouTube video to use as a thumbnail. Late last year Google announced Smart Reply, a deep learning network that writes short email responses for you. Deep learning is clearly powerful, but it also may seem somewhat mysterious. What is deep learning and how can it be useful to you if you’re not Google?
Although the terms are sometimes used synonymously, deep learning and machine learning are not the same thing. Deep learning is a kind of machine learning just as cycling is a kind of exercise. Understanding deep learning is easier if you have a basic idea of what machine learning is all about.
Machine learning is often described as a type of AI where computers learn to do something without being programmed to do it. To take a simple example, you could program a computer to identify an animal as a cat by writing code that tells the program to say “cat” when it sees a picture of a particular cat. This works well if the only cat the program will ever see is the cat in the picture. It doesn’t work very well if the program is going to see a lot of different animals, including a lot of different cats, and it has to pick out the cats from all of the other animals.
Machine learning programs can be trained in a number of different ways. In one type of training, the program is shown a lot of pictures of different animals and each picture is labeled with the name of the animal; the cats are all labeled “cat”. The program will eventually learn that the animals that look like cats are called “cats” without ever being programmed to call a picture of a cat a “cat”.
The program does this by learning combinations of features that tend to appear together. Cats have visual features – such as their body shape, long whiskers, and the way their faces look – that make them visually different from other animals. The program learns to associate this distinctive combination of features with the word “cat”. This learning process is usually called constructing a model of a cat.
Once it has constructed the cat model, a machine learning program tests the model by trying to identify the cats in a set of pictures it hasn’t seen before. The program measures how well it did at identifying the new cats and uses this information to adjust the model so it will do a better job of picking out cats the next time it tries. The new model is then tested, its performance is evaluated, and it receives another adjustment. This iterative process continues until the program has built a model that can identify cats with a high level of accuracy.
Deep learning carries out the machine learning process using an artificial neural net that is composed of a number of levels arranged in a hierarchy. The network learns something simple at the initial level in the hierarchy and then sends this information to the next level. The next level takes this simple information, combines it into something that is a bit more complex, and passes it on the the third level. This process continues as each level in the hierarchy builds something more complex from the input it received from the previous level.
Continuing the cat example, the initial level of a deep learning network might use differences in the light and dark areas of an image to learn where edges or lines are in a picture of a cat. The initial level passes this information about edges to the second level which combines the edges into simple shapes like a diagonal line or a right angle. The third level combines the simple shapes into more complex objects likes ovals or rectangles. The next level might combine the ovals and rectangles into rudimentary whiskers, paws and tails. The process continues until it reaches the top level in the hierarchy where the network has learned to identify cats. While it was learning about cats, the network also learned to identify all of the other animals it saw along with the cats.
Why deep learning is useful
Deep learning has attracted a lot of attention because it is particularly good at a type of learning that has the potential to be very useful for real-world applications. The earlier introduction to machine learning described a training method in which all the pictures that are used to train the program are labeled with the name of the thing in the picture. In the cat example, the pictures of cats are all labeled “cat”. Each iterative step in testing and refining the model involves comparing the label on a picture with the label the program assigned to the picture to determine whether the program labeled the picture correctly. This method of training is called supervised learning.
Supervised learning is relatively fast and demands relatively less computational power than some other training techniques that are used in machine learning. It has an important drawback for real-world applications, however. An immense amount of information about people is gathered everyday from social media, hardware and software service agreements, app permissions and website cookies. This information has the potential to be very valuable to businesses at all levels. The problem is that all of this data is unlabeled and can’t be used to train machine learning programs that depend on supervised learning. A person is needed to label the data and the labeling process is time-consuming and expensive.
Deep learning networks can avoid this drawback because they excel at unsupervised learning. The key difference between supervised and unsupervised learning is that the data are not labeled in unsupervised learning. Even though the pictures of cats don’t come with the label “cat”, deep learning networks will still learn to identify the cats. Other forms of machine learning are not nearly as successful with unsupervised learning.
The ability to learn from unlabeled or unstructured data is an enormous benefit for those interested in real-world applications. Deep learning unlocks the treasure trove of unstructured big data for those with the imagination to use it .