SECRET OF CSS

Three Ways to Build Machine Learning Models in Keras


If you’ve looked at Keras models on Github, you’ve probably noticed that there are some different ways to create models in Keras. There’s the Sequential model which allows you to define an entire model in a single line, usually with some line breaks for readability, then there’s the functional interface that allows for more complicated model architectures, and there’s also the Model subclass which helps reusability. In this article, we’re going to explore the different ways to create models in Keras, along with their advantages and drawbacks to equip you with the knowledge you need to create your own machine learning models in Keras.

After you completing this tutorial, you will learn:

  • Different ways that Keras offers to build models
  • How to use the Sequential class, functional interface, and subclassing keras.Model to build Keras models
  • When to use the different methods to create Keras models

Let’s get started!

mike szczepanski MnSYYYqunXA unsplash

Three Ways to Build Machine Learning Models in Keras
Photo by Mike Szczepanski. Some rights reserved.

Overview

This tutorial is split into 3 parts, covering the different ways to building machine learning models in Keras:

  • Using the Sequential class
  • Using Keras’ functional interface
  • Subclassing keras.Model

Using the Sequential class

The Sequential Model is just as the name implies. It consists of a sequence of layers, one after the other. From the Keras documentation,

“A Sequential model is appropriate for a plain stack of layers where each layer has exactly one input tensor and one output tensor.”

It is a simple, easy-to-use way to get started building your Keras model. To start, import the Tensorflow, and then the Sequential model:

Then, we can start building our machine learning model by stacking various layers together. For our example, let’s build a LeNet5 model with the classic CIFAR-10 image dataset as the input:

Notice that we are just passing in an array of the layers that we want our model to contain into the Sequential model constructor. Looking at model.summary(), we can see the model’s architecture.

And just to test out the model, let’s go ahead and load the CIFAR-10 dataset and run model.compile and model.fit:

which gives us this output.

That’s pretty good for a first pass at a model. Putting the code for LeNet5 using a Sequential model together,

Now, let’s explore what the other ways of constructing Keras models can do, starting with the functional interface!

Using Keras’ Functional Interface

The next method of constructing Keras models that we’ll be exploring is using Keras’ functional interface. The functional interface uses the layers as functions instead, taking in a Tensor and outputting a Tensor as well. The functional interface is a more flexible way of representing a Keras model as we are not restricted only to sequential models that have layers stacked on top of one another. Instead, we can build models that branch into multiple paths, have multiple inputs, etc.

Consider an Add layer that takes inputs from two or more paths and adds the tensors together.

add layer

Add layer with two inputs

Since this cannot be represented as a linear stack of layers due to the multiple inputs, we would be unable to define it using a Sequential object. Here’s where Keras’ functional interface comes in. We can define an Add layer with two input tensors as such:

Now that we’ve seen a quick example of the functional interface, let’s take a look at what the LeNet5 model that we defined by instantiating a Sequential class would look like using a functional interface.

And looking at the model summary,

As we can see, the model architecture is the same for both LeNet5 models that we have implemented using the functional interface or the Sequential class.

Now that we’ve seen how to use Keras’ functional interface, let’s look at a model architecture that we can implement using the functional interface but not with the Sequential class. For this example, we’ll look at the residual block introduced in ResNet. Visually, the residual block looks like this:

residual block

Residual block, source: https://arxiv.org/pdf/1512.03385.pdf

We can see that a model defined using the Sequential class would be unable to construct such a block due to the skip connection which prevents this block from being represented as a simple stack of layers. Using the functional interface, this is one way we can define a ResNet block:

Then, we can build a simple network using these residual blocks using the functional interface as well.

Running this code and looking at the model summary and training results,

And combining the code for our simple network using residual blocks,

Subclassing keras.Model

Keras also provides an object-oriented approach to creating models, which would help with reusability and allows us to represent the models that we want to create as classes. This representation might be more intuitive, since we can think about models as a set of layers strung together to form our network.

To begin subclassing keras.Model, we first need to import it.

Then, we can start subclassing Model. First, we need to build the layers that we want to use in our method calls since we only want to instantiate these layers once instead of each time we call our model. To keep in line with previous examples, let’s build a LeNet5 model here as well.

Then, we override the call method to define what happens when the model is called. We override it with our model which uses the layers that we have built in the initializer.

It is important to have all the layers created at the class constructor, not inside the call() method. It is because the call() method will be invoked multiple times with different input tensor. But we want to use the same layer objects in each call so we can optimize their weight. We can then instantiate our new LeNet5 class and use it as part of a model:

And we can see that the model has the same number of parameters as the previous two versions of LeNet5 that we built previously and has the same structure within it as well.

Combining all of the code to create our LeNet5 subclass of keras.Model,

Further Reading

This section provides more resources on the topic if you are looking to go deeper.

Papers:

APIs:

Summary

In this post, you have seen three different ways to create models in Keras, in particular, using the Sequential class, functional interface and subclassing keras.Model. You have also seen examples of the same LeNet5 model being built using the different methods and seen a use case which can be done using the functional interface but not with the Sequential class.

Specifically, you learned:

  • Different ways that Keras offers to build models
  • How to use the Sequential class, functional interface, and subclassing keras.Model to build Keras models
  • When to use the different methods to create Keras models

Develop Deep Learning Projects with Python!

Deep Learning with Python

 What If You Could Develop A Network in Minutes

…with just a few lines of Python

Discover how in my new Ebook:

Deep Learning With Python

It covers end-to-end projects on topics like:

Multilayer PerceptronsConvolutional Nets and Recurrent Neural Nets, and more…

Finally Bring Deep Learning To

Your Own Projects

Skip the Academics. Just Results.

See What’s Inside



News Credit

%d bloggers like this: