Image Style Transfer CoreML Model Training – Create ML and Turi Create on Colab.

Using CreateML and Turi Create

Photo by CHUTTERSNAP on Unsplash
1*aGZVO rXySjSvFdbgGt oQ
Content Image. Yellow Labrador Looking, from Wikimedia Commons by Elf. License CC BY-SA 3.0

Given two images, the style transfer technique allows you to transfer the style from one to the other while preserving the content.

1*aGZVO rXySjSvFdbgGt oQ
Content Image. Yellow Labrador Looking, from Wikimedia Commons by Elf. License CC BY-SA 3.0

The resulting image will be composed of the content of the first (content image) merged with the style of the second (style reference), as shown below:

1*Ih7JPhN9hw29cTHY8zR wg
Style reference.

In this article, we will learn how to create a machine learning model and train it to apply the style transfer technique. We will use two approaches, using CreateML and Turi Create.

To use Create ML it is necessary to have Xcode installed. Once installed, open Create ML by going to Xcode > Open Developer Tool > Create ML from the menu bar.

1*SCzcCzHjKc1UYXwOgoO qg
Xcode menu bar.

Click on New Document. Choose “Style Transfer” from the available templates. Click next. Choose a name for the project and click next. Choose a folder to save your project.

Empty Core ML Style Transfer project

To create and train the model you will need:

  • Training Style Image: Image containing the style to be transferred.
  • Validation Image: Image that will receive the style.
  • Content Images: Set of example images used to train the model.

For Content Images, we will use an image bank val2017. Download and unzip the file. Load the images into the project. Click on Train.

Training Style, Validation and Content images loaded on Create ML project.

Be patient. Training can take hours. As the model is trained, we can track the result in our validation image.

Create ML training process

We can use Turi Create and Colab to speed up the creation of style transfer models.

“Turi Create simplifies the development of custom machine learning models. You don’t have to be a machine learning expert to add recommendations, object detection, image classification, image similarity or activity classification to your app.”

“Colaboratory, or “Colab” for short, is a product from Google Research. Colab allows anybody to write and execute arbitrary python code through the browser, and is especially well suited to machine learning, data analysis and education. More technically, Colab is a hosted Jupyter notebook service that requires no setup to use, while providing access free of charge to computing resources including GPUs.”

Go to Collab. On the menu bar, go to Runtime > Change runtime type. Choose GPU under Hardware Accelerator. Click Save.

1*MyyWkzoH0 FJNiSWTQy tQ
Colab runtime type

Create a code block with the code below. This block will be responsible for configuring the parameters and downloading the image dataset.

  • image_data_set: val2017 or Selfie-dataset to be used as content images.
  • style_folder: folder where the Training Style Images should be saved.
  • test_folder: your Validation Images.
  • model_name: name of the .model and .mlmodel files.

Upload your training style images (you can use more than one image) and your validation images to your Colab files.

1*IIrS XRpNjApbVGo30gOEQ
Colab files

Create a second code block with the content below. We will install Turi Create and TensorFlow.

Create a third block with the code below. This is responsible for creating the model and performing the training.

The result of the trained model applied to the validation images is shown at the end of the training process.

1*sh60z lZuY5dldHbJDj3eA
Validation image


Full colab notebook

News Credit

%d bloggers like this: