Using CreateML and Turi Create
Given two images, the style transfer technique allows you to transfer the style from one to the other while preserving the content.
The resulting image will be composed of the content of the first (content image) merged with the style of the second (style reference), as shown below:
In this article, we will learn how to create a machine learning model and train it to apply the style transfer technique. We will use two approaches, using CreateML and Turi Create.
To use Create ML it is necessary to have Xcode installed. Once installed, open Create ML by going to Xcode > Open Developer Tool > Create ML from the menu bar.
Click on New Document. Choose “Style Transfer” from the available templates. Click next. Choose a name for the project and click next. Choose a folder to save your project.
To create and train the model you will need:
- Training Style Image: Image containing the style to be transferred.
- Validation Image: Image that will receive the style.
- Content Images: Set of example images used to train the model.
For Content Images, we will use an image bank val2017. Download and unzip the file. Load the images into the project. Click on Train.
Be patient. Training can take hours. As the model is trained, we can track the result in our validation image.
“Turi Create simplifies the development of custom machine learning models. You don’t have to be a machine learning expert to add recommendations, object detection, image classification, image similarity or activity classification to your app.”
“Colaboratory, or “Colab” for short, is a product from Google Research. Colab allows anybody to write and execute arbitrary python code through the browser, and is especially well suited to machine learning, data analysis and education. More technically, Colab is a hosted Jupyter notebook service that requires no setup to use, while providing access free of charge to computing resources including GPUs.”
Go to Collab. On the menu bar, go to Runtime > Change runtime type. Choose GPU under Hardware Accelerator. Click Save.
Create a code block with the code below. This block will be responsible for configuring the parameters and downloading the image dataset.
image_data_set: val2017 or Selfie-dataset to be used as content images.
style_folder: folder where the Training Style Images should be saved.
test_folder: your Validation Images.
model_name: name of the .model and .mlmodel files.
Upload your training style images (you can use more than one image) and your validation images to your Colab files.
Create a second code block with the content below. We will install Turi Create and TensorFlow.
Create a third block with the code below. This is responsible for creating the model and performing the training.
The result of the trained model applied to the validation images is shown at the end of the training process.