Skip to main content Link Search Menu Expand Document (external link)
ORML

Table of contents

  1. What can I do with it?
  2. How do I use it?
    1. Using StyleEncoder
    2. Using StyleTransformer
      1. Result
    3. Blending style vectors
  3. Example work
  4. Credits and references

orml-style-transfer

Encodes style from one image and transfers style to another image.

What can I do with it?

One can capture style from an image (style image) and encode it into a numerical representation (a style vector). This style can be applied –or transferred– to another image (content image).

Style vectors can be transformed, which may lead to interesting new styles.

How do I use it?

orml-style-transfer has two main components: StyleEncoder for the encoding of styles in an image, and StyleTransformer to transfer the encoded style to an image.

Using StyleEncoder

Load the encoder once

val encoder = StyleEncoder.load()

Encode a style image into a style vector

val styleVector: FloatArray = encoder.encodeStyle(styleImage)

Note that styleVector is a FloatArray which values can easily be changed. For example to blend between two style vectors one can

Using StyleTransformer

StyleTransformer comes in two tastes, an accurate one and a faster one.

To load the accurate version:

val transformer = StyleTransformer.load()

To load the faster version:

val transformer = StyleTransformer.loadSeparable()

To transfer style:

val transformed = transformer.transformStyle(contentImage, styleVector)

Result

Content Style Result
content image style image result image
content image style image result image

Blending style vectors

One can make blends between two style vectors to create new styles.

Consider two style vectors produced by the encoder:

val styleVector0 = encoder.encodeStyle(styleImage0)
val styleVector1 = encoder.encodeStyle(styleImage1)

We can blend them as follows:

val blendFactor = 0.5f
val styleVector = (styleVector0 zip styleVector1).map {
    it.first * blendFactor + it.second * (1.0f - blendFactor)
}.toFloatArray()

Then we use styleVector in the transformer like we’d use any style vector.

blend sequence

See BlendST01.kt for a demonstration of style blending.

Example work

Credits and references

Based on:

  • https://github.com/reiinakano/arbitrary-image-stylization-tfjs
  • https://github.com/magenta/magenta/tree/master/magenta/models/arbitrary_image_stylization

Exploring the structure of a real-time, arbitrary neural artistic stylization network. Golnaz Ghiasi, Honglak Lee, Manjunath Kudlur, Vincent Dumoulin, Jonathon Shlens, Proceedings of the British Machine Vision Conference (BMVC), 2017.