Mastering Style Transfer: Transforming Images with Artistic Flair

Shikha Sharma 30 Sep, 2023 • 6 min read


A powerful means of expression is an art that captivates our senses and stirs our emotions. In this advanced era of generative artificial intelligence (AI), a new avenue has emerged to blend the realms of creativity and technology. One exciting and trending application of generative AI is style transfer, a technique that allows us to transform the visual style of an image or video. In this blog, we will explore the role of Generative AI in style transfer, explore its concept, implementation, and potential implications.

Style transfer using neural networks | AI image generator
Source: v7labs

Learning Objectives

  • Understand what style transfer is and how it combines artistic styles with content.
  • Learn to implement style transfer techniques on our own.
  • Understand the applications of style transfer in multiple industries.

This article was published as a part of the Data Science Blogathon.

Understanding Style Transfer

At its core, style transfer seeks to bridge the gap between artistic style and content. Style transfer is based on the principle of fusion, which extracts the style of one picture and applies it to another in order to combine one image’s content with another’s aesthetic qualities and generate a brand-new image. Basically, it depends upon deep learning algorithms, specifically convolutional neural networks (CNNs) to perform this style transfer process.

Style transfer using generative AI and neural networks.

Implementation: Unveiling the Magic

First, we need to explore some of the key techniques to understand the implementation of style transfer. Let’s understand the basic techniques followed by code.

Preprocessing: The input images are generated by resizing them to a desired size and normalizing their pixel values. In this preprocessing step, we need to collect and modify the input images.

Neural network architecture: A pre-trained CNN (often a VGG-19 or similar model) is used as the basis for style transfer. This network has layers that capture the image’s low-level and high-level features.

Neural network architecture in style transfer.
Source: stackoverflow

Content presentation: The content representation of the image is generated by passing the image through selected layers of her CNN and extracting feature maps. This representation captures the content of the image but ignores its particular styling.

Style expression: A technique called Gram matrix computation is used to extract the style of an image. Compute correlations between feature maps in different layers to get the statistical properties that define the style.

Style expression using Gram matrix
Source: ziqingguan

Loss function: The loss function is defined as the weighted sum of content loss, style loss, and total variation loss. Content leakage measures the difference between the input image’s content representation and the generated image’s content representation. Style leak quantifies the style mismatch between the style reference and generated images. The complete loss of variation promotes spatial smoothness in the resulting image.

The Artistic Implications

Style transfer has opened up exciting possibilities in art and design. It enables artists, photographers, and enthusiasts to experiment with different styles, pushing the boundaries of visual expression. Moreover, style transfer can serve as a tool for creative inspiration, allowing artists to explore new aesthetics and reimagine traditional art forms.

Real-World Applications

Style transfer extends beyond the realm of artistic expression. It has found practical applications in industries such as advertising, fashion, and entertainment. Brands can leverage style transfer to create visually appealing advertisements or apply different styles to clothing designs. Furthermore, the film and gaming industries can utilize style transfer to achieve unique visual effects and immersive experiences.

Ethical Considerations

As with any technological advancement, style transfer comes with ethical considerations. Simple manipulation of visual content by style transfer algorithms raises concerns about copyright infringement, misinformation, and potential abuse. As technology advances, it is important to address these concerns and establish ethical guidelines.


Simplified implementation of style transfer using the TensorFlow library in Python:

import tensorflow as tensor
import numpy as np
from PIL import Image
# Load the pre-trained VGG-19 model

vgg_model = tensor.keras.applications.VGG19(weights='imagenet', include_top=False)

# Define the layers for content and style representations

c_layers = ['b5_conv2']

s_layers = ['b1_conv1', 'b2_conv1', 'b3_conv1', 'b4_conv1', 'b5_conv1']

# Function to preprocess the input image

def preprocess_image(image_path):

    img = tensor.keras.preprocessing.image.load_img(image_path)

    img = tensor.keras.preprocessing.image.img_to_array(img)

    img = np.exp_dims(img, axis=0)

    img = tensor.keras.applications.vgg19.preprocess_input(img)

    return img

# Function to de-process the generated image

def deprocess_image(img):

    img = img.reshape((img.shape[1], img.shape[2], 3))

    img += [103.939, 116.779, 123.68]  # Undo VGG19 preprocessing

    img = np.clip(img, 0, 255).astype('uint8')

    return img

Here, we’re extracting features from intermediate layers

def get_feature_representations(model, content_img, style_img):

    content_outputs = model(content_img)

    style_outputs = model(style_img)

    content_feat = [c_layer[0] for content_layer in content_outputs[len(style_layers):]]

    style_features = [s_layer[0] for style_layer in style_outputs[:len(style_layers)]]

    return content_feat, style_features

# Function to calculate content loss

def content_loss(content_features, generated_features):

    loss = tensor.add_n([tensor.reduce_mean(tensor.square(content_features[i] -
            generated_features[i])) for i in range(len(content_features))])

    return loss

# Function to calculate style loss

def style_loss(style_features, generated_features):

    loss = tensor.add_n([tensor.reduce_mean(tensor.square(gram_matrix
           (style_features[i]) - gram_matrix(generated_features[i]))) 
            for i in range(len(style_features))])

    return loss

Function to calculate Gram matrix

def gram_matrix(input_tensor):

    result = tensor. linalg.einsum('bijc,bijd->bcd', input_tensor, input_tensor)

    input_shape = tensor.shape(input_tensor)

    num_locations = tensor.cast(input_shape[1] * input_shape[2], tensor.float32)

    return result / (num_locations)

# Function to compute total variation loss for spatial smoothness

def total_variation_loss(img):

    x_var = tensor.reduce_mean(tensor.square(img[:, :-1, :] - img[:, 1:, :]))

    y_var = tensor.reduce_mean(tensor.square(img[:-1, :, :] - img[1:, :, :]))

    loss = x_var + y_var

    return loss

# Function to perform style transfer

def style_transfer(content_image_path, style_image_path, num_iterations=1000, 
        content_weight=1e3, style_weight=1e-2, variation_weight=30):

    content_image = preprocess_image(content_image_path)

    style_image = preprocess_image(style_image_path)

    generated_image = tensor.Variable(content_image, dtype=tensor.float32)

    opt = tensor.optimizers.Adam(learning_rate=5, beta_1=0.99, epsilon=1e-1)

    for i in range(num_iterations):

        with tensor.GradientTape() as tape:

            content_features, style_features = get_feature_representations(vgg_model, 
                                               content_image, generated_image)

            content_loss_value = content_weight * content_loss(content_features, style_features)

            style_loss_value = style_weight * style_loss(style_features, generated_features)

            tv_loss_value = variation_weight * total_variation_loss(generated_image)

            total_loss = content_loss_value + style_loss_value + tv_loss_value

        gradients = tape.gradient(total_loss, generated_image)

        opt.apply_gradients([(gradients, generated_image)])

        generated_image.assign(tensor.clip_by_value(generated_image, 0.0, 255.0))

        if i % 100 == 0:

            print("Iteration:", i, "Loss:", total_loss)
            # Save the generated image

            generated_image = deprocess_image(generated_image.numpy())

            generated_image = Image.fromarray(generated_image)



To push the boundaries of creativity and imagination, Generative AI shows its potential by combining art with technology and proving the blend as a game changer. Whether as a tool for artistic expression or a catalyst for innovation, style transfer showcases the remarkable possibilities when art and AI intertwine, redefining the artistic landscape for years to come.

Key Takeaways

  • Style transfer is an exciting application of Generative AI that allows us to transform the visual style of an image or video.
  • It uses deep learning algorithms, or convolutional neural networks (CNNs), to perform this process of style transfer.
  • Brands can leverage style transfer to create visually appealing advertisements or apply different styles to clothing designs.

Frequently Asked Questions

Q1. What is style transfer?

Ans. Style transfer is a technique that combines the content of one image with the artistic style of another to get a visually appealing fusion as a result. It uses deep learning algorithms to extract and blend different images’ style and content features.

Q2. How does style transfer work?

Ans. Style transfer utilizes pre-trained convolutional neural networks (CNNs) to extract content and style representations from input images. By minimizing a loss function that balances content and style differences, the algorithm iteratively adjusts the pixel values of a generated image to achieve the desired fusion of style and content.

Q3. What are the applications of style transfer?

Ans. Style transfer has practical applications in many industries, including:
1. Advertising Industry: Style transfer helps the advertising industry create visually appealing campaigns for companies, improving brand values.
2. Fashion Industry: In the fashion industry, we can use style transfer to create new clothing designs by applying different styles that can change the clothing trend and shift from normal patterns to new and stylish clothing patterns.
3. Film and Gaming Industry: Style transfer allows the creation of unique visual effects that can help the gaming and movie industries create more VFX.

Q4. Can style transfer be applied to other forms of media beyond images?

Ans. Yes, style transfer can be extended to other forms of media like videos and music. Video style transfer involves applying the style of one video to another, while music style transfer aims to generate music in the style of a given artist or genre. These applications broaden the creative possibilities and offer unique artistic experiences.

The media shown in this article is not owned by Analytics Vidhya and is used at the Author’s discretion.

Shikha Sharma 30 Sep 2023

Frequently Asked Questions

Lorem ipsum dolor sit amet, consectetur adipiscing elit,

Responses From Readers