The core of neural-style is the pre-trained neural network. Pre-trained means that the net has been trained using a large number of images, usually with the goal that the net be able to recognize objects and features in images. The default neural network for neural-style, VGG-19 is very effective for such purposes. In addition, when used with neural-style, it is relatively easy to produce high-quality images. The only drawback is that it consumes a lot of memory. With 8 GB of memory one cannot get resolutions beyond 800 pixels.
It is now possible to switch to using another neural network in neural-style. I have tried nin-imagenet-conv, and found that I can get 1280px without running over 4 GB. The drawback is that with the default settings, the results might not be what you would expect having used VGG-19. I almost gave up myself, but then suddenly started making progress with nin-imagenet-conv. It is different, but not inferior. It may not be the best choice for copying someone’s style, but it is a great creative tool when you are looking for your own style.
In this image, I have used neural-style together with nin-imagenet-conv to convert a webcam view from Helsinki using style from a painting by Lionel Feininger.
As always, I am not merely interested in the final results of neural-style after several hundreds of iterations. I like to look at the intermediate images, to see the beauty in them, and also to use them in animations. Like in the following video clip, where the Helsinki view gradually emerges from the abstractness of the initial image.
Next, I’ll give an example of how I work using neural-style and other tools to work on original images. Let us take a photo from Finland late last autumn.
For the style image, I select an image which I have myself created, using a Processing script and webcam together with some acting in front of the camera.
Applying this style to the photo, with suitable parameters, gives me this:
Now, I might like it, the stripes in particular, but want to try with some color. I could use the original photo as a style image, but this time I’ll use another derivative of the same photo to add color. I had created this image earlier using a Processing script, webcam, some photos and neural-style.
The result, combining the grey stripes and the stylized autumn photo, currently looks like this. It may not be what I was looking for, and if not, then I can go on like this. At this point, I have used several photos, Processing scripts and neural-style runs. At each phase, I can choose the style image and in addition I have control of many parameters.
Furthermore I have the freedom to watch neural-style producing intermediate results, so if I happen to like a rougher version from an early iteration, I can use it instead of the more polished results after 1000 to 2000 iterations. Here, for example, I have used same content and style images, with different weight parameters and only 50 iterations.
Credits:
Neural-style: https://github.com/jcjohnson/neural-style
Nin-imagenet-conv: https://gist.github.com/mavenlin/d802a5849de39225bcc6
Nice post. I wish there were more people into this topic. I wrote a script for working with the nin model that creates a gradient of styles to compare numbers to, so if you are animating video you will have a good idea of which parameters to pick for the result you want. http://imgur.com/6eHsVtG
Hi, your work looks good and nicely demonstrates the effect of the different weights. I, too, have been thinking of automating the trials but have not done that so far. In addition to the parameter values, I am also experimenting with the contribution of various layers, which means that there are even more variables to experiment with. Thinking about it, I might try implementing user definable weights for the layers too.
Speaking of animations, the video in this particular post, like several others in this blog, consist of intermediate saves from a single run with neural-style (instead of making an animation of the final results of several runs using different parameters).
Pingback: Using nin-imagenet-conv in neural-style | Erratic Explorations
Hmm is anyone else having problems with the pictures on this blog loading?
I’m trying to find out if its a problem on my end
or if it’s the blog. Any responses would be
greatly appreciated.
I have myself noticed that sometimes the page is loaded with one or more images missing. I suspect that the problem in the the wordpress theme used. I have changed settings and haven’t seen the problem since.
Hannu