{"id":500,"date":"2017-04-04T13:17:42","date_gmt":"2017-04-04T13:17:42","guid":{"rendered":"http:\/\/liipetti.net\/erratic\/?p=500"},"modified":"2018-05-06T09:41:37","modified_gmt":"2018-05-06T09:41:37","slug":"learning-multiple-styles","status":"publish","type":"post","link":"https:\/\/liipetti.net\/erratic\/2017\/04\/04\/learning-multiple-styles\/","title":{"rendered":"Learning multiple styles at the same time?"},"content":{"rendered":"

New innovative GAN implementations are published every week. A particularly interesting one is CycleGAN<\/a>, which promises to learn to do transformations, similar to\u00a0what pix2pix<\/a> does, but without needing pairs of corresponding images. In other words, it claims to be able to transform horses to zebras using a training set consisting of a) any horses and b) any zebras.\u00a0Also it also appears to manage transform a painting by Monet into a photo image.<\/p>\n

As the task of assembling a dataset of matching image pairs is often huge, being able to compose the datasets more freely feels a great advance. I was tempted to try what it can achieve if given a set of modern paintings of greatly varying style. So I assembled a set of 2000+ images of modern works of art, mostly but not exclusively paintings, and trained a CycleGAN using my own landscape photos as the other set, with the goal of seeing what this GAN would make of my photos. Will it make them look like art, and if so, in what style.<\/p>\n

The results are interesting, and many of them have the look of\u00a0works of art. Moreover, the style varies from image to image. One is tempted to think that the GAN has succeeded to match each image to those paintings which are most suitable as style models. Yet I am hesitant to make such a claim.<\/p>\n

The results as a whole do not look so modern as the dataset did. In particular, the abstract works seem to have had no visible effect. This is understandable, because even the authors of CycleGAN write that it performs well with transformations of color and texture, but fails when expected to transform larger structures (such as shape, I assume). This appears to be a general handicap with neural network based image transforms.<\/p>\n

\"Image079\"<\/a> \"Image087\"<\/a> \"IMG_0323\"<\/a> \"IMG_2809\"<\/a> \"IMG_2856\"<\/a> \"IMG_4054\"<\/a> \"k0709-115\"<\/a> \"k2006-491\"<\/a> \"k2006-559\"<\/a> \"m09-081\"<\/a> \"m09-129\"<\/a> \"mar05-0144\"<\/a> \"mar05-0204\"<\/a> \"Norge04b-162\"<\/a> \"resa03-0305\"<\/a> \"st2007-124\"<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

New innovative GAN implementations are published every week. A particularly interesting one is CycleGAN, which promises to learn to do transformations, similar to\u00a0what pix2pix does, but without needing pairs of corresponding images. In other words, it claims to be able to transform horses to zebras using a training set consisting \u2026<\/p>\n

Continue reading<\/i><\/a><\/p>\n","protected":false},"author":1,"featured_media":504,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2,3],"tags":[],"class_list":["post-500","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-art","category-neural-networks"],"_links":{"self":[{"href":"https:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/posts\/500","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/comments?post=500"}],"version-history":[{"count":2,"href":"https:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/posts\/500\/revisions"}],"predecessor-version":[{"id":518,"href":"https:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/posts\/500\/revisions\/518"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/media\/504"}],"wp:attachment":[{"href":"https:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/media?parent=500"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/categories?post=500"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/tags?post=500"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}