{"id":55,"date":"2015-10-14T09:20:45","date_gmt":"2015-10-14T09:20:45","guid":{"rendered":"http:\/\/liipetti.net\/erratic\/?p=55"},"modified":"2018-05-07T12:18:40","modified_gmt":"2018-05-07T12:18:40","slug":"responsive-portraits-in-real-time","status":"publish","type":"post","link":"http:\/\/liipetti.net\/erratic\/2015\/10\/14\/responsive-portraits-in-real-time\/","title":{"rendered":"Responsive Portraits in Real-time"},"content":{"rendered":"<p>As we have seen, neural networks can create great\u00a0pictures, but the process is still too slow to be applied in real-time or to implement responsive applications in which the user can affect the resulting image while it is being generated.<\/p>\n<p>The Processing framework and programming language shines in such applications. Here I will present a responsive portrait generator, implemented in Processing.<\/p>\n<p>This Processing &#8220;sketch&#8221;, as programs are called in Processing, is using a webcam to continuously view the view to be painted. At the same time it is painting a picture, in random strokes that take their color from the webcam view. In addition, the sketch recognized if there is a face in the view, and paints the face in finer strokes.<\/p>\n<p><a href=\"http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00000198.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-60\" src=\"http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00000037-300x225.jpg\" alt=\"00000037\" width=\"300\" height=\"225\" srcset=\"http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00000037-300x225.jpg 300w, http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00000037-200x150.jpg 200w, http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00000037-150x113.jpg 150w, http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00000037.jpg 640w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/>\u00a0\u00a0<img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-66\" src=\"http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00000198-300x225.jpg\" alt=\"00000198\" width=\"300\" height=\"225\" srcset=\"http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00000198-300x225.jpg 300w, http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00000198-200x150.jpg 200w, http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00000198-150x113.jpg 150w, http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00000198.jpg 640w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>The output\u00a0image is continuously updated on the screen. It will take some time for the sketch to paint a complete picture. Then, if the subject moves, or if the camera is moved, it will again take some time before the complete picture is updated. Therefore, during some time images as if of a broken world will be generated, until finally, if the subject and the camera keep still, a harmonious image will be reached.<\/p>\n<p><a href=\"http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00001776.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-61\" src=\"http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00001776-300x225.jpg\" alt=\"00001776\" width=\"300\" height=\"225\" srcset=\"http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00001776-300x225.jpg 300w, http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00001776-200x150.jpg 200w, http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00001776-150x113.jpg 150w, http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00001776.jpg 640w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a> <a href=\"http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00001814.jpg\"><img loading=\"lazy\" decoding=\"async\" class=\"alignnone size-medium wp-image-64\" src=\"http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00001814-300x225.jpg\" alt=\"00001814\" width=\"300\" height=\"225\" srcset=\"http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00001814-300x225.jpg 300w, http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00001814-200x150.jpg 200w, http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00001814-150x113.jpg 150w, http:\/\/liipetti.net\/erratic\/wp-content\/uploads\/2015\/10\/00001814.jpg 640w\" sizes=\"auto, (max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>Here we can see the portrait generator in action. The sketch is running, people come into the webcam view to have their portrait painted. Then this is what happens.<\/p>\n<p><iframe loading=\"lazy\" width=\"560\" height=\"315\" src=\"https:\/\/www.youtube.com\/embed\/JoW1WS575dQ?feature=oembed\" frameborder=\"0\" allow=\"autoplay; encrypted-media\" allowfullscreen><\/iframe><\/p>\n<p>And here we can see some images painted by the portrait generator, broken and whole.<\/p>\n<p><iframe loading=\"lazy\" width=\"560\" height=\"315\" src=\"https:\/\/www.youtube.com\/embed\/_fGUd6bWIr8?feature=oembed\" frameborder=\"0\" allow=\"autoplay; encrypted-media\" allowfullscreen><\/iframe><\/p>\n<p>Finally, here is the source code of the sketch:<\/p>\n<pre>import gab.opencv.*;\r\nimport processing.video.*;\r\nimport java.awt.*;\r\n\r\nCapture video;\r\nOpenCV opencv;\r\nPImage img ;\r\nfloat x;\r\nfloat y;\r\nfloat cx;\r\nfloat cy;\r\n\r\nvoid setup() {\r\n  size(640, 480);\r\n  img = new PImage(width,height) ;\r\n  video = new Capture(this, 640, 480);\r\n  opencv = new OpenCV(this, 640, 480);\r\n  opencv.loadCascade(OpenCV.CASCADE_FRONTALFACE);  \r\n \/\/ Start x and y in the center\r\n  x = width\/2;\r\n  y = height\/2;\r\n  cx = x ;\r\n  cy = y ;\r\n  video.start();\r\n}\r\n\r\nvoid draw() {\r\n  if (video.available()) {\r\n    video.read() ;\r\n    opencv.loadImage(video);\r\n  }\r\n\r\n  pushMatrix() ;\r\n  scale(-1.0,1.0) ;\r\n  popMatrix() ;\r\n\r\n  video.loadPixels() ;    \r\n\r\n  Rectangle[] faces = opencv.detect();\r\n  \r\n  if (faces.length &gt; 0) {\r\n    cx = faces[0].x + faces[0].width\/2 ;\r\n    cy = faces[0].y + faces[0].height\/2;\r\n    cx = width - cx ;\r\n    }  \r\n    \r\n  video.loadPixels();\r\n  \r\n  for (int k=0; k&lt;2080; k++) {\r\n    float d = dist(x, y, cx, cy) ;\r\n    float w = constrain(d \/30, 2, 32) ;\r\n    float maxlen = constrain(d\/30, 4, 32) ;\r\n    \r\n    \/\/ Pick a new x and y\r\n    float newx = constrain(x + random(-maxlen,maxlen),0,width-1);\r\n    float newy = constrain(y + random(-maxlen,maxlen),0,height-1);\r\n  \r\n    \/\/ Find the midpoint of the line\r\n    int midx = int((newx + x) \/ 2);\r\n    int midy = int((newy + y) \/ 2);\r\n  \r\n    \/\/ Pick the color from the video, reversing x\r\n    color c = video.pixels[(width-1-midx) + midy*video.width];   \r\n    stroke(c);\r\n    strokeWeight(w);\r\n    line(x,y,newx,newy);\r\n  \r\n    \/\/ Save newx, newy in x,y\r\n    x = newx;\r\n    y = newy; \r\n  }\r\n}\r\n\r\nvoid keyPressed() {\r\n   saveFrame(\"########.jpg\") ; \r\n}\r\n\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>As we have seen, neural networks can create great\u00a0pictures, but the process is still too slow to be applied in real-time or to implement responsive applications in which the user can affect the resulting image while it is being generated. The Processing framework and programming language shines in such applications. \u2026<\/p>\n<p class=\"continue-reading-button\"> <a class=\"continue-reading-link\" href=\"http:\/\/liipetti.net\/erratic\/2015\/10\/14\/responsive-portraits-in-real-time\/\">Continue reading<i class=\"crycon-right-dir\"><\/i><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2,5,4],"tags":[],"class_list":["post-55","post","type-post","status-publish","format-standard","hentry","category-art","category-interactive-art","category-processing-framework"],"_links":{"self":[{"href":"http:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/posts\/55","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/comments?post=55"}],"version-history":[{"count":8,"href":"http:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/posts\/55\/revisions"}],"predecessor-version":[{"id":612,"href":"http:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/posts\/55\/revisions\/612"}],"wp:attachment":[{"href":"http:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/media?parent=55"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/categories?post=55"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/liipetti.net\/erratic\/wp-json\/wp\/v2\/tags?post=55"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}