Introduction to Neural Networks + Art

Editor's Notes

• #3: These are really advanced topics, so we will only give the high-level intuition here.
• #5: https://www.usnews.com/dims4/USNEWS/853ba66/2147483647/thumbnail/970x647/quality/85/?url=%2Fcmsmedia%2Ffd%2F95%2F26afe35a4dde8cf716252ac8ad7d%2F140721-designmain-editorial.jpg
• #6: Each edge out of each neuron corresponds to an image So what does each hidden layer appear to be doing here? Each hidden layer is composing the output of the previous layers to identify the key parts of the face.
• #8: Here we can see an example. So our program creates these faces from scratch, from this image of noise and static. To clarify, these faces aren’t readymade pictures taken by a human--they are generated entirely by our program. (Source: https://raw.githubusercontent.com/torch/torch.github.io/master/blog/_posts/images/model.png)
• #9: Let us get back to our earlier neural network. Suppose we change the output layer that tries to make its output image to be as close to a face as possible (meaning it has all the attributes of a face), when given a white noise image as input. That is, roughly speaking a face generetor. https://www.slideshare.net/roelofp/python-for-image-understanding-deep-learning-with-convolutional-neural-nets
• #10: So here we are giving the program a description: two plates of food that include beans, guacamole and rice. We are asking it to generate an image which contains everything in this description. It seems to have done pretty well! This type of generation problem is called text-image synthesis. This is a very high-level explanation, feel free to talk to me later to find out more: We create something called embeddings for both the text and the images in the training data such that the distance between text and any image is meaningful. That is, if they are close in meaning, then they have a small distance; if they are far away, they have a large distance. By distance I mean vector distance http://imgur.com/Zt7W2vI
• #11: This is called a GAN, and it is a system that can generate images out of white noise. (hard concept, will explain intuitively) A GAN is made up of two networks. One network makes these images. How does it know whether its image is realistic or not? Going back to our linear regression days, we were able to measure a loss--how good or bad we were doing. For GANs, to evaluate the image, we need a whole other network called a discriminator network. This network classifies whether the image the G network produces is fake or not, and challenges the other to improve. Let’s take an analogy here. Suppose we have a bank, and someone who counterfeits money. At the beginning, it’s easy for the bank to detect which money is fake, because the counterfeiter isn’t very good. But the counterfeiter learns from which ones succeed, so some money may get past the bank’s defences. The bank also keeps learning how to tell the fakes apart. So, end result, the counterfeiter is able to produce near-exact replicas of money. Back to our situation, the bank is a discriminator network--it tells the generator network, aka the counterfeiter, what it is doing wrong. The key observation here is that both networks keep learning, so over time, the image quality produced ges better and better. This is how a GAN works. When I say “real image,” I mean Source: https://camo.githubusercontent.com/1925e23b5b6e19efa60f45daa3787f1f4a098ef3/687474703a2f2f692e696d6775722e636f6d2f644e6c32486b5a2e6a7067
• #12: Ask Pamela all the deep technical questions! I understand that we went through a lotta concepts. My goal is to provide an intuitive understanding behind these concepts.
• #14: Remember how yesterday we talked about a pastiche, an art piece which imitates the style of another painting? Style transfer is when a computer makes a pastiche. So you take an image, and say, for example, “I want this to look like an oil painting.” Then you transfer (Source: http://static.boredpanda.com/blog/wp-content/uploads/2015/09/cute-bunnies-25__605.jpg, www.deepdream.com)
• #19: This is hard to define, but let me try to explain it this way. Suppose we have a line. This line, no matter if we flip it, rotate it, or compose it with something else, it will always be red.
• #20: Remember that slide we saw, with the faces and the different hidden layers? Source: https://blog.paperspace.com/art-with-neural-networks/
• #21: look at the weights of each layer, and look at the correlation between them and then we pick out the features that the neural network thinks is most important. this is the style. Content is something that is specific to a couple of hidden layers. But all hidden layers preserve style. If the weights at all the hidden layers emphasize a certain feature, then we know it’s important. And
• #25: Remember that these are represented by pixels.
• #26: Do the exact same for style. So for style we’d run both the style image and our_image through a style extractor, and then compute style_loss by taking the difference between the two styles.
• #29: DeepDream converts an image into another. Let’s look at a bunch of examples, and see if we can guess how
• #30: Once upon a time, this was Starry Night by Van Gogh. Then it was invaded by snakes, fish, birds, ducks, cars… :)
• #32: This is my all-time favorite. Thanks to Pamela for showing this to me. Can anybody guess how DeepDream is created?
• #33: DRINK WATER
• #34: Suppose you’re playing the game Telephone. You say a phrase: “Tiny elephants like bubbles.” One person thinks they hear something else, convinces themselves that they’re right, and then passes it on. The original phrase ends up massively distorted, like “Slimy sea slugs slime trouble.” This is basically what the computer is doing. It detects a pattern, convinces itself it is absolutely right, and adjusts the original image so it’s more confident about the patterns it sees.
• #35: So basically, what the computer is saying to itself is: “Oh, I’ve detected a really small pattern right here. But I feel really confident, and I’m totally right that is a dog! So let me go back and adjust the picture so I’m more confident that my pattern exists.”
• #36: Where do we get these patterns? From the hidden layers, of course!
• #38: So yesterday, with AI Experiments for Google, you may have played with some music-generation stuff! I think the intersection of AI + Music is cool, it’s something that’s very deeply human. Intuition behind RNN: We have a sequence of observations. We assume the next observation depends on the current state as well as previous “hidden states”. (By states, I mean N-dimensional vectors) We need an RNN in music because music falls into many patterns. If we finish a little early, we can watch some AI-generated music called “Daddy’s Car” which is supposed to sound like the Beatles.
• #41: http://colah.github.io/posts/2015-08-Understanding-LSTMs/img/RNN-unrolled.png
• #42: Note that with image recognition, there’s always a right or wrong. There’s no universal definition of what “good art” is.