How the parallel use of different GenAI tools have changed my creative writing process
Getting started with creative writing is a struggle between immediate gratification and an effort to create something new. I can easily invent a million other things to do than work on my fictional storyline. Like all of the sudden, the cupboard door handles seem to demand my attention, and ask accusingly: “When was the last time you wiped us?” I. Must. Clean. Now. As if cleaning was a really tempting thing to do!
Recently, I discovered how different GenAI tools can help co-create my story and develop the character further. No, I’m not just talking about ChatGPT or other Large Language Model tools, but generative tools for images, videos, and sounds.
Coping with a parallel processes
Lately, I have dived deep (or, as ChatGPT would say, delved) into AI filmmaking tools and processes. Traditionally, the process goes something like this: ideation, script, budgeting, casting, location scouting, storyboard, pre-production, production, post-production, etc. But because I am just one person who comes outside the film industry, I can happily run these processes in parallel.
This wasn’t a chosen strategy, but I have the attention span of a fruit fly, which means I easily get bored doing just one task. Jumping between different production phases stimulates my brain and helps me work on my protagonist’s background.
What Was She Made For?
Let me stop here and explain a bit. Usually, the creative writing process starts with the plot, narrative, and character development, and I am no exception. I had a clear vision (or so I thought) of my character — her gender, age, hair color, key personality traits, relationship status, and what she’s about to do next.
I wanted my character to have a visual form as early as possible. It might sound strange, but this way, my character begins to live a life of her own. It’s like giving a jolt of life to Frankenstein’s monster; my character comes alive and starts to talk to me inside my head. And when this happens, I just have to start listening and write down whatever she is saying.
The Essence of the character
Next, with the Midjourney tool, I created a thing called a character sheet. The character sheet is a collection of images showing your character’s looks with different expressions and poses, like when she is smiling, focused, or angry. It shows her in various postures, choice of clothes, etc. This sheet is needed to keep the character consistent through different scenes in the storyboard and film, but I found out this process also reveals gaps in my creation process.
Suddenly, I’m faced with numerous decisions about my character’s “essence”: What does she wear, and what do her clothes say about her? What is her favorite color? Have her life experiences shaped her appearance in any way? Do her hobbies influence her look? How does she style her hair, if at all, its length, and whether she wears makeup or glasses? As I had to work on the visual representation, I updated my script on the go.
..And What she it is not
It is interesting that the visual process also generates new ideas. For instance, when Midjourney presented an option for her to wear jeans decorated with fake diamonds or spider web patterns (oh dear, where does Midjourney even get these ideas?) I knew definitively, “No, this is not her; she would never wear those.” Wrong type of images help me define what kind of person she is not.
Another example: I prompted that my character has a messy bun, and when I received the image, I realized she also needed to be sloppy in other ways, which helped me complete one idea in my plot. I also wanted her to have a piercing, intense gaze, and I discovered that gray-green eyes suited this purpose better than blue or brown eyes.
Same Same But Not Different: When Everyone Looks Like an Instagram Model
A side note: it’s become very hard to generate a female character that looks anything other than a white 25-year-old Instagram model with porcelain skin. After version 5.1, all the generated females began to look the same: flawless beauty queens from cosmetic commercials.
Sorry, Midjourney, but you’re shooting yourself in the foot by democratizing your model training. When you let people adjust the model’s parameters through RLHF (reinforcement learning from human feedback), of course, they’ll opt for pretty girls over distinctive, personally looking women. Then all you get is more of the same.
From Static Images to Dynamic Videos
Impatient as I am, the minute I had my character visualized, I wanted to create a video clip on the opening shot that introduced my character and the emotions she was going through. However, moving from a still image to a video format increases the details I must take into account. I have to start thinking spatially in three dimensions: the atmosphere I want to create, camera placement, movement, focus, the space, lighting, and light sources
What is the setup? What time of day is it? For instance, my character is typing on a keyboard, but I need to define the kind of room she’s in. My script already mentions it’s late evening and she’s alone in her three-room flat, but in my mind, I hadn’t really detailed her home.
Decisions like whether there’s a dirty coffee mug, a water bottle, or energy drinks on the table, or the room’s size, cleanliness, and decoration all add layers to my character’s story, revealing new aspects of her. I realize I hadn’t done my homework and needed to update my script to be more descriptive about the space my character lives in.
Sounds of Silence: Desperately Seeking for the Perfect Soundscape
Then comes the soundscape. Now I have the video of the opening shot, but it doesn’t have any sounds. I loop the clip, which stimulates me to imagine the missing sounds that reflect the setting and my protagonist’s preferences. When I watch it, I feel that my character loves hard rock from the 2000’s. Before, I had a vision that she was an EDM listener, but no, somehow I can hear her listening to Limp Bizkit’s “Rollin’ Air Raid Vehicle”. Unfortunately I can't attach here something I created with Udio. (a second side note: that’s not my kind of music at all…)
The background music is important, revealing my character’s musical taste and mood. I also “hear” ambient sounds like traffic that indicate she is living in an urban neighborhood.
Finding the Right Voice. Spoiler:It’s not Karen.
Uh-uh, and the voice, what kind of voice does my character have? She is 38 years old and an adult but not too mature. High pitch, low pitch voice? I discovered what her voice could be like by exploring different libraries of AI voices (ElevenLabs, Murf, Artflow). Her voice is a bit on the low side, hasty, and somewhat sarcastic.
The voices in the AI voice library for females tend to be friendly, soft, and empathetic — the kind of voice you’d want to hear before falling asleep. Even a bingo host has more charisma than typical AI female voices. (I found a meme voice called “Karen,” though). I guess AI female voices are made for commercial purposes, not for the film industry. However, for my character, I want more charisma in her voice; I’m not planning to fall asleep or sell face cream online.
I Can Has Cheezburger, no, Controllability?
AI video also has its limitations. The biggest challenge right now is the controllability of the video output. I am waiting for better tools to come out, so I am not planning to tell the whole story with the tools I have at my hands right now. Runway is good if you want to animate dreamlike stories because pretty much all the human motion it creates is slow motion, which turns blurry and weird after 4 seconds.
GenAI: Not Just for Text Anymore
Parallel and iterative multitasking process allows me to evaluate different options and enrich my story on the go. I learned more about my character than I initially thought, which added more depth to the character. When I have all the details of her place, image, voice, and persona modeled, I can even start a dialogue with her while I am still in the process of writing her story. I, for one, am embracing these tools in a creative writing process. Or you can always start polishing your door handles, you must make them sparkle!
#GenAI #CreativeWriting #Filmmaking #Storytelling #GenerativeAITools