A couple of weeks ago I published an article in which I used Dall-E and tried to recreate the first page of Killing Joke by Alan Moore and Brian Bolland. The article was a smashing success (at least for a small blog like mine) and got great feedback from both AI and the comic book community. Well, great is an optimistic word. The comic book community was mostly negative about the final output and my conclusions, but what are you gonna do? Try out other AI algorithm? Yes, that is exactly what I am going to do 🙂
Some comments suggested trying out Midjourney for the same task, which was kind of a plan all along. After all, people created comic books with it already. In fact, people created whole alternative worlds with it. So, in this article same target, same distance, different weapon – recreate the comic book Killing Joke using the script written by Alan Moore and Midjourney. And yes, in case you were wondering the cover image for this article is created with Midjourney and the prompt “Alan Moore as a superhero flying, cinematic, extremely detailed, 8K, –ar 16:9”.
This bundle of e-books is specially crafted for beginners.
Everything from Python basics to the deployment of Machine Learning algorithms to production in one place.
Become a Machine Learning Superhero TODAY!
Why do I want to do this experiment, you ask? Well, I grew up reading a bunch of comic books and even tried to write some of my own. Drawing comic books based on some scripts is a hard and long process. So, I wanted to see what kind of impact recent AI developments will have on this industry.
Picking a Comic Book
Since readers of Rubik’s Code are mostly techies and not all of them are comic book fans, in the previous article, I wrote about why I picked this comic book and how comic books are created. So, I am going to repeat some of those things here.
What a lot of people don’t know is that comic books are usually created by two creators – the first one writing a script and the second one creating illustrations based on that script. Today, more people are involved in the process, and there are artists which are just doing the coloring, the lettering, and so on, but these two are the leading roles. That is why when people are stating their favorite comic book authors they are usually referring to the writer – the guy who wrote the comic book script.
For example, the Sandman Overture that I hold in my hands in the image above is written by Neil Gaiman, but was illustrated by J. H. Williams III. So yes, Neil Gaiman is one of my favorite authors, but also Grant Morission (dare I put his name in the Alan Moore article?), Warren Ellis (even though I guess he is canceled now, Transmetropolitan is still some of my favorites) and self-proclaimed wizard Alan Moore. So, why did I pick Alan Moore? There are a couple of reasons.
Writing Style
Well, when writing a comic book script, authors usually have two styles. The first one is more “free”, meaning the authors generally explain what is happening on each page, or panel without…well, over-obsessing about details. This style is too ambiguous for Dall-E 2 to make a good output. So, I picked another style – excruciatingly deitaled explanation of what is happening in each panel. Alan Moore is having this style. For example, here is description for one of the panels for the Youngblood:
As you can, it is a pretty detailed description.
The other reason, of course, is that I love so many of Alan Moore’s novels and you are probably familiar with some of them too. For example, he is the author of V for Vendetta (he hates this movie adaptation), Watchmen (he hates this movie adaptation even more) and From Hell (I will not repeat myself). Finally, I know that there is a real danger of messing with Mr. Moore’s work, so hopefully, he will not summon Astaroth (yes, this is Jimmy’s End reference) or turn me into a frog.
For this experiment, I have chosen Killing Joke. It is a cult Batman episode drawn by Brian Bolland, with a script that can be easily found online. Because recreating the whole comic book would be a hell of a work, I decided to recreate only the first page. Here is the original page, which we want to recreate with Midjourney. Ok, let’s dive into it!
Generating Panel 1. with Midjourney
Alan Moore writes long descriptions. While this is really good if you are an illustrator, this is a problem for my experiment, because Midjourney is just not built to work like that. Before this experiment, I played with Midjourney just briefly and used simple phrases. This is how it is intended to be used, but even the description of the puddle in the first panel of Killing Joke is lengthy. Here it is:
Here is how Mr. Bolland illustrated this first panel:
Yes, it is a perfect representation of what mighty Alan described. What I learned from working on this first panel is that Midjourney is a completely different beast than Dall-E. As I mentioned, I played around with it just a little bit in the past, but I quickly realized that I have to change my approach.
Following what are people working on in the chat, I realized I don’t know jack sh*t about what the prompts should look like. So, my first experiment was, well, weird. I realized I have to chop the script into simple phrases, reconnect them and hope that the output will be good. Here is what I got:
This image is a bit weird, but it is having an artsy feeling that I really like.
Generating Panel 2. with Midjourney
Generating the second panel was quite a challenge with Dall-E. Namely, the problem was focusing on the details and describing a panel in a way that all the aspects are present. Of course, while sticking to the script as much as possible. With Midjourney there was a similar problem. Here is the description:
And here is what I managed to get with Midjourney:
Ugnh… Not so good. The problem that I got is whenever I mentioned “base of the post”, I would get an image of some weird ancient pillar. Here my lack of skill in creating a correct prompt really slowed me down. So, considering other results, this one was pretty good. I mean, I was not going to use this:
Generating Panel 3. with Midjourney
The third panel was there to test what I have learned from the previous two. In a way, it is the first panel with something substantial. Anyhow, here is the description for the third panel:
This panel was illustrated like this:
With this one, I was really happy. In terms of style, it is not similar to the previous ones, but I was expecting problems with the unification of the style between the panels. Anyhow, here is my result:
I dare to say that I have a feeling that this one would be approved by Alan Moore himself. I mean, it is so dark and gritty and it has that Dave McKean vibe to it. I just love it. The only problem is that I would need to flip it horizontally.
Taking a break (part 1)
I got so happy that I got something substantial, so I decided to do something fun. Here is the prompt:
Alan Moore playing guitar
And here is the result:
Ah yes, for the rest of the experiment turn on some Electric Wizard. Back to work!
Generating Panel 4. with Midjourney
At this point, let me just say that script is fantastic. All my frustrations aside, I can imagine that if I would illustrate this I would be super into it. The atmosphere is built in such a great way. Here is how it looks in the comic book:
The cool thing about Midjourney is that even if you don’t get a good result, you end up with something very weird and beautiful. I can say that this made my stress levels much lower than when I worked with Dall-E. Here is the image I decided to use:
I know, there is no fence and the Batmobile looks so weird, to say the least, but look at that! So weird and so cool! I could use some of the other results as well:
Generating Panel 5. with Midjourney
I knew I would finally have to come to panel 5, and I was dreading it. I had so many failed attempts with Dall-E with this panel, that I knew it is going to be a problem. And guess what? I was right. Here we go, panel number five:
And the illustration:
After spending so much time trying out different things, I realized I can not do it. Everything that I tried resulted in something which is not really fit in and/or has some extremely weird things in it. I mean, look at these:
Still, even though these are failed attempts, they are really pretty. So, I decided to embrace the weirdness and turn it up to the max. Here is my choice for this panel:
I like to think of this one as if Batman is some kind of spirit able to occupy machines as well as humans, so exiting out of a car would be something like astral…I know, it is really bad, but as I said embrace the weirdness. Onwards!
Generating Panel 6. with Midjourney
And the illustration is here:
This one was hard to get as well and I was not very happy about it. In style, it is completely different from the rest. However, I was keen to keep the weirdness going on once I embraced it! So, my result looks like this:
Taking a break (part 2)
Alright, I finished two-thirds of the page, so I decided to spend some of the credits on pure fun. So, here is a cool image of Alan Moore generated by Midjourney:
Generating Panels 7, 8 and 9. with Midjourney
Let’s bring this home! I am putting panels 7 , 8 and 9 together because they were more of the same. Picking up the text from the script, removing long descriptions so they can fit the prompt and that Midjourney can “focus” properly. Here are the descriptions of panels 7, 8 and 9, respectively:
Aaaaaaand finally my results:
Final Result and Conclusion
So, here is the comparison.
Before going into my thoughts on this approach, let me start by saying that I am a complete Midjourney noob. I realized that while observing what people put in #show-and-tell channel of this Discord server. I also observed how people craft their prompts and was amazed by the results they got. So, what I realized is that I am restricted by Alan Moore’s script. The amazing brilliant script that was building a world for the illustrator and slowly pulling her in and putting her in a state to draw something beautiful, was ripped apart…by me. This was done with Dall-E and with Midjourney as well.
However, in general, I really liked the experiment. It showed me that I need to learn more about crafting prompts for Midjourney and that this is a skill on its own. In the previous article, I said that there will definitely be a niche of AI-generated comic books, both scripts and illustrations. I still stand by it. My feeling is that their style will be similar, so bringing something that is your own into the mix will come down to your skills with words and prompts. I think Alan Moore actually would like playing around with these tools, because of the magic feeling of turning words into images.
Apart from that, I noticed that Midjourny has its own style so the whole thing looks more uniform. For now, I can see this as a tool that might assist authors in visualizing their scripts better, and for illustrators to shorten their work by generating some details/background or drafts for each panel.
In the previous article, I was a bit doomy/gloomy about the future of the creation of comic books and content in general. I still stand by that one as well. I can definitly see a future where comic book author uses tools to generate script based on a few keywords (take any GPT-3, T5 or whatnot) and then use another tool to generate illustrations, like Dall-E for example. Who knows maybe that will be just one tool. Looking even more into the future, the author herself might be made obsolete. Imagine a crawler that pulls keywords from the tweets from comic book fans (hopefully those tweets are not generated as well by some other AI) and turns them into a comic book by the tool which generates scripts and illustrations.
Thanks for reading!
This bundle of e-books is specially crafted for beginners.
Everything from Python basics to the deployment of Machine Learning algorithms to production in one place.
Become a Machine Learning Superhero TODAY!