Project Prompt
What if Data & A.I. were gods in hindu mythology, with their own religious following?
Project Context
- Type: Comic for DeepHack Hackathon
- Duration: 1 day (~12 hrs)
- My role: Story writer, Art director for A.I., Panel stitcher & Pun maker
- Collaborators: Set of AI tools such as ChatGPT, Lexica, Stable Diffusion, VectorizeAI & BeatovenAI.
Final Comic
I decided to make a comic for the Creative Chronicler track at the DeepHack hackathon in Bangalore, organized by my talented ex-colleague Nirant, & the wonderful folks at Hasgeek.
The result was a short comic (8 pages × ~3 panels per page) about the rise & fall of data as a Hindu god.
You can watch me narrate the comic in this video:
…or read it at your own pace without my voice in your head:
Read on further to know more the process, tools, response & reflections.
Beginning: Concerns vs Potential
I have been hesitant to explore AI for making art for two key reasons:
- Ethical concerns about its disregard for artists’ IP.
- A business model that pays to those who build the tech while ignoring the labor of those whose work is used to train models, or those who spend hours manually tagging & training these models.
However, unlike my poor opinion of NFTs which hasn’t improved with time (re: Philosophy, Art Market, Climate), AI projects have been increasingly making me curious, excited & terrified about our future with AI. I also loved Akshay’s talk at DesignUp last year on how AI+human > AI. I wanted to explore it to form my own opinion.
When I found out that an upcoming AI hackathon has a track for creating content using AI, I decided to see if AI can help me make good art.
Inception & Ideation, ft. ChatGPT
I find illustrations inspiring, and comics engaging, but I’m neither an illustrator, nor a comic artist. So I went to the hackathon with a goal to make a comic, and use AI whenever I needed help.
I also love data — I’ve spent more than a decade working with design & data. In fact, the only comics I’ve ever made were data-driven comics, like at Gramener’s data comic hackathon, where characters changed based on data using Gramener’s Comicgen. This time, instead of a data-driven comic, I thought I’ll make a comic about data.
I also wanted to connect it with Hindu mythology. I grew up in a traditional Hindu family, listening to several कथा (Kathā / Stories) about humans & gods. I loved the simple narrative arcs, chants praising gods as part of the story, and some relevance to life & reality.
I have often heard folks connect today’s inventions with ancient tales, to prove that Indians invented/created it first. I can never forget sketches from Goodness Gracious Me on how Da Vinci, Christianity, Santa & even Superman is Indian. So I wondered, what if I could bring data & AI to be a part of Hindu mythology — and highlight that they were both actually Indian-origin!
For the story, I tried using ChatGPT for the script. While it summarized popular stories & explained the eternal cycles of creation & destruction of universe, it struggled with specific prompts like exploring stories where data was key.
Dissapointed & eager to progress further, I came up with a story arc with an intriguing story, playing with puns on the terms Data (as दत्ता / Dattā) & AI (as आई / Aaee). I used समय (Samay / Time) as a narrator, just like the Mahabharata tv serial that I grew up watching. I also hid a reference to my mentor Jay Dutta’s name in the story, along with some lines from LOTR.
Storyboarding & Aesthetics, ft. ChatGPT
I checked with ChatGPT about story structures, which was faster than checking my own notes/searching online. It gave a crisp response: Exposition, Rising Action, Climax, Falling Action, Resolution.
I quickly sketched out a story board, where each page was mapped to one step of the story structure. I decided to use the conventional layout of 3-panel per page, and now had an idea for generating each of the ~18 images (6 pages × 3 images per page).
I wanted to play safe for the comic art style. I knew that getting a consistent style would be hard, so I decided to use a few constraints on the aesthetic to make the image generation is easier. I decided that sticking to black & white visuals would be easier, something similar to my earlier experiments with linocut printing. Worst case, I could make colored visuals grayscale on my end. So I asked ChatGPT for comic book artists that are famous for this style.
I explored each of the comic artist to make a moodboard of art style that I wanted.
Art Directing through Prompts, ft. Lexica & Stable Diffusion
With a broad direction in place, I started exploring examples of AI-generated art on Lexica to understand how prompts were written.
I only knew about popular prompt-based generation tools like Dall-e & Midjourney. I got a quick intro to using prompts during a speculative art workshop by Computational Mama (with GooeyAI, a great aggregated platform). But I was slightly worried about the daily limits & amount of control I had on those platforms.
Thankfully, Nirant introduced me to Amogh, who has been doing some incredible experiments with AI tools. He gave me a quick intro to Stable Diffusion, and after a lot of stumbling with TheLastBen’s notebook (Errors setting up models, struggles with Google drive, finding an older working commit…) I found a Collab notebook that worked. This ran the Automatic1111’s Stable Diffusion Web UI in the browser.
One aspect that I loved about this UI was the incredible amount of options, as well as the ability to add weights in the prompt. I spent hours trying out different prompts, till I narrowed down to a set of interesting keywords that described the aesthetic I was going for:
(opart:1.2) dark (black & white:1.2) comic (illustration:1.4) ink (high contrast:1.2)
Running this via Collab was quite glitchy: the runtime would often stop abruptly, some features would just not work (I saw Amogh getting frustrated often), and I didn’t know how long I could use it (for free). But it definitely had a much bigger limit than MidJourney & Dall-e, and so much more control.
I went ahead and generated a bunch of images, sometimes doing multiple options for a single panel till I was happy. I felt pretty proud of the result, most folks who saw it were impressed with the style feeling consistent. After fixing some of the images manually (cropping/patching), I decided to call it a night and went home around 2AM.
Laying it out, ft. VectorizeAI & Powerpoint
On Saturday, I went to hang out with my friends who recently moved to the outskirts of the city. I squeezed out some time between conversations to polish the images.
I had generated all square images, at 512px × 512px size, but images were often blurry, less detailed or too sketchy. I decided to run all the images through Vectorize.AI, which smoothened images that had simple shapes, and merged/simplified images that had a lot of detail. I would have ideally wanted to spend more time on this to get each image to feel more consistent, but I was running out of time.
I was unsure what was the fastest way to put the comic together, and I wanted to demo the comic by narrating it to the judges. So I decided to bring it all into PowerPoint: this gave me the ability to animate each panel to appear, yet allowed me to do a high-res export if I wanted to upload or print it.
By afternoon, I was done with laying everything in panels. It was time to add text.
Attention to Details, ft. ChatGPT
I added the story I was narrating as blocks of text in the comic. There wasn’t enough ‘speech’ in my comic, but I tried to add a bit of variation in how I placed the text around.
To pick a typeface, I asked ChatGPT for: best google font for comic with devanagari & english and it recommended Baloo by Ek Type, which didn’t feel like a comic typeface. I asked it again for another option, and Kalam by ITF came up — which looked perfect!
For the cover, I thought of adding some fictional reviews. I had done something similar for a data comic as a quiz that I had made with S Anand. But this time, I used ChatGPT to come up with names of pioneers in Data & AI, and asked it to write fictitious reviews. It worked pretty well at rewording & rewriting the reviews into short puns.
I also asked ChatGPT to write a disclaimer for the comic, and help me explain some of the Sanskrit, Hindi & Marathi terms that I could add on the comic. It worked surprisingly well for both:
Demo & Response (I WON!)
The hackathon demos took quite a while to get through (~3 hrs!). I loved the energy in the room, and having a few familiar faces in the crowd helped!
The judges were Amod Malviya, Manisha Raisinghani, Sunit Singh & Toshi Prakash. I first presented Amogh’s entry (who was enjoying a concert!), and then presented my comic.
You can watch the livestream to see me present my comic.
I was pleased that most of the folks got the jokes, and found several bits amusing right from the start. Apart from a few projector glitches, the demo was a lot of fun, and I got a wonderful applause at the final punchline.
Judges had some great feedback, digging deeper into what part was human-led vs AI-led and the lack of consistency in aesthetic (Mongolian? Japanese? Woodcut?). Overall, the response was incredibly positive.
There were a few other entries in this track, and I was pleased to find out that my entry was picked as the winner!
Final edits, ft. OBS, Adobe Enhance & BeatovenAI
To get to the final recording (shown above), I made some minor improvements to the comic (adding borders to panels, better positions for explanations, making first & last pages with black background…). I then recorded myself narrating it using OBS, separated the audio from the video file, and ran it through Adobe Enhance. It definitely transformed the audio from my cheap headphone mic, and cut out all background noise of the neighbors.
I got a bit more ambitious, and explored Beatoven.AI to generate a background track for my recording. I thought I’ll just take what it generates, but it had a lot of interesting customizations, and I ended up breaking the story into chapters & selecting moods & Indian instruments that would match the content. The free tier limited my exploration, but overall I was happy with the result.
I combined the video from OBS with the enhanced audio & background music using iMovie, and the uploaded the final result to Youtube. My first Youtube video!
Reflections
This was quite a long post to document the whole process. I’ll add some reflections soon!