ICM - Final Project - Science Fiction Movie Generator
I have absolutely loved my first semester at ITP, but with my main stated interest in Physical Computing coming in, it was a total surprise for me that my favorite class this semester ended up being Intro to Computational Media. I was endlessly inspired by the things we learned each week, and I enjoyed working on my projects for the class so much that it almost became the reward I gave myself for completing my other weekly tasks. I think to some extend this high expectation for the course was what led me to feel initially unexcited about my first few ideas for this final project.
There were two seemingly disparate concepts I was toying with. The first was based on my affinity for tactile, scientific, button-driven interfaces from around the time of the Cold War. I love the look of this sort of control panel, and I was considering developing a digital facsimile of a panel of buttons and meters which would display some sort of collection of data. I had even done a sketch earlier in the year inspired by vintage vector displays that I imagined I could incorporate somehow. However my main hang-up was that I had no idea what the source of the data displayed would be, and that felt crucial to imagining more clearly what this would look like.
The second idea was a more comedic one, based on the humor generated by computers attempting to write like humans. There are all sorts of fantastic examples of this all over the web, from this twitterbot created by Daniel Sullivan that offers rhyming WikiHow advice to these recipes and recipe names generated by a neural network with the help of Janelle Shane. I knew that if I made something along these lines I would find it hilarious to work on, but I also didn’t have a real idea of what the content for this idea would be either.
Thankfully part of the project process involves discussing one’s ideas with their classmates, and it was in these discussions that I had a eureka moment and settled on my idea of combining both of my ideas into a mechanical (or mechanical-looking) device which would have a sole purpose of spitting out computer generated text. Specifically I imagined it as a ‘found object’ from an alternate reality in which Hollywood studio execs at the height of the studio era would have used said machine to generate film titles and plot summaries for the countless movies that were being churned out at the time.
Once I had this core idea, the first step was to collect all the raw data which would be used to train the model writing these titles and summaries. I spent a decent amount of time sifting through various JSON files and movie database APIs online looking for the most robust and simplest source for all this movie data before ultimately settling on the API from The Movie Database. It had all the elements I was looking for: a vast collection of movies featuring accessible titles and descriptions, the ability to pull data from it via the JSON format, a well functioning system on their website for testing different API calls, a very lenient limit on the number of calls allowed and most importantly a ‘Discover’ feature which allowed me to sort by country of origin, years released and genre.
To begin with, my aim was to create a database of every movie released in the US by a major studio between the years of 1927-1948. I ended up writing a program which sorted each movie’s title or plot into a slot within an array, and then I used the saveStrings() function in p5.js to create a text file with each movie on a separate line. Because of the versatility of the API I was using, the most complicated part of the process was simply iterating through each successive page of results. This is the code for that program.
At this point I had decided to begin my text generation process by developing an LSTM machine learning model using ml5.js. Having never really worked in Python before, and certainly never in the Terminal, I enlisted the help of Yining Shi to set up the training of my model following the ml5.js tutorials. After working through a few idiosyncrasies in MacOS, the model was successfully trained and I loaded it into a generic p5.js sketch that would display results from the model. However as I began testing it out and tweaking the various settings, I was a bit disappointed with the results. I found that using this method, and with such a broad dataset, the text generated was too abstract and did not end up having any real discernible style. For reference, that code is here.
I had spoken with Alisson Parrish about this project around that time, and she had suggested to me the possibility of also using a Markov chain to achieve the effect I was after. So I decided to give that a go using as a resource her code along with the associated Dan Shiffman tutorials. The results turned out much more in line with what I was going for. By dialing in the N-Gram length I was able to create both titles and plot descriptions that felt nearly real but were off just enough to be very silly.
At this point the content was totally there, but I realized that realistically I was not going to have time to fabricate a physical object in time for the due date. Instead I was imagining that I would just develop a simple digital UI to allow users to generate their own movie titles and plots using the code I created. But with this shift from a physical, tactile object to a purely digital experience I no longer felt like the specific content I had focused on was a good fit. I wanted the movies being generated to feel connected to the interface with which they were made and a digital, on-screen interaction felt completely removed from the movies and culture of the 30s and 40s. I am interested in creating experiences that feel genuine and ‘real’ and no one would confuse a computer program that takes design cues from studio-era Hollywood for something actually from that period.
So instead I pivoted my thinking and decided to focus on my personal favorite genre of movies: Science Fiction. I created a new data set based on every english language science fiction movie in the TMDb database and adjusted my thinking about the user interface to match. Instead of looking at Cold War era mechanical interfaces for inspiration, I looked ahead in time slightly to retro future interface design from the 1980’s and 90’s. I first designed the UI by hand and then adapted it into a PNG file in Photoshop with strategically placed holes. When I imported it into my P5 sketch I was able to then animate flickering ‘lights’ in these holes to give life to the interface every time a user clicks the generate button. In placing these lights I made sure to calculate every position relative to the height and width so that the program is adaptable to any size display.
By switching to this retro future aesthetic and content I created a project that I feel much more realistically imitates something from the past - just this time it is a vintage piece of software instead of a retro machine. And not only that but it also turns out science fiction movies in general have some of the most identifiable and ridiculous plots descriptions which has led to fantastically wonky, fun and distinct text output.
As happy as I am with the final product, I have no intention of letting go of the goal to turn this project into a physical, standalone object. In a way this project feels more like a proof of concept than anything else and I absolutely intend on continuing to expand on this idea of generating movies with the help of the computer in my future.