As computer scientists, architects, and artists, what are the ways in which we can alter and enhance our understanding of literature and explore its relation to space and material?
We wanted to explore interdisciplinary relationships between literature and design. How can we employ formal analysis of one discipline to inform creation in another? How does this alter our understanding of each discipline and allow us to see resonance between different fields of study?
Feeling compelled by pieces of literature archetypal to the contemporary zeitgeist and having strong interest in architecture and computational design, we wanted to explore the ways in we we can visualize and shape literature into physical space, using contemporary technology.
What it does
Our system generates a 3-dimensional model of a piece of literature, traversing its plot chronologically and employing various methods of formal analysis to inform the shape and space of the design. As the system reads in the literature incrementally, it generates the objects form incrementally, thus alluding to narrative structures within the text. The design was created by mapping the parameters that are gathered from textual analysis to inputs for the generative algorithm used to create shape. The analysis of the text is then used to alter and create space, and thus represents the unconscious experience of formal elements of literature.
Once the final shape is created, it can the be repurposed. More than an object that symbolizes a narrative, the shapes can become practical elements objects used in everyday scenarios.
How we built it
The pipeline was built using Java and Processing (a visual-arts programming language), Autodesk Maya, Autodesk Forge (including Node.js).
The formal analysis of the text is done utilizing an NLP library in Processing and the generative algorithm was also written in processing, allowing use to map the output of the textual analysis to the generative algorithm while incrementally scanning the text. This data was piped into Autodesk Maya in real-time using MEL commands sent from Processing using the Open Sound Control (OSC) communication protocol. The final shape was then rendered in Maya for aesthetic purposes and its ultimate deployment into the browser.
Challenges we ran into
Sending data from Processing to Maya proved to be difficult. Maya contains its own internal Python interpreter, which we had originally intended to use. However, the interpreter did not support import external python libraries, and thus we were not able to use the pythonOSC protocol to listen to data from Processing. Instead, we used Processing to generate commands in Maya's Native MEL language and send them directly into the Maya MEL interpreter, thus bypassing the need to use Python as an intermediary between the two.
Accomplishments that we're proud of
We are proud to successfully implement fairly abstract notions while maintaining the project's creative spirit all while harnessing the power of modern software to realize our artistic ideas.
What we learned
Having worked closely Jaime, from Autodesk Forge, we learned about the new ways in which Autodesk is making complex 3D geometries accessible throughout lightweight web browser apps. The ability to not only view, but interact with 3d geometry through a browser, in a quick and responsive manner seemed a difficult engineering problem solved. We also learned about Autodesk Maya’s internal workings -- it’s python interpreter, MEL interpreter, and the constraints that accompany each tool.
What's next for 'Generative Processes // Materializing Text'
While implementing the pipeline on other texts is the most immediate step, we find that each module in the process can be further optimized: the Text Analysis Tool can dig into semantic structures and sentiments to produce a more holistically significant dataset, the Generative Modeling algorithm could engage with the texturing of the object to potentially provide for a more in depth visual relationship however it then pushes the model entirely into the sphere of hermeneutics from a place of aesthetic prospective.