Ways of Seeing with Computer Vision: Artificial Intelligence and Institutional Critique
During 2018, as part of a research funded by Deviant Practice Grant, artist Bruno Moreschi and digital media researcher Gabriel Pereira worked with the Van Abbemuseum collection (Eindhoven, NL), reading their artworks through commercial image-recognition (Computer Vision) Artificial Intelligences from leading tech companies. First, Pereira created a platform that analyzes the images of the collection using six different commercial AIs, generating a myriad of results. Then, Moreschi went through the results of 654 works, and meticulously analyzed a large portion of them. He iteratively created analytical categories for the results. After that, in a process of understanding and organizing the collected data, Pereira and Moreschi looked for glitches, errors and other unexpected readings by the AI (from their human perspective, evidently). The aim of this was to look for new ways of understanding artworks, by both levelling and expanding their meanings. The images of artworks were often read without their inherent contexts – much different than a traditional human reading. Lastly, the research speculated on how the AIs are constructed, and what these “mistakes” reveal about them and their training data. This led to a brief analysis on the crucial invisible labor of Amazon Mechanical Turkers.
The main takeaways were: somewhat as expected, AI is constructed through a capitalist and product-focused reading of the world (values that are embedded in this sociotechnical system); that this process of using AI is an innovative way for doing institutional critique, as AI offers an untrained eye that reveals the inner workings of the art system through its glitches. This research, unlike some already existing critical analyses of AI, does not focus on accountability and bias. It aims to regard these glitches as potentially revealing of the art system, and even poetic at times. They reveal the inherent fallibility of the commercial use of AI and machine learning to catalogue the world: it cannot comprehend other ways of knowing about the world, outside the logic of the algorithm. But, at the same time, due to their “glitchy” capacity to nivelate and reimagine, these faulty readings can also serve as a new way of reading art; a new way for thinking critically about the art image in a moment when visual culture has changed form to hybrids of human-machine cognition and “machine-to-machine seeing” (Paglen, 2016).
Finally, the results of this research are presented in different ways. Along with this article, the researchers have produced the video Recoding Art: Van Abbemuseum Collection, relating non traditional footage of the museum with some of the conclusions here discussed; two proposals to intervene in the exhibition space; and a draft for a possible publication that presents the museum's collection through their readings by AI. We find it important to speak through these different outputs, as they show how this project uses both art and social research methodologies, occupying an interdisciplinary position.
Bruno Moreschi (São Paulo, Brasil, 1982) is a researcher and visual artist with projects that approach the system of visual arts itself, specially its physical and virtual spaces of legitimization, and focus on decoding the field, revealing its hidden procedures. For your PhD (Unicamp, BR, and exchange at University of Arts of Helsinki, FI), Moreschi conducted emancipated experiences at historical museums in South America and Europe. Artworks in collections and projects in São Paulo Museum Contemporary Art, 33rd São Paulo Biennial, Colombia National Museum and CA2M. Visit Bruno Moreschi's website here (brunomoreschi.com).