A group of researchers from the Donald Bren School of Information and Computer Sciences (ICS) recently published a journal article describing the computing systems used in their AI-based artwork displayed at the Venice Biennale International Art Exhibition in Italy in 2019. “This is, to our knowledge, the first time a scientific writing describes a system that applies to virtual beings — the conjunction of chatbots and art — and that involves the use of edge computing tools,” says ICS postdoctoral researcher Neftali Watkinson.
The paper, “EdgeAvatar: An Edge Computing System for Building Virtual Beings,” appears in the 10th anniversary edition of Electronics, a semimonthly journal published by MDPI. Watkinson co-authored the work with computer science Ph.D. students Fedor Zaytsev and Mike Heddes, Ph.D. candidate Aniket Shivam, CSE student Michael Demirev, and Computer Science Professors Alex Nicolau, Alexander Veidenbaum and Tony Givargis.
EdgeAvatar is the system the team designed to implement two virtual beings displayed at the Venice Biennale: one piece represented the internationally renowned Jewish-Romanian poet Paul Celan, and another represented Romanian communist leader Nicolae Ceausescu. Both of these “Talking Head” portraits, developed in collaboration with artist Belu-Simion Fainaru, interacted with visitors by answering their questions.
“The system uses principles of Edge computing to distribute computation among multiple devices/processing modules in order to create an immersive experience,” explains Watkinson. “Part of this experience is maintaining a natural conversation flow; therefore, a major section of the article deals with experiments and research on the adequate timings and silences between interactions that would achieve a human-like conversation.” Their findings reveal that more than 80% of the conversations that visitors had with these two virtual beings during the festival were successful. And the conversations continue. “One of the agents [Celan] was featured during UCI’s Brilliant Future inaugural dinner where, among other people, Chancellor Howard Gillman interacted with our system,” says Watkinson.
The team, led by Professors Nicolau and Veidenbaum, has continued developing works that explore the connections between art and technology. Another piece from the team and Fainaru, “King of the Jungle,” was on display at an international art event in Taiwan in 2020. This work “aims for a more visceral — direct, immediate — effect, with an element of surprise or mild shock,” says Nicolau. “AI is used more subtly here, in the animation/transformation of the image [of a leader].”
The team plans on further improving the EdgeAvatar system, leveraging advances in Edge computing, machine learning, computer graphics and engineering, and experimenting with different implementations in a variety of environments.
— Shani Murray