The ancient Greeks believed that a man’s grace in dance equaled his prowess in battle, so it is perhaps fitting that the pioneers of a brave new virtual world of dance are embedded in the heart of old Nicosia in Cyprus.
Operating from a stylishly modern, glass-fronted office that sits a stone’s throw from the city’s medieval wall, a team of professors and research scientists have been quietly creating the world’s largest database of 3D dancers.
It wasn’t a conscious decision, more a happy coincidence as they strove to carve out a name for themselves within the increasingly competitive market of immersive technologies, but the goal is now to build the world’s first virtual museum of dance.
Yiorgos Chrysanthou, the hugely enthusiastic research director of the Center on Interactive Media, Smart Systems and Emerging Technologies (CYENS), said: “For many years we have been working on virtual humans and how to animate correctly the body and how to simulate virtual characters that look realistic.
“Many people were doing walking, running, but we thought, ‘What’s the hardest thing you can do?’ Walking is a relatively simple thing, but dancing, there are a lot of variations. Each dance carries a unique narrative, capturing a spectrum of emotions through intricate movements and diversion.
“So we started experimenting with modern dance, which was the hardest dance we could think of. We then started capturing local dances, which involved a lot of research into how to capture someone, but also add a circle of emotion to the animation while still being realistic.
“Currently we have the biggest database of 3D dancers in the world.”
Partly funded by the European Union, CYENS is a research and innovation center of excellence that focuses on innovation and emerging technologies to empower knowledge and technology transfer in the region.
It is also a joint venture among the island’s three public universities – the University of Cyprus, Cyprus University of Technology, and the Open University of Cyprus – the Municipality of Nicosia, the Max Planck Institute for Informatics in Germany, and University College London.
From humble beginnings in 2018 when it was pretty much a one-man show starring Yiorgos Chrysanthou, CYENS now employs 145 of the sharpest minds in Cyprus working in 17 different areas of research involving interactive media.
One of these minds belongs to Andreas Aristidou of the University of Cyprus. Armed with a PhD from Cambridge, Aristidou is leading the team currently causing a stir in the virtual ballroom of augmented reality.
He said: “We have captured approximately 400 dance performances through motion capture, and our collection is continually expanding.
“It encompasses a diverse range of dance styles, including folk dances from various countries, modern and contemporary ballet, Latin-based, and capoeira, a martial art.
“Looking ahead, we intend to incorporate group dances, as our motion-capture setup allows for the simultaneous capture of up to three people, and pair dances like tango and salsa are also on our agenda.”
Working with local and international dance schools – CYENS will host a party of Spanish flamenco dancers in the coming weeks – the time needed to process each dance can range from a few hours to a few days, depending on the quality of the motion-capture data and the realism the team aims to achieve.
In order to capture any dance, the CYENS team first sets the stage by calibrating studio cameras while the performer slips into a motion-capture suit. The actual recording time for a dance should take no longer than the duration of the dance itself. It is the processing that is time-consuming.
Marker data, bearing their own ID, requires cleaning and the restoration of occluded data, which can take anywhere between a few hours to a few days. From this, a skeleton is reconstructed in a motion file – such as BVH or SMPL formats – and retargeted to an avatar model, the creation of which tends to be outsourced to a graphic designer.
“After retargeting the motion to the avatar, some refinement may be needed to address issues like self-body penetrations or floor penetrations,” Aristidou said. “The final step is then modeling and rendering the environment.”
Among CYENS’ ever-growing catalogue of dances are Cypriot, Greek, Serbian and Turkish folk dances, African masquerade, ballet – courtesy of the Hungarian Ballet Academy – salsa, bachata, and a host of contemporary dances from reggaeton to hip-hop.
For Aristidou, it has been an intriguing journey that has thrown up a number of surprises.
He said: “I consistently find our work on safeguarding, documenting, and studying our intangible cultural heritage, particularly in the realm of digital dance ethnography, to be the most compelling.
“In this work, we have introduced and developed a contextual motion analysis method that organizes dance data semantically, giving rise to the first digital dance ethnography. This method adeptly exploits contextual correlations among dances, discerning nuanced differences between semantically similar motions.
“It presents diverse organization trees, offering a digital representation of the chronological and geographical evolution of dances.
“Notably, our method yielded impressive insights, revealing intriguing correlations, such as the connection between Chinese Xin-Jiang and Egyptian belly dance. At the time, we were unaware of this linkage, but as validated by our affiliated dance professionals, both dances share Oriental roots and influences.”
While CYENS’ 3D dance database is a world first, research director Yiorgos Chrysanthou understands that Cyprus will need to fight for its place on the world stage of virtual and augmented reality.
But though CYENS operates on a minuscule budget compared with other EU Centers of Excellence, Chrysanthou is confident that the wealth of talent and expertise on the island, coupled with low taxation and an investment-friendly government, will catapult the nation into a hub of research and innovation capable of attracting international small and medium-sized enterprises (SMEs) and startups.
“We have a unique setup here,” Chrysanthou said. “As well as lower taxes it’s a very pleasant place to work, with a good standard of living, and we have a number of areas of expertise to help launch projects whether they be in virtual production or digital twins, as well as the infrastructure.”
For Chrysanthou, each success story achieved by CYENS is another foundation stone upon which to build the dream, and creating the world’s first virtual dance museum is another goal within reach.
And it promises to be a spectacular experience should the vision be realized, with visitors able to explore and interact with the rich history of dance.
“Users will have the opportunity to access and engage with archived data through advanced 3D character visualization in three ways: an online 3D virtual environment; virtual reality with a headset; and augmented reality,” Andreas Aristidou explained.
“In augmented reality, the 3D characters seamlessly integrate into the real world. In addition, we aspire to develop diverse e-learning applications wherein users, captured by an affordable motion-capture device – for example, a simple camera or a depth camera – can mimic the movements of a teacher, using data from the archive, to receive real-time qualitative and quantitative feedback.
“This approach will make learning dance both entertaining and educational. In other words, our objective extends beyond merely showcasing dances in our virtual museum. We aspire to incorporate a 3D dance platform for e-learning within a gamified virtual reality environment.
“This feature enables users to learn dance, synching with rhythm through a virtual-reality headset. Ultimately, our goal is to establish a highly immersive VR/AR platform hosting an interactive virtual dance museum. These applications are designed to captivate and sustain the interest of visitors, providing an engaging and educational experience.”