Darmstadt/Höxter: Visitors to the Westwork in Corvey will soon be getting an even more intense impression of its stucco figures and wall paintings and of the Carolingian basilica in its entirety, because the remarkable features of the historic Johanneschor (St. John’s Choir) that are currently described orally during guided tours will soon be made visible courtesy of a new app. Researchers at the Fraunhofer Institute for Computer Graphics Research IGD have developed software that visualizes the monastery as it looked centuries ago.

From the start of the 2023 season next April, visitors will be issued with a tablet that they hold up in front of them and point at various features in the room. The screen will then display the same view shifted back through time, together with additional information about what the visitor is seeing. Annika Pröbe, research assistant at Corvey Monastery, explains, “We aim to fill young people with enthusiasm about this World Heritage Site and to get them talking. Academics from various disciplines have been working on visual reconstructions for many years, and we would now like to make their work accessible to our visitors. Augmented reality opens up a host of new possibilities.”

Historic preservation poses challenges for development team

Though not the first historical project to be undertaken by the Darmstadt Institute, this one is unique. In 2014, the Carolingian Westwork and the remains of the Civitas were designated a UNESCO World Heritage Site and therefore enjoy a special protected status. This posed a challenge for the Fraunhofer IGD research team, as project manager Andreas Zapf explains: “We had to rethink and find new solutions. In the process, we worked closely with our client. Otherwise, the project would not have been feasible in its full scope and in the time available.”

One way of mapping the physical environment together with supplementary features, such as reimagined 3D stucco figures and murals in their correct position, would be to place markers on the walls. Because of the restrictions in place at the Corvey site, this technique was not possible, and a certain creativity was required. Zapf's team used features on the walls as natural markers and oriented themselves to the existing viewing angles. Zapf explains: “When the user turns the tablet towards one of these features, the image aligns with the surroundings.” The content of the app is based on academic research.

In addition to the complex architectural visualization based on existing plans of the Westwork, the reimagination and replication of the 3D stucco figures, which only exist as fragments, presented a further major challenge of this project. Here, Zapf and his team received input from colleagues at a sister Fraunhofer IGD Competence Center led by Pedro Santos. The department, which had established a name for itself from past work on autonomous scanning techniques, supported the team in advanced high-resolution scanning of fragments. From sketches made by the researchers, they modeled matching 3D replicas of the stucco figures. As a result, the original overall look of the Westwork gradually emerges from many individual parts and is brought to life by means of the app.

Trial run of the app with a real audience

Guests at the anniversary event on September 25th of this year will be invited to try out a preview version of the app. It will then go into use with the general public in April 2023 at the start of the next visitor season. The app is not intended as a replacement for guided tours of the Westwork and adjacent palace buildings, but rather to fulfill a supplementary role, adding fascinating insights.  “A marvelous experience that we can offer groups of visitors in the future,” says Professor Christoph Stiegemann, Project Manager World Heritage Site Westwork Corvey.