Replica Project: building ARCHiVe

Replica table © Noemi La Pera
Replica flashes © Francesca Occhi
Replica aluminium structure © Francesca Occhi
ARCHiVe team using Replica © Joan Porcel

2015 – 2018

This project gave rise to ARCHiVe, marking the first collaborative effort among the three partner organizations (Fondazione Giorgio Cini, Factum Foundation, and EPFL DHLAB). It tested the synergy of these entities for the first time, setting an ambitious goal achievable only through the collective effort of all involved. 

The Project

The Fondazione Giorgio Cini preserves an extremely rich iconographic photo library, encompassing local and international art history, architecture, urbanism, and culture. It comprises approximately 730,000 photographic positives stored in the photo libraries of the Institute of Art History. 

The Replica project addresses two crucial issues: 

  • How to rapidly digitise this vast number of documents while preserving the original heritage and ensuring optimal technological accuracy. 
  • How to make the resulting database quickly searchable without the necessity of relying solely on textual search methods. 

To meet the first of these needs, Factum has specifically designed a circular scanner (Replica 360 recto/verso) for digitising the Historic Photo Library in the most efficient way possible. Additionally, Factum has equipped Replica with software that allows for the storage of data and metadata (such as the file and its archival position, ID number, and other essential details) during the digitisation process. In this manner, Fondazione Giorgio Cini has initiated the creation of an extensive database of images and information. This substantial volume of data is being stored and analysed by the Digital Humanities Lab at the École Polytechnique Fédérale de Lausanne. 

 In order to develop a geometrically-based search engine for this database, DHLAB team conducted their research focusing on recognising patterns of similarities in the pictures of the Historic Photo Library. DHLAB incorporated one of the most advanced technologies in artificial intelligence, known as ‘deep learning,’ into the process. By leveraging a vast network of artificial neurons, a capability only recently achievable in image processing, various forms of visual entities can be simultaneously analysed in a unified approach. 

In the realm of machine learning experimentation, DHLAB also devised an automatic segmentation process that separates text and images, subsequently annotating them. 

In fact, the Historic Photo Library comprises thousands of photographic positives dating from the beginning of the 20th century until the 90s, affixed to cardboards, with each displaying additional information such as the subject, location, date, and the artist of the depicted artwork. 

objectives

The aim of the project is the digitisation and enhancement of Fondazione Giorgio Cini Photo Libraries through innovative mass recording and analysis techniques.

The overarching objective is to enable the online availability of exceptionally large archives. The enhanced computational capacity, tailored to handle specific data, facilitates user searches at various, increasingly refined levels.

Methodologies

1. PRELIMINARY PHASE

The Fondazione Cini conducted a comprehensive check and reordering of the documents stored in the photo libraries. The team subsequently created inventories and tools for metadata collection (thanks to an XML file system associated with the documents), and ID numbers were applied to the back of the positives or their supports.

2. DEVELOPMENT OF THE RECORDING SYSTEM

The team at Factum developed the Scanner Replica 360 Recto/Verso, a unique recording system specifically designed for this project. The prototype of the system was conceived and constructed in Madrid, encompassing both hardware and software components, before being transported to and installed in Venice.

3. RECORDING PHASE

Fondazione Giorgio Cini formulated a dedicated digitisation workflow for the mass recording and storage project.

Within two years from the project’s initiation, the Cini team successfully accomplished the digital acquisition of the Historic Photo Library (671,564 images), the Bruno Alfieri Photo Library (2,073 images), and the Bernard Berenson Photo Library (30,275 images).

4. POST-PROCESSING AND DATA ANALYSIS

EPFL’s research led to the post-processing of the documents, which encompassed segmentation (utilising dhSegment, a segmentation library for complex documents developed at the DHLAB) and the storage of information present on each piece. Simultaneously, the team developed a geometrically based search engine for the established database.

Results

The aim of Fondazione Giorgio Cini is to proceed with the project by digitising the additional photo libraries stored in the Biblioteca Manica Lunga and to make this heritage accessible online.

Technologies

Replica

The Replica 360 Recto/Verso is a cutting edge recording system designed and realised by Factum Foundation. Its first prototype was conceived for Fondazione Cini back in 2015 for the Replica Project wh[...]

Heritage Lab Italgas

Historical images from the photo archives © Italgas
Historical images from the photo archives © Italgas
Historical images from the photo archives © Italgas
Historical images from the photo archives © Italgas
Historical images from the photo archives © Italgas

Heritage Lab is the Italgas museum-laboratory born to digitize the company’s historical heritage and make the most of the narrative potential of its officially recognised archive through continuous exchanges with local, national and international partners, and as part of the vast European Time Machine consortium network.

the project

The Italgas Historical Archive is an ever-growing patrimony, the study of which makes it possible to reconstruct not only the history of the Company and the people who worked there, but above all the links with the main events of the country and with the world of energy, the role Italgas played in the process of industrialization of Italy, urban development and public services. ARCHiVe carried out the project pre-study and drafting, in synergy with other project partners.

Today, Italgas Historical Archive consists of an original nucleus of more than 1,000 linear meters of documents, 6,000 volumes, pamphlets and magazines, 35,000 prints, photographs and posters, 350 vintage equipment and instruments declared of considerable historical interest by the Italian State and subject to notification and conservation restraint.

The Heritage Lab © Italgas
The Heritage Lab © Italgas
The Heritage Lab © Italgas
The Heritage Lab © Italgas
The Heritage Lab © Italgas
The Heritage Lab © Italgas

Education

Collaboration between ARCHiVe and Heritage Lab also takes place in the field of training for operators, involved in the photographic acquisition and description of archives.
Recently, cataloguing and enhancement processes have also begun for the Italgas library, that represents the result of acquisitions and donations over the years.

Finally, in the last year, two training and information events were held within AOA (ARCHiVe Online Academy), with the aim of conveying, to a public not exclusively specialised, the results achieved with a view to a general valorisation of business archives.

The Mask of Time

2021 – 2022

Artificial Intelligence, 3D creation and modelling, sound design and fashion, interwoven with the Fondazione Giorgio Cini heritage of collections and archives have given rise to a highly original work of art: The Mask of Time. This audiovisual work by Mattia Casalegno and Martux_m raises awareness of the need to enhance the Teatro Verde.

Video still from The Mask of Time
Video still from The Mask of Time
Video still from The Mask of Time
Video still from The Mask of Time
Video still from The Mask of Time
Video still from The Mask of Time
Video still from The Mask of Time

èquipe

the project

Commissioned by Vittorio Cini and inaugurated in 1954, the open-air amphitheatre designed by architect Luigi Vietti (1903-1998) has often been an important cultural venue in Venice and has hosted excellent productions of the performing arts. In 2021 the theatre was the subject of a major restoration project that, thanks to a partnership with Cartier,  brought its architecture back to life and made the most of all the qualities of the building materials, the surrounding greenery, the fascinating spaces and the stunning landscape views.

To enhance this beautiful theatre and relaunch its leading creative role, Fondazione Giorgio Cini invited Mattia Casalegno, one of Italy’s most successful digital artists based in New York, to create a work that would showcase the beauty of the theatre starting from the photogrammetric acquisition carried out with drones by Factum Foundation, an institutional partner of ARCHiVe. Curatorship is by Ennio Bianco.

In The Mask of Time, the architecture and history of the Teatro Verde have been brought back to life in a world inhabited by hyper-realistic avatars, made through the creative use of pioneering 3D animation and text processing software, combined with sound designed by Maurizio Martuscello aka Martux_m. The avatars will become actors performing the theatre’s great historical productions. Photorealistic digital human beings are immersed in real sets as well as dreamlike environments in a post-human sci-fi world, somewhere between fiction and representation, in which Nature and Art breathe and flourish. The narrative unfolds through intense imagery in four acts on the subject of the Teatro Verde: History, Performances, Present and Future. It thus explores themes such as teatri di verzura (garden theatres), the golden age of the amphitheatre, and its dialogue with Nature, both dominated and dominating and now a creative force cooperating in the theatre’s new lease of life.

© Fondazione Giorgio Cini
© Fondazione Giorgio Cini
© Fondazione Giorgio Cini
© Fondazione Giorgio Cini
© Massimo Pistore for Fondazione Giorgio Cini

objectives

Mattia Casalegno’s work starts from the idea of theatre as a place of fiction and representation. Situated at the intersection between Nature and Culture, it explores the relationships and tensions that unite the natural environment, the human world, and technologies.

The project starts from the notion that people interact with the world through multiple sensory streams: we see objects, hear sounds, read words, feel textures, taste flavours, combine information, and form associations between the senses. Similarly, real-world data is made up of various signals occurring simultaneously, such as video frames and audio tracks, web images, captions, and voice tracks. Casalegno adopts this common logic in the creation of the multimodal work The Mask of Time.

In The Mask of Time, thanks to a combination of in-depth archival research on material held by the Fondazione Giorgio Cini’s Institute of Theatre and Opera and the innovative application of Artificial Intelligence with the visual power of new kinds of software, the architecture and the history of the Teatro Verde have been brought back to life in a world inhabited by hyper-realistic avatars. Fundamental is also the creative use of pioneering 3D animation and text processing software, combined with sound designed by Martux_m and the costume designs of the 3D garments by Amin Farah. All of these competencies are perfectly blended into a single artistic meta-language.

documents + technologies

1.

The Institute of Theatre and Opera provided over 10000 images of stage designs and performances held at the Teatro Verde, including photos of costumes from the plays on the Greek myth of Pasiphaë and the Minotaur to the Goldonian Commedia dell’Arte.

2.

Factum Foundation provided a complete survey of the Teatro Verde by making expert use of drones for the photogrammetry as part of the 3D digitisation project of the Island of San Giorgio.

3.

Mattia Casalegno was thus able to move some high-definition realistic human-digital actors within the space of the virtual Teatro Verde, created using the Unreal Engine MetaHuman Creator and the Reallusion Character Creator.

4.

MetaHuman Creator is a software enabling the designer to create photorealistic digital human beings. The character is refined by sculpting tools and control guides to achieve infinite variations in facial and body features.

5.

Character Creator 3 allows the designer to easily create and customise realistic subjects for use in the Unreal Engine and Unity platforms, combining 3D character generation, animation, rendering and interactive design in a single system. It thus allows 3D scanning technology, produced by Scanlab Photogrammetry and adopted from Hollywood game productions, to be applied to a stationary person to create a fully clothed speaking character moving freely in space.

6.

These digital human actors, or hyper-real avatars, can thus move or be fixed in digital tableaux vivants within a virtual theatrical spatiality, made possible by Unity Technologies Unity software.

future developments

In the future it will be possible to draw on all the materials used for this video, for example, to recreate paths inside the Teatro Verde enabling audiences to converse with characters generated in augmented reality, or to offer an interactive virtual tour experience of the Teatro Verde in a metaverse.

Large Scale Digitisation Recording the Cini Collection: Tapestries

Detail of the Lucida 3D Scanner recording © Oscar Parasiego for Factum Foundation
Detail of the tapestry © Oscar Parasiego for Factum Foundation
The ARCHiVe team unrolling the tapestry © Oscar Parasiego for Factum Foundation
The Lucida 3D Scanner recording the surface of the tapestry © Oscar Parasiego for Factum Foundation
The Lucida 3D Scanner recording the surface of the tapestry © Oscar Parasiego for Factum Foundation
Detail of the Lucida 3D Scanner custom software creating a shaded render of the 3D data © Oscar Parasiego for Factum Foundation
Detail of the tapestry © Oscar Parasiego for Factum Foundation

2022

ARCHiVe has successfully concluded the first high-resolution digital recording of a significant tapestry from the Fondazione Giorgio Cini collection: The Entry into Palestine of the Army of Vespasian.

Current équipe

  • Luca Massimo Barbero | Fondazione Giorgio Cini
  • Carlos Bayod Lucini | Factum Foundation
  • Costanza Blaskovic | Factum Foundation
  • Teresa Casado | Factum Foundation
  • Chiara Casarin | Fondazione Giorgio Cini
  • Osama Dawod | Factum Foundation
  • Carolina Gris | Factum Foundation
  • Eduardo Lopez | Factum Foundation
  • Ilenia Maschietto | Fondazione Giorgio Cini

the project

Dated between 1470 and 1480, this Franco-Flemish tapestry, crafted from cartoons by the Master of Coëtivy, stands as one of the earliest and most pivotal pieces in the entire collection. It is part of a series depicting the destruction of Jerusalem.

Recently recognized as the right half of another textile work housed in the collection of the Musée des Arts Décoratifs in Lyon, the tapestry’s recording, preservation, and restoration have become top priorities for the Fondazione Giorgio Cini. The digital documentation of the tapestry’s surface marks a crucial stride in comprehending its material structure. ARCHiVe undertook a comprehensive documentation process, capturing the tapestry’s shape, texture, and color through a blend of 3D and 2D non-contact scanning technology.

objectives

The project’s objective is to facilitate the preservation, study, and dissemination of the collection’s tapestries through the utilization of digital technologies.

Developments

1. PRELIMINARY PHASE

Following a preliminary assessment and careful selection of the tapestries based on their preservation status, the chosen piece has been relocated to ARCHiVe in anticipation of the recording process. A training session was conducted for the team to adeptly handle the manipulation and unrolling of the tapestry. The tapestry was strategically positioned in a horizontal orientation on the floor, placed atop a protective cloth.

The scanner structure was then configured horizontally, facing downward to capture the surface of the tapestry consistently from a fixed distance of approximately 10 cm. This scanning apparatus was securely mounted on a bridge-like structure spanning about 4 m.

2. recording

The Factum Foundation team conducted a high-resolution 3D and colour recording of the tapestry using advanced non-contact digital technology, specifically tailored for the preservation of art and heritage. Lucida is designed to digitise the surface of low-relief objects with high precision. Composite photography produces a single colour image through the integration of multiple photographs in high resolution. The combination of these individual photographs ensures accuracy up to an average error of less than 1 pixel. It’s noteworthy that the colour recording took place separately from the 3D scanning, encompassing both the front and back of the tapestry. The camera was securely affixed to the bridge-like frame at a distance of approximately 50-70 cm. 

3. image processing

The final phase involves image processing and the development of a visualisation browser to explore the tapestry (both its color and texture) at a high resolution.

The digital acquisition of the object is crucial for accurately documenting its current state of preservation, monitoring changes over time and assessing possible restoration processes. But it is also fundamental for art historical research, as browser-based visualisation allows analysis in a completely new and enhanced way compared to the past.

Results

The recording of the tapestries constitutes a component of ARCHiVe’s initiative to digitise the entire island of San Giorgio Maggiore in Venice. This comprehensive effort spans from the architectural level to the precise sub-millimetre scanning of particular artworks and documents within the Fondazione Giorgio Cini collections.

The result is available for online viewing through a multi-layered browser developed by Factum.

viewer

Technologies

Lucida 3D Scanner

The Lucida 3D Scanner is a non-contact laser recording system that captures high-resolution surface texture data for low-relief surfaces. It records 3D data in 48 cm x 48 cm ’tiles’ by projecting [...]

Panoramic Composite Photography

Panoramic Composite Photography is a 2D non-contact method for capturing the color surface of objects such as works of art. A specialist version of the technique can capture accurate and high-resoluti[...]