The narrative of Onirica () was constructed from the selection of a defined number of dream memories within the dataset. Thirty dreams, peculiar in terms of richness of visual elements, bizarreness, level of involvement in the narrative, presence of engaging and imaginative themes, unrealistic visions, and subversion of physical laws, were chosen from the 807 accounts of the Laboratory of Sleep Psychophysiology at the University of Bologna.

In addition to the selection of narrative material, the second element that guided the construction of the work was the analysis of the architecture of dream activity. Sleep, like most human activities, is cadenced by a rhythm influenced by complex biological processes. Within a night there are cycles lasting about 90 minutes, alternating between non-REM and REM stages. In the first half of the night, most of each cycle is characterised by deep NREM sleep, and less REM sleep; in the second half of the night, however, this balance reverses, and most of the time is dominated by REM sleep.

The compositional structure of Onirica () thus echoes the actual alternation of NREM and REM dreams present in the course of a night's sleep, distributing the thirty dreams catalogued by the Laboratory of the University of Bologna into five cycles of six dreams each, forming the narrative basis of the entire installation.

The transition between contiguous cycles is represented by what we call bridges, introducing a semantic journey that crosses the database through similar dreams, leading to visualising - through a chain of adjacent words in space - the path between the last word of the sixth dream of one cycle and the first word of the first dream of the next cycle. Material from the DreamBank at the University of California Santa Cruz was used to create the bridges due to its composite and heterogeneous nature, which is essential to enhance the variety and vastness of the data complex.

Underlying the visual development of Onirica () is a text-to-image diffusion model: a generative model that exploits neural networks to learn how to synthesise images from textual descriptions. The first step toward generation was a textual analysis of the thirty selected dreams and their semantic structure. Each account was deconstructed into paragraphs, sentences and visual images and then processed through a Large Language Model (LLM), a Machine Learning model capable of working with natural language, used to extract the structure and visual information contained in the individual dream accounts.

We then proceeded to write short texts (prompts) containing precise compositional and subject descriptions capable of directing the text-to-image model toward a visual translation of the textual content of the dream. The syntax used in the prompts is aimed at placing greater emphasis on the emergence of the main concepts and subjects in the generated image and to overcome the interpretive biases inherent in the algorithm used. Parallel to the individual compositional fragments, the experience in its totality is described and addressed by overall prompts, which define its aesthetic and sensory atmospheres that contribute to unifying the entire dream stream.

Just as dreams are characterised by more or less coherent streams of images, Onirica () is developed through a dynamic succession of visualisations very similar to that of our mind, referring back to the stream of consciousness that is generated simultaneously with its narrative.
The visual flow that characterises the work was realised thanks to a custom pipeline that allowed us to make the creation process more customised and smooth. This system was divided into different sections: the first one is image generation, where a particle system is responsible for generating the initial image. This is subsequently reinterpreted by the Diffusion Model on the basis of the dream text and through the management of some characteristic parameters (e.g. prompt embeddings, guidance scale, strength, etc.). The versatility of the Diffusion Models allow for a dynamic departure from the source image by giving greater or lesser relevance to the interpretation of the dream text: precisely the management of this balance turned out to be one of the most interesting expressive tools on which the entire visual development of the project is based.

A second step included a particle simulation in a GLSL shader on top of the generated image: it has the goal of adding a further layer of customisation by visually referring to each dream data (sleep cycle, the polysomnography and EEG data) and adding visual elements that react in real-time to the soundtrack of the piece.

The final step is a set of subsequent, additional features that contribute to the visual reworking and artistic customisation of the generated image. This last layer makes it possible to increase the overall compositional complexity and, thanks to a feedback process, ensures visual consistency and becomes the basis for the next frame.

To dive deeper into the creation process of the work, check out our online workshop dedicated to Onirica ().

The particle system is responsible for generating the initial image; this is subsequently reinterpreted by the Diffusion Model on the basis of the dream text and through the management of some characteristic parameters (e.g. prompt embeddings, guidance scale, strength, etc.). The versatility of the Diffusion Models allows for a dynamic departure from the source image by giving greater or lesser relevance to the interpretation of the dream text: precisely the management of this balance turned out to be one of the most interesting expressive tools on which the entire visual development of the project is based. Finally, the image returned by the diffusion is modified by the third layer that deals with the introduction of additional compositional elements into the image. This last layer makes it possible to increase the overall compositional complexity and, through the feedback process, generate the basis for the next frame.