AVATARS - advanced virtuality and augmented reality approaches in seeds to seeds

Abstract

The identification of genetic and environmental factors determining yield, yield stability and quality traits as well as their complex interactions have been at the focus of cell- and molecular-biological research and applied breeding research since decades. Recent advances in genome analysis, system wide omics approaches and high-throughput automated plant phenotyping offer unprecedented opportunities to increase our knowledge and to enhance crop improvement strategies, but the enormous increase in data volume and complexity bears novel challenges for data analysis, visualization and interpretation. The immense scale covered by analysis techniques, from cellular structures to growth parameters of plant stands, in combination with the enormous volumes of different types of data, encompassing thousands to millions of SNP markers and spatio-temporal transcriptome, proteome, or metabolome data for hundreds to thousands of plant lines, illustrate the complexity of today’s biological data structure. To meet the challenges, novel approaches for data integration and visualization are needed that allow efficient navigation, analysis and exploration. The project began with a requirements analysis in collaboration with researchers from Leibniz Institute of Plant Genetics and Crop Plant Research (IPK) and breeders from Norddeutsche Pflanzenzucht Hans-Georg Lembke KG (NPZ). Based on their input, initial prototypes for data visualization were developed, accompanied by iterative user interface design and testing. Regular meetings with project partners ensured continuous feedback and adjustments. Data, provided via database and file server (IPK), were processed and structured for visualization. Interfaces to the IPK data server were developed and refined to ensure seamless integration. The resulting framework enables dynamic visualization within a virtual workspace, where multiple data types can be arranged, compared, and analyzed interactively. To ensure usability, ongoing feedback was gathered during project meetings and annual status seminars, where external experts evaluated and provided input on the VR application. Furthermore AVATARS was expanded to broader audiences. The "VR Plant Journey," an educational VR experience for children, was published in the Steam store and also ported to the standalone device Meta Quest and published in the Sidequest Store. Additionally, the mobile app "Plant Labs AR" was developed and globally released free of charge, extending accessibility to plant science education. As a solution, we created an interactive environment based on techniques of advanced virtuality and augmented reality (AV/AR) to make data more accessible. This virtual environment allows users to easily navigate through the multidimensional data space across various levels of resolution to select important information and to efficiently perform analyses. It will hence serve as an efficient tool for hypothesis generation and decision- making. We developed an AR/VR-based environment for visualizing large-scale biological datasets in plant research. It enables interactive exploration of genome analysis, omics data, and high-throughput phenotyping, with a focus on a 4D seed formation model of oilseed rape (Brassica napus). By integrating MRI, histology, omics, and environmental data, the system facilitates spatial and temporal analysis, supporting research and breeding efforts. Aligned with "Plant Breeding Research for the Bioeconomy – National Research Strategy BioEconomy 2030," AVATARS built on prior BMBF-funded projects and integrates genome research with quantitative genetics. The project united biologists, plant breeders, data scientists, and AR/VR specialists to enhance data visualization, supporting knowledge transfer and international collaboration in plant science and breeding This immersive approach enhances pattern recognition, supports machine learning predictions, and extends visualization to other growth stages using field phenotyping and drone data.

Key visualizations implemented include: - 3D models of seeds and their organs, highlighting variability from reference models. - 3D scatterplots and volcano plots for genotype analysis (Express, AV01, AV02). - CT and MRI scans of seeds. - Various graphs for cluster analysis, genomic browser views, and metabolic pathways. - 2D representations for field observations, germination tests, TD-NMR measurements, charts, spider plots, and bar graphs. - Porting of VR data visualization for easy accessibility via web browser. - Creation of two edutainment applications for raising awareness of plant science.

AVATARS revolutionizes plant data analysis with AR/VR tools, benefiting researchers, breeders, and educators. It fosters digital transformation in agricultural sciences, providing a scalable framework for broader applications in breeding and bioinformatics.