BioCLIP: A Vision Foundation Model for the Tree of Life, The Ohio State University
We utilize the CLIP objective © to train a ViT-B/16 on more than 450K distinct class labels, all of which are taxonomic labels from the Tree of Life (a). Due to the text encoder functioning as an autoregressive language model, the order representation is constrained to higher ranks such as class, phylum, and kingdom (b). This results in hierarchical representations for labels, facilitating the learning of image representations by the vision encoder that are closely aligned with the Tree of Life.
Credit: BioCLIP: A Vision Foundation Model for the Tree of Life, The Ohio State University. Link to Imageomics Website
Imageomics, an emerging scientific field, has made remarkable advancements in the past year and is on the brink of significant discoveries concerning life on Earth, as per one of the pioneers of the discipline.
Tanya Berger-Wolf, the faculty director of the Translational Data Analytics Institute at The Ohio State University, delineated the state of imageomics in a presentation on February 17, 2024.
“Imageomics is maturing and poised for groundbreaking discoveries,” Berger-Wolf expressed in an interview preceding the event.
Imageomics represents a novel interdisciplinary scientific domain focused on leveraging tools to comprehend the biology of organisms, particularly their biological traits, through images.
These images can be sourced from various platforms including camera traps, satellites, drones, and even tourists’ vacation photos capturing animals like zebras and whales, shared Berger-Wolf, who serves as the director of the Imageomics Institute at Ohio State.
The wealth of information contained in these images was previously challenging for scientists to analyze effectively before the advent of machine learning.
While the field is nascent—the Imageomics Institute was established in 2021—significant developments are underway, as highlighted by Berger-Wolf to AAAS.
A key area of study that is materializing pertains to the relationship between phenotypes (observable animal traits depicted in images) and their genome—the DNA sequence responsible for these traits.
“We are on the verge of unraveling the direct correlations between observable phenotypes and genotypes,” she remarked.
“Imageomics plays a pivotal role in this advancement, propelling both artificial intelligence and biological science forward.”
Berger-Wolf cited recent research on butterflies as an illustration of the strides being made in imageomics. Researchers are investigating mimics—species that bear a resemblance to a different species. One purpose of mimicry is to resemble a species that predators, such as birds, avoid due to unappealing taste.
In such scenarios, both birds and humans struggle to differentiate between the species visually, although the butterflies themselves can discern the disparities. Nonetheless, machine learning can scrutinize images and discern the subtle differences in color or other traits that set apart various butterfly types.
“Our inability to distinguish them stems from the fact that these butterflies did not evolve these traits for our benefit. Their evolution aimed at signaling to their own species and predators,” she explained.
“The signal exists—we just lack the ability to perceive it. Machine learning empowers us to grasp these distinctions.”
Furthermore, leveraging the imageomics approach enables researchers to manipulate butterfly images to gauge the extent of differences required by mimics to deceive birds. Scientists intend to produce lifelike images of butterflies with nuanced variations to determine which ones elicit responses from real birds.
This innovative use of AI represents a novel approach that sets it apart from conventional practices.
“We are not merely using AI to reiterate what we already know. Instead, we are leveraging AI to formulate fresh scientific hypotheses that are empirically testable. It’s truly thrilling,” Berger-Wolf emphasized.
Researchers are pushing the boundaries of the imageomics methodology to establish connections between the subtle visual differentiations among butterflies and the specific genes responsible for these distinctions.
“In the forthcoming years, there is a wealth of knowledge to be gained that will propel imageomics into uncharted territories beyond our current imagination,” she anticipated.
A primary objective is to leverage the insights garnered through imageomics to devise strategies for safeguarding endangered species and their habitats.
“The future holds immense promise for the contributions of imageomics,” Berger-Wolf affirmed.
Berger-Wolf’s presentation at AAAS, titled, “forms part of the session.”