In another part of the project, Seltmann’s group used computer vision and machine learning to quantify how hairy different bees are, in terms of both density and color. They were able to characterize 611 bee species across 377 genera. The team published a paper on the correlations they found between hair coverage and how bees adapt to different climates and environmental changes.
Seltmann has also teamed up with engineering professor B.S. Manjunath, an expert in computer vision at UC Santa Barbara, to analyze the structure of bee wings. Wing venation can be a diagnostic characteristic between species, but taking advantage of this currently requires hours of expert analysis. Automating this process could enable rapid, non-invasive species identification from a photo out in the field.
Beyond the bee
UC Santa Barbara’s contribution to the Big Bee Project has already produced 36 publications, 63 posters and talks, and 23 shared datasets and reports. But the project’s impact extends beyond entomology. The initiative enabled Seltman’s group to pilot a number of innovative techniques and codify new standards for natural history collection research. It also enabled the group to carry out fundamental work, like deciding what metrics to use to quantify these complex traits and developing the methods to do so.
It’s simple to count things, and measurements are straightforward to take. But it’s challenging to describe the various forms that life can take. The shape of a leaf or the venation on a wing can be quite subjective. “So scientists have really been pushing to turn these into quantitative things, like numbers, matrices and graphs,” Seltmann explained. At the same time, they’re building large language models and neural networks to identify and organize this new kind of data.
This novel approach to natural history collections changes what is possible to do with them. “We can ask brand new questions because it is now far easier than it ever was to pull patterns from specimens and turn those into things that we can analyze with statistics,” Seltmann said.
This is particularly true for image data, which has never been more ubiquitous in science as in daily life. “Storing images is a no-brainer,” said Manjunath, whose lab developed BisQue, a platform to store, visualize, organize and analyze images in the cloud. But his involvement is taking this to the next stage: “What can we do with those images? What can we learn with those images using computer vision and AI?”
In addition to working with Seltmann on bee wings, Manjunath’s team has applied BisQue to topics as sundry as detecting neural pathologies from CT scans and identifying prairie dogs in aerial photos. The system accepts over 100 file types and can be augmented with custom software modules for each project. The group is currently working to integrate natural language processing into BisQue’s interface to enable more organic interactions between scientists and software.
While the current Big Bee Project is sunsetting, Seltmann’s work on natural history collections will continue. The next phase of her work will be to expand access to the wealth of data they’ve documented to researchers and professionals in other disciplines. She’s particularly interested in collaborations with engineers and material scientists mining biology for solutions to human challenges.
