GLOBE and Artificial Intelligence

A GLOBE volunteer examines a magnified photo of a mosquito larva during the classification stage of GLOBE Observer Mosquito Habitat Mapper.

Artificial intelligence or AI is more than a buzz word: it can be a powerful tool and it is enhancing GLOBE for good. A recent paper published in the journal Citizen Science: Theory and Practice summarized how AI is smoothing both how you submit GLOBE data and how scientists analyze GLOBE data. In particular, AI screens your photos the moment you submit them, and later, scientists are using AI image analysis to classify mosquito larvae or land cover in your photos. 

Artificial Intelligence Improving Photo Submission

When you submit photos to the GLOBE Program, they become part of a free and open database accessible to anyone. In particular, GLOBE students are encouraged to use the data for their own science research projects. To protect students, all photos are reviewed for inappropriate content such as violence, nudity, or offensive or harmful language. Additionally, the photos are screened to ensure privacy. A photo containing a recognizable face or identifying text such as a car license plate can not enter the GLOBE database. 

When GLOBE Observer launched in 2016, the number of photos being submitted to GLOBE grew substantially. In 2021, more than 429,500 photos were submitted to citizen scientists and reviewed by a person before entering the database. This meant that you couldn’t see your photos in My Observations or in the database for up to several days while they were being reviewed. 

Starting in 2022, the GLOBE Program employed artificial intelligence to improve the review process. Now, an AI tool screens your photos the moment you submit them. Most photos are approved within 15 minutes, so you can see your photos in the database much earlier than before. 

These photos were all submitted through GLOBE Observer. The system AI blurred faces and text to protect privacy.

When the AI flags a photo as being potentially inappropriate or violating GLOBE’s privacy policy, it goes into the manual photo review system to be checked by a person. The AI blurs faces and text in all photos, and the GLOBE team checks each photo to ensure the blurring is working correctly and that the photo meets requirements before it enters the database. We also approve photos that were falsely flagged - even computers sometimes see faces in the clouds!

Using AI and blurring faces and text means that photos that couldn’t be accepted before are now going into the GLOBE database while still protecting people’s privacy. This is increasing the amount of data available for research. 

AI for Research

To use the photos for research, scientists need a way to identify what is in them. Both GLOBE Observer Mosquito Habitat Mapper and Land Cover ask volunteers to collect photos and then, optionally, sample and identify mosquito larvae or classify the land cover. Since the labeling or identification step is optional, most volunteers submit photos without providing that information. This leads to a challenge: how do scientists find the distribution of a particular species of mosquitoes or land cover type from the photos? How do they extract that data from thousands of photos? Image recognition software may provide the solution, but before it can be used, scientists must train models to recognize the right patterns in images. 

Identifying Mosquitoes with AI

Mosquito larvae are difficult to identify, and the effort to train AI image recognition models to label species accurately started years ago. The difficulty was that there simply weren’t enough images to teach the models. Mosquito Habitat Mapper has helped fill the photo gap. 

GLOBE is part of the Global Mosquito Observation Dashboard, a collection of mosquito photos and data from citizen science projects that also include Mosquito Alert and iNaturalist. Scientists at the University of South Florida has been using mosquito larvae photos from the Mosquito Observation Dashboard to train image recognition software. The citizen science data has been extremely valuable. Photos of mosquito larvae from labs are taken in controlled conditions, providing nearly pristine views of larvae. Photos submitted through Mosquito Habitat Mapper, on the other hand, come from phones with a variety of camera capabilities and lighting conditions. This diversity gives modelers rich data to refine the capabilities of AI model. 

By focusing on sections of a mosquito larva in photos, one AI model can identify a particularly dangerous invasive mosquito. Figure from GLOBE Observer: A Case Study in Advancing Earth System Knowledge with AI-Powered Citizen Science, Citizen Science Theory and Practice.

By focusing the model on one segment of a mosquito larva, say the head or thorax, the University of South Florida model is now able to recognize one bad actor, Anopholes stephensi, 96 percent of the time. This aggressive, disease-spreading mosquito is expanding its range in parts of Africa, posing a significant health threat. Being able to quickly recognize the species in citizen science photos provides another valuable data point to scientists and planners trying to mitigate its spread and health impacts. 

Classifying Land Cover with AI

Two research projects have also reported success in classifying land cover in GLOBE Observer photos. The first assessed how well AI image recognition identified land cover types in just under 3,000 GLOBE Observer photos. The model correctly classified the land cover 90 percent of the time. Errors occurred because of differences in cameras, lighting, and variations in what Observers choose to include in their land cover photos. The most common errors were differentiating between between buildings, roads, and sidewalks and between trees and grasses. 

Multiple views of a location helps AI models classify land cover with greater accuracy.  Figure from GLOBE Observer: A Case Study in Advancing Earth System Knowledge with AI-Powered Citizen Science, Citizen Science Theory and Practice.

Another research project found that by providing an image recognition model with all six land cover photos to determine the land cover classification instead of just one photo, the classification was much more accurate. With more information, the model was better able to focus on the dominant land cover type and dismiss irrelevant details.

Looking to the Future

Science research using AI to identify mosquitoes and classify land cover can happen because GLOBE data is available to anyone offering a potential source of training data. The photos have provided a rich source of training data for researchers. That said, even with great advances, AI will never replace on-the-ground volunteers. Your eyes and life experiences are needed to find and collect data significant to your community. 

Nelson, P. V., Low, R., Kohl, H., Overoye, D., Yang, D., Huang, X., … Schwerin, T. (2024). GLOBE Observer: A Case Study in Advancing Earth System Knowledge with AI-Powered Citizen Science. Citizen Science: Theory and Practice, 9(1), 33. https://doi.org/10.5334/cstp.747

Comments