Seeing the World from All Angles: How Multi-View Crowdsourcing is Revolutionizing Land Cover Mapping

Join us for a conversation about this blog during the GLOBE Observer Connect session, 14 December 2023 at 12:00 ET/17:00 UTC.

---

The Power of Crowdsourcing

Our dynamic planet is an intricate patchwork of forests, fields, cities … However, mapping Earth's diverse landscapes is an immense challenge for scientists. Satellite imaging has its limitations, while traditional field surveys are time-consuming and sparse. Now, help is coming from an unexpected source: crowds of everyday citizen scientists.

Around the globe, volunteers are collecting land cover images to generate maps through the power of crowdsourcing. One initiative at the forefront of this movement is The GLOBE Program’s GLOBE Observer (GO) app. GO provides unique protocols for users to capture standardized land cover photos, which are then used as data to identify terrain types based on land cover characteristics. Volunteers with smartphones are collecting multi-directional images of their surrounding environment. This crowdsourced data holds enormous potential for scientists looking to track changes in ecosystems, cities, and natural habitats. With accurate and up-to-date land cover maps, scientists can monitor concerning trends like habitat loss and deforestation. These maps also help communities prepare for floods or other disasters by highlighting at-risk areas.  Detailed land classification also enables local areas to make smarter decisions about infrastructure, development, and conservation.

But how can seemingly random photos from phone-wielding citizens produce accurate landscape mapping? The key lies in the approach. While most crowdsourcing programs just have users take single landscape shots, GLOBE Observer does something different. Their system instructs users to take multi-view images in six cardinal directions: north, south, east, west, upward, and downward. This provides a more complete perspective of a location. In most cases, the up (e.g., sky) and down (e.g., dirt, soil, stones, or pavement) directions offer less information for inferring land cover categories compared to the other four cardinal directions. 

This figure shows the GLOBE Observer data collecting protocols, where contributors collect six cardinal directions. Here examples of four directional views, including west, north, south, and east, are presented. Figure from article: Land cover mapping via crowdsourced multi-directional views: The more directional views, the better.

Putting Multi-View Crowdsourcing to the Test

In a new research paper, scientists put this approach to the test, asking if adding more viewing angles improves mapping accuracy compared to single images.

To find out, they tapped into over 9,000 sets of crowdsourced multi-view images from the GLOBE Observer app. The scientists also developed advanced intelligent algorithms to classify land cover types. First, they trained models on single images labeled with land cover categories like forests, fields and water bodies. Then, they steadily increased the data, training new models on two, three and four multi-view images together.

The More Direction-Views, the Better

The results verified their hypothesis. The model images with more views performed significantly better. For instance, with four cardinal directions, the top EfficientNet, a widely used deep learning model, improved its classification F1 score from 0.628 to 0.712. The extra views provide complementary visual information that enhances the models' learning. This effect was consistent across diverse model architectures, from popular options like VGG and ResNet to more advanced choices like MobileNet.

Think about how your perception changes when you observe a scene from multiple angles. A single view only reveals one perspective; but turn right or left, and new details emerge. Seeing the scene from every angle provides a holistic sense of the whole picture. “Now imagine multiplying this effect with thousands of crowdsourced images,” said assistant professor at the Wyoming Geographic Information Science Center, Di Yang, who led this investigation. “Powerful patterns will show themselves even from disjoint snapshots taken by citizens worldwide if we can invite them to contribute to this project.”

Mapping with a Crowd's Eye View

While combining all directions improved accuracy, future work could go further, potentially even picking optimal image subsets on a case-by-case basis. Nevertheless, the study provides convincing evidence that crowdsourcing programs should adopt multi-directional observation protocols. GLOBE Observer's innovative approach enables robust land cover analysis that eludes other global mapping efforts.

“Interestingly, when faced with complex multi-view pairs, the models demonstrated an uncanny ability to zero in on relevant features and sidestep the irrelevant ones,” said Xiao Huang, an assistant professor at Emory Department of Environmental Sciences and a key investigator on this project. “Our team's visual depictions revealed the precision with which these deep learning networks discern and prioritize informative sections of an image, overlooking the contradictory regions. The implications of this nuanced fusion of multi-view data are indeed profound.”

As mapping apps empower more citizen scientists, crowdsourcing is actively revolutionizing environmental monitoring. When coordinated effectively, locationally dispersed crowds can join forces to build maps with a level of detail, scope, and timeliness that surpasses centralized efforts. Scattered perspectives aggregate into collective awareness -- creating a new era in geographic mapping. The crowdsourced perspective helps scientists and the general population alike, see the world anew. 

WyoGLOBE

Citizen scientists are not only collecting images for The GLOBE Program but also helping analyze the data. Through WyoGLOBE Hackathons, students are invited to participate in crowdsourced image labeling challenges. These competitions encourage innovative ways to label the platform's land cover and mosquito habitat images, which improves data quality for researchers. Prizes give students incentives to label as many images as possible using provided datasets. By tapping into crowdsourced image analysis, the hackathons complement the crowdsourced image collection of projects like GLOBE Observer. Together, this demonstrates the full power of citizen science, with the public involved in both gathering and making sense of geographic data. This two-pronged approach helps accelerate both land cover and mosquito habitat mapping protocols.

2022 Hackathon Flyer

So grab your phone and open your favorite crowdsourcing app! Look in all directions and snapshot your surroundings. Your images weave the tapestry of our world, mapping Earth through citizen eyes.

Read the paper: 

Land cover mapping via crowdsourced multi-directional views: The more directional views, the better.

Xiao Huang, Di Yang, Yaqian He, Peder Nelson, Russanne Low, Shawna McBride, Jessica Mitchell, Michael Guarraia. Land cover mapping via crowdsourced multi-directional views: The more directional views, the better. International Journal of Applied Earth Observation and Geoinformation, Volume 122, 2023, 103382, ISSN 1569-8432, https://doi.org/10.1016/j.jag.2023.103382.

---

About the Authors

Di Yang has always been fascinated by the intersection of technology, nature, and society. With boundless curiosity, she explored the landscapes around her as a child, captivated by the diversity of life. This sense of wonder drove her to become a geographic information scientist, harnessing the power of geospatial data to study complex human-environment dynamics. After earning her master's degree in environmental engineering, Di was compelled by the potential of remote sensing and artificial intelligence to advance sustainability. She dove into doctoral studies in geography at the University of Florida, determined to apply novel geo-spatial techniques to conservation challenges. Her research harnessed citizen science, remote sensing mapping, and other innovations to elucidate pressing ecological mysteries. Now an assistant professor at the University of Wyoming, Di revels in mentoring students, colliding disciplines, and pushing boundaries. While her work spans diverse realms like land cover change, biodiversity monitoring, and climate impacts, an underlying passion shines through: illuminating the interconnectedness of nature and civilization with technology and creativity. Whether tracking unprecedented environmental events using citizen science data or tracing historical land use with AI, Di finds joy in the journey of discovery. She believes cross-cutting research holds the key to overcoming environmental threats. By embracing the spirit of exploration, Di hopes to empower communities to build a more sustainable future.

Xiao Huang is an assistant professor in the Department of Environmental Sciences at Emory University. His research focuses on developing and applying novel geospatial techniques to understand human-environment interactions. After earning a master's degree in geographic information science, Xiao wanted to leverage emerging geospatial technologies to address sustainability challenges. This led him to pursue a PhD in geography at the University of South Carolina, where he honed his skills in remote sensing, spatial modeling, and social media mining. Now as a professor, his work spans diverse topics like tracking human mobility during COVID-19, mapping flood risk, and monitoring biodiversity loss. But underlying all these projects is his passion for data-driven discovery and sustainable solutions. Whether harnessing crowdsourced observations or satellite imagery, Xiao finds joy in illuminating the interconnectedness of natural and human systems. From rapid disaster response to unraveling social inequities, Xiao believes cross-disciplinary geospatial research can empower communities to build resilient futures. By embracing creativity and collaboration, he hopes to keep exploring new frontiers where geography and technology meet.

Comments