During my final bachelor project, I designed a tool that allows for crowdsourcing the annotation of geospatial datasets. I set out to innovate on classic crowdsourcing platforms, and created and tested a chatbot for the task.
This project was graded with a 9 out of 10.
In a near future, computers will be able to answer geo-analytical questions like: how densely are trees located in Amsterdam. In order to enable this future, geospatial datasets must be organized in such a way that the computer can choose the right one for the task.
In Utrecht, they developed a classification to allow this organization. However, annotating the geospatial datasets using this classification is a task that requires abstract thinking and can not yet be executed by a computer.
This project explores the creation of a tool that allows annotating these datasets on a large scale by volunteers. After exploring tangible tools and a way to create tangible interfaces using Augmented Reality, a prototype was created that employs a chatbot. The chatbot asks users multiple-choice questions about a geodata set, in order to annotate them in the right categories. This prototype was evaluated to be quicker to use than the control study, in which an instruction manual was used to teach the annotating task.
The chatbot was more effective in annotating one dimension of the datasets (its geometry), but less effective in annotating the entities. The chatbot scores lower on cognitive load, meaning it requires less mental resources to complete the task. While the chatbot was evaluated positively on usability and cognitive load, the quality of the data it generates leaves a lot to be desired.
The quality of data can be improved by further nuancing the questions the bot asks, or challenging the users to learn a little more context about the complex task at hand.
The chatbot tool takes a leading role in the annotating task, while the tool could also play as a guide and let users create their own flow in the annotating task. The difference in this needs to be evaluated in the future, and a possible tool for evaluating a guiding role could be an interactive flowchart interface.
Tjeu van Bussel, 2021