Web crawling developers design and implement algorithms that automatically discover and extract information from sources across the web. They work closely with data scientists and engineers to ensure that the data collected is of high quality and can be used to power various applications and services. In addition, they also research and develop new methods for web crawling as well as improve existing ones.