Web crawling at Google scale: Jordan Janeiro at the Computer Science Insights talks

We had the pleasure of hosting Jordan Janeiro, PhD, Technical Program Manager at Google, in the Computer Science Insights @HSG series of the School of Computer Science, in the Arena at SQUARE.

His talk, Web Crawling Infrastructures in Digital Advertising, tackled an enormous practical challenge: keeping product information such as pricing and availability accurate and reliable at massive scale, across many different players. When a shopper’s decision hinges on whether a price is right, the crawling infrastructure that verifies, validates, and acquires that data becomes critical. Jordan walked through the approaches and techniques used to get there, the limits of information presentation and synchronization, and the standard web application techniques that counteract them, and he outlined key technical challenges that can inspire research.

Many thanks, Jordan, for the insightful talk!