Natural language processing pipeline with Apache Spark (Vitalii Kotliarenko, Ukraine) [RU]

This talk will put the light on challenges of building natural language processing pipelines at scale. Practical questions to be covered: – quick infrastructure setup using AWS and Terraform – rapid prototyping with Apache Zeppelin – turning a proof of concept into the product – testing Spark applications – monitoring, performance optimization, and capacity planning Code examples in Scala will demonstrate work with various Spark APIs.