Big Data Engineer
- Via Filippo Corridoni, 117, 56121 Pisa PI, Italy
Cloud4Wi offers Volare, a location analytics and marketing platform designed for understanding and engaging mobile users. With Volare, customers can leverage existing wireless networks to build their business.
Volare is distributed through channel partners and connects more than 65 million mobile users across 15,000 locations in more than 80 countries. Customers include Adecco, Bulgari, Burger King, Clarks, Gruppo FS Italiane, Liverpool, the Moscow City Government, Olive Garden, Prada Group and VTB24.
We are looking for a Big Data Engineer to join our Engineering team in Pisa, Italy, who will work on collecting, storing, processing, and analyzing of very large amounts of data collected by our award winning Volare (™) platform. Your primary focus will be to implement optimal big data solutions and contribute to maintain and monitor those. You will also be responsible for seamlessly integrating the solution with the other components of the Volare architecture.
● Select and integrate big data tools and framework to work both in a public cloud environment and in our customers private data centers
● Implement improvement on our ETL process
● Implement streaming queries and batch processes on our data pipeline
● Proficient understanding of distributed computing principles
● Operations and management of Hadoop clusters
● Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
● Good knowledge of Big Data querying tools, such as Hive
● Experience with Spark
● Experience with integration of data from multiple data sources
● Experience with NoSQL databases, such as HBase, Cassandra, MongoDB
● Knowledge of various ETL techniques and frameworks, such as Airflow
● Experience with various messaging systems (Kafka preferred)
● Experience with Big Data ML toolkits (SparkML preferred, Sklearn optional)
● Good understanding of Lambda Architecture, along with its advantages and drawbacks
● Master degree in computer science strongly preferred