Data Engineer
- Full-time
Job Description
Our Ideal Candidate will have
- 5+ years of progressively responsible work experience as a Data Engineer in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field.
- Deep hands-on expertise with architecting, building, and maintaining a large-scale distributed data processing pipeline.
- Strong analytic skills related to working with unstructured datasets.
- Experience with end-to-end data pipeline automation process, including data input, data quality validation, data extraction and data reporting
- Build processes supporting data transformation, data structures, metadata, dependency and workload management.
- Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
- Excellence in communication-, collaboration-, and organizational skills. supporting and working with a diverse cross-functional remote teams in a dynamic environment.
Qualifications
Experience using the following software/tools:
- Experience with big data technologies: Spark, Distributed File Systems, Hive, Parquet and column stores in general
- Experience with data streaming and stream-processing systems: Azure Stream Analytics, Spark-Streaming, Azure Event Hub and Service Bus, Rabbit MQ, etc.
- Experience with relational SQL and NoSQL databases, including MS/Azure SQL, CosmosDb, Azure Data Explorer and PostgreSQL.
- Experience with data pipeline and workflow management tools: Azure Data Factory, Azure Logic Apps
- Experience with Azure cloud services: Azure SQL, CosmosDb, Azure DW/Synapse, Azure Data Lake, IoT Hub/Edge, Azure Databrick
- Experience with object-oriented/object function scripting languages: Python, C#, Scala, etc.
- Experience with data service frameworks such as GraphQL and OData
- Understanding of Data visualization tools such as Power BI and Grafana
Additional Information
All your information will be kept confidential according to EEO guidelines.
By clicking the link above or any third-party link within this posting, you are leaving this site and going to a third-party website where the third-party website's terms and privacy policy apply