- RMZ Centennial Internal Road, EPIP Zone, Brookefield, Bengaluru, कर्नाटक 560066, India
You will be responsible for developing, maintaining, and testing data solutions with a wide variety of data platforms including relational databases, big data platforms and no-sql databases. You will collaborate with BI visualization experts, Industry SMEs, Data scientists and client Experts to drive data related projects. You will be responsible for Modelling the Data Lake. You will develop various data ingestion transformation routines to acquire data from external data sources. You will be involved in securing access to the data based on appropriate rights, implementing data quality routines and mechanisms to flag bad data for correction, and building QA and automation frameworks to monitor daily ingestion of data and provide alerts on errors and other problems.
· 8+ years of experience in a Data Engineering/Architecture or similar role
· BE/B.Tech or Advanced degree in a relevant field (Computer Science and Engineering, Technology and related fields)
· Expertise in data engineering, data modeling, data analysis, data quality and end to end data management solutions for on-premise and cloud platforms
· Experience working with ETL tool like SSIS or Informatica Power Center
· Strong Experience working on API integration using ETL tools
· Experience on scripting languages like Unix shell script/Python
· Excellent knowledge of data technologies including traditional (RDBMS) , non-traditional (NoSQL) platforms and BigData platforms
· Knowledge on Data Modelling. Good to have experience with Data Modelling tools(ErWin)
· Experience implementing Data Quality in Data Lake solutions
· Proficient with writing SQL queries, Stored Procedures and Views
· Prior experience in big data implementations (Batch/Stream/Real-time processing), ELT/ETL and Hadoop.
· Hands-on experience with Azure services such as Azure Data Factory, Azure Databricks, Azure functions, Logic aps, Synapse, ADLS, Azure SQL server, SQL/PYTHON etc.
· Handling unstructured data extraction from REST API and Web sources and building data pipelines, Scheduling in Azure data factory.
· Proven success in communicating with client counterparts, technical teams, and senior management to collect user requirements, describe data modelling decisions and data engineering strategy.
· Experience with Agile project delivery methodology and knowledge of using JIRA is highly preferred
• Insurance Domain preferred
• Sr ETL Lead