Sr Data Engineer
- Cottonwood Ln, Greenwood Village, CO 80121, USA
CompuGain is an Information Technology and Business Consulting firm providing project-based solutions, software solutions, and professional staffing services.
Responsibilities will include:
Create content maps (decomposition of Tableau reports and data sources)
Analyze data discrepancies to determine root cause and identify correct course of action
Document data lineage (source to target), data transformations, business logic, calculations, data dictionary
Validate data and transformations throughout the entire data lifecycle
Validate expectations and accuracy of the data with business and development leaders
Provide guidance on automated data QA checks throughout data lifecycle
Revamp/optimize ETL scripts/processes and reporting database
Desired technologies (looking for familiarities, not necessarily exact technologies; except for AWS. We are looking for a strong competency in AWS.)
EC2, CloudFrontRoute5, API Gateway, CloudWatch, CloudTrail, CloudFormation
Tableau, Power BI, Looker
Databricks, Alteryx, Snowflake
AWS: Glue, Lambda, Step, EMR, Data Pipeline, Kenisis, Athena, Quicksight
AWS: RDS, Redshift, DynamoDB, S3, Aurora
Required: Postgres, MySQL
Nice to have: Oracle, MS SQL
Required: Python, SQL
Nice to have: Scala, Java, .Net
Desired concepts and other experiences:
ETL and ELT
Data Warehousing and Data Marts
Big Data and Data Lakes
Kimball Vs. Inmon architecture patterns
Business Intelligence (end-to-end implementation and requirements gathering is highly desired.)
The Data Engineer is responsible for the maintenance, improvement, cleaning, and manipulation of data in the business’s operational and analytics databases. The Data Engineer works with the business’s software engineers, data analytics teams, data scientists, and data warehouse engineers in order to understand and aid in the implementation of database requirements, analyze performance, and troubleshoot any existent issues.
• Design, construct, install, test and maintain data management systems.
• Build high-performance algorithms, predictive models, and prototypes.
• Ensure that all systems meet the business/company requirements as well as industry practices.
• Integrate up-and-coming data management and software engineering technologies into existing data structures.
• Develop set processes for data mining, data modeling, and data production.
• Create custom software components and analytics applications.
• Research new uses for existing data.
• Employ an array of technological languages and tools to connect systems together.
• Collaborate with members of your team (eg, data architects, the IT team, data scientists) on the project’s goals.
• Install/update disaster recovery procedures.
• Recommend different ways to constantly improve data reliability and quality.
• Bachelor’s degree in computer science, software/computer engineering, applied mathematics, or physics statistics.
• 2-3 years working experience in data engineering, data warehousing, data integration or business intelligence
• 2-3 years of experience writing complex SQL, with experience working with multiple database platforms
• 2-3 years of experience managing, debugging, and optimizing databases that are critical to the business’s mission.
• 2-3 years of experience working with ETL tools
• Strong experience with Python writing data transformation jobs
• Strong working and conceptual knowledge of building and maintaining physical and logical data models
• Excellent communication skills with the ability to collaborate with non-technical partners
• Familiarity with cloud-based data engineering (AWS)
• Experience working in an Agile environment working on a Scrum team
N. Jagdish Reddy
All your information will be kept confidential according to EEO guidelines.