Big Data Engineer/ Cloud Data Engineer
- Contract
Company Description
Orchestrated by adept technical architects with over fifty years of applied expertise, KYNITE is an advanced technology company specializing in the disciplines of: Blockchain, Cloud Services, Big Data & Analytics, Artificial Intelligence, Enterprise, Staff Augmentation and Managed Services
- We are BigData Experts
- We are Cloud Experts
- We are Enterprise Architects
- We are Artificial Intelligence Innovators
- We are Technological Evangelists
- We are Doers
- We are Kynite
Job Description
· 5+ years of experience designing, building, deploying, testing, maintaining, monitoring and owning scalable, resilient and distributed data pipelines.
· High Proficiency in at least two of Scala, Python, Spark or Flink applied to large scale data sets.
· Strong understanding of workflow management platforms (Airflow or similar).
· Familiarity with advanced SQL.
· Expertise with big data technologies (Spark, Flink, Data Lake, Presto, Hive, Apache Beam, NoSQL, ...).
· Knowledge of batch and streaming data processing techniques.
· Obsession for service observability, instrumentation, monitoring and alerting.
· Understanding of the Data Lifecycle Management process to collect, access, use, store, transfer, delete data.
· Strong knowledge of AWS or similar cloud platforms.
· Expertise with CI/CD tools (CircleCI, Jenkins or similar) to automate building, testing and deployment of data pipelines and to manage the infrastructure (Pulumi, Terraform or CloudFormation).
· Understanding of relational databases (e.g., MySQL, PostgreSQL), NoSQL databases (e.g., key-value stores like Redis, DynamoDB, RocksDB), and Search Engines (e.g., Elasticsearch). Ability to decide, based on the use case, when to use one over the other.
· Familiarity with recommendation and search to personalize the experience for millions of users across million items.
· Masters in Computer Science or related discipline
Qualifications
Nice to have
Data bricks
Aws
Python Programming
Spark.
Additional Information
All your
- This job is only for individuals residing in US
- US Citizens, Green Card holders, EAD's can apply
- W2
Information will be kept confidential according to EEO guidelines.