At KMS Technology, we are dedicated to delivering cutting-edge solutions and services that empower businesses to achieve their goals. Our team is composed of highly skilled professionals who are passionate about technology and innovation. We provide a dynamic and collaborative work environment where you can grow your career and make a significant impact.

We are seeking a highly skilled and motivated Senior Software Developer to join our Data Engineering team. This role is responsible for designing and implementing complex, high-performance data integration solutions using Apache Spark. The ideal candidate will have deep expertise in Spark internals, PySpark development, and performance tuning, with the ability to lead the creation of custom Spark connectors and optimize core data transformation logic at scale.

You will work with large, heterogeneous datasets, contribute to architecture and ingestion strategies, and ensure that Spark pipelines operate reliably, efficiently, and cost-effectively.

Responsibilities:

• Design, implement, and optimize high-volume data ingestion pipelines using Apache Spark, integrating internal and external data sources through both standard and custom connectors.

• Lead the development of data integration strategies for moving, transforming, and loading large-scale, diverse datasets using Spark (PySpark) across cloud environments such as AWS, Azure, or GCP.

• Translate complex business and technical requirements into scalable PySpark notebooks and jobs with clear, maintainable structure.

• Apply advanced performance tuning techniques for Spark applications, including optimizing cluster configuration, partitioning strategies, and resource allocation.

• Port existing ETL/ELT logic and transformation processes from legacy systems into optimized PySpark-based implementations.

• Implement testing, monitoring, and robust error-handling mechanisms within Spark pipelines to ensure data integrity and operational reliability.

• 5+ years of professional software development experience, with 3+ years focused on large-scale data engineering.

• Expert-level proficiency in Apache Spark (PySpark), including Spark RDDs, DataFrames, Spark SQL, and deep understanding of Spark internals (e.g., execution plans, DAGs, memory management).

• Proven experience designing and implementing large-scale, high-throughput Spark-based ingestion pipelines, using both custom and standard data integration patterns.

• Strong hands-on experience with ETL/ELT pipelines and data warehousing concepts.

• Advanced Python programming skills, particularly in data processing with PySpark and Pandas.

• Practical experience identifying and resolving Spark performance bottlenecks in distributed computing environments.

• Familiarity with Spark-based services such as Databricks, AWS EMR, Azure Synapse Analytics, or GCP Dataproc.

• Experience with Microsoft Fabric Spark workloads.

• Strong knowledge of SQL and experience with relational and NoSQL databases.

Perks you enjoy at KMS Mexico

• Mexican law benefits
• 15 days of PTO (in year zero, from the first year onwards it is 3 days per year).
• 5 days' leave for the death of immediate family members, negotiable.
• Major Medical Expenses Insurance with coverage for immediate dependents (spouse and children).
• Annual performance bonus (≈10% of annualized salary).
• Annual salary adjustment.
• Employee Referral Bonus.
• Paid Certifications / Courses
• Coursera License.
• 5% Savings Fund.
• 5% Grocery Vouchers.