Data Engineer (Must have professional experience with Spark and Python)

  • 1365 Fourth Avenue, San Diego, CA, United States
  • Full-time

Company Description

Are you a fun, spirited, hardworking individual with a can-do attitude? Then, join our team of successful employees. The entire team at Luth Research takes pride in hiring diverse individuals of all backgrounds. We believe what makes a top-notch employee is personality, attitude, and work ethic combined with solid skills and experience.

Located in the heart of beautiful sunny San Diego, Luth Research has been a leading market research business for over 40 years, growing rapidly in the process by advancing next generation consumer intelligence with innovative market research approaches anywhere from data collection to digital tracking. Thus, our commitment to innovation and technology is driving much needed change in the market research industry. While providing multiple opportunities for growth from within, what we offer you is an amazing chance to advance your career. Luth Research seeks team members that model our Core Values: Passionate and Driven, Delivering Excellence, Embracing and Driving Change, Career Growth and Development, and Keeping a Connected Fulfilled Team. Together, our vision is to connect millions of minds and devices worldwide to inspire business decisions.

Luth Research offers full benefits including medical, dental, vision, life, 401K, paid time off, paid holidays and a flexible and fun work environment.

Job Description

Luth Research seeks a Data Engineer that directly support the organization’s business goals of growth, profitability, accuracy, and quality through developing repeatable, documented, and data-oriented engineering solutions that provide growth and scalability via automation, process improvements, or other similar means. The Data Engineer will be working in a fast paced and demanding environment and will report directly to the VP of Online Operations. If you have experience as a Data Engineer and are interested in working collaboratively in a growth environment, apply to become part of our dedicated team today.

Essential Job Functions and Responsibilities:

  • Develop code to support assigned projects and core company initiatives in a clear, concise, and object oriented way containing comments and following standard software development principles/procedures in order to achieve maximum business growth, scalability, profitability, and accuracy.
  • Work with Management, Development, and Data teams to develop, architect, implement, and deploy systems and processes supporting company goals related to business growth, scalability, profitability, and accuracy.
  • Build QA checks and planning into the development process and actively work with management, data, and QA teams in ensuring all releases are bug free and ensures quality and usability across the organization to drive growth and profitability.
  • Support Management in creating technical documentation and test plans to ensure features are documented and progress is communicated clearly and concisely across the organization to ensure cohesion and maximum effectiveness in testing, launching, and understanding newly developed/enhanced products and services in order to increase client satisfaction, revenue, and profitability.
  • Actively participate in discussions with Management, Development, and Data teams to provide suggestions on direction and architecture of upcoming projects to ensure the team is communicating with each other in a professional matter in order to promote cohesion, comprehension, and a clear goal to work towards which would lead to increased productivity, profitability, accuracy, and growth.
  • Work effectively across various data systems (i.e., OLTP databases, Data warehouses, HDFS, etc) using specific programming languages (i.e., Python, SQL and SPARK) to best support the team and provide efficient and productive strategies to ensure quality, accuracy, growth and profitability.
  • Define data quality plans to address data quality issues; work with business users to build business understanding of data issues to promote accuracy, communication, and client satisfaction along with increased revenue, growth, and profitability.
  • Create well-written, clear, concise and accurate document artifacts including functional specifications, mapping documents process diagrams test plans and test scripts for the purposes of data warehouse construction, report development, verification etc. to allow for consistency and proficiency within the organization promoting speediness, growth, and profitability.
  • Practice and demonstrate exemplary work ethics, self-motivation in learning and taking on new challenges to support sales for the company to strengthen team culture and promote growth and profitability.
  • Any special projects as requested by senior management that would aid the company in efforts to support one another as a team, provide customer satisfaction in a timely fashion, and generate growth and profitability.


Education, Experience & Skills:

  • BS or comparable work experience.
  • 2-3 years of experience in IT as an Engineer, Analyst, or SQL Developer using SQL, Spark, and Python.
  • Familiarity with core technology required: Hadoop/HDFS, CLIs such as Bash, SQL, Spark, and Python.
  • Solid understanding of SQL, stored procedures and ad-hoc query development in Postgres environments:
  2. Nested SQL queries.
  3. Windowing queries (postgres has a distinct flavor).
  4. Pivot queries (crosstab function).
  6. EXPLAIN and how to interpret query plans.
  7. Compound indexes and query optimization for compound indexes.
  8. Data loading and dumping.
  9. Foreign key constraints as they relate to data manipulation.
  10. Hashing for id generation.
  11. Understanding of triggers and functions.
  • Ability to write requirements to extract data from existing databases, develop transformation rules based on business requirements gathered from non-technical people and test them.
  • Comprehensive understanding of relational databases and the data in them.
  • Conceptual understanding of OLTP vs Data Warehousing.
  • Experience with Data Conversions, Data Profiling and Root Cause Analysis a plus.
  • Understanding of digital media analytics and reporting is a plus.
  • An understanding of Business Objects, Tableau, Excel pivot tables is a plus.
  • Demonstrates an ability to think outside the box and take initiative upon themselves in order to solve problems as they arise.
  • Flexibility in hours including some evening and weekend hours.

Luth Research is committed to the principles of Equal Employment Opportunities and to making employment decisions based on merit and value. We will comply with all Federal, State and Local laws providing Equal Employment Opportunities, as well as all laws related to terms and conditions of employment.

Additional Information

All your information will be kept confidential according to EEO guidelines.

Videos To Watch