Data Engineer (Must have professional experience with Spark and Python)

  • Full-time

Company Description

Are you a fun, spirited, hardworking individual with a can-do attitude? Then, join our team of successful employees. The entire team at Luth Research takes pride in hiring diverse individuals of all backgrounds. We believe what makes a top-notch employee is personality, attitude, and work ethic combined with solid skills and experience.

Located in the heart of beautiful sunny San Diego, Luth Research has been a leading market research business for over 40 years, growing rapidly in the process by advancing next generation consumer intelligence with innovative market research approaches anywhere from data collection to digital tracking. Thus, our commitment to innovation and technology is driving much needed change in the market research industry. While providing multiple opportunities for growth from within, what we offer you is an amazing chance to advance your career. Luth Research seeks team members that model our Core Values: Passionate and Driven, Delivering Excellence, Embracing and Driving Change, Career Growth and Development, and Sustaining a Connected Fulfilled Team. Together, our vision is to connect millions of minds and devices worldwide to inspire business decisions.

Luth Research offers full benefits including medical, dental, vision, life, 401K, paid time off, paid holidays, and a flexible and fun work environment.

Job Description

Luth Research seeks a Data Engineer that will be developing repeatable, documented, and data-oriented engineering solutions via automation, process improvements, or other similar means while generating improved workflow, user experiences, efficient communication, growth, scalability, increased revenue, and profitability. The Data Engineer will be working in a fast paced and demanding environment and will report directly to the SVP of Digital Operations. If you have experience as a Data Engineer and are interested in working collaboratively in a growth environment, apply to become part of our dedicated team today.

Essential Job Functions and Responsibilities:

  • Develop code to support assigned projects and core company initiatives in a clear, concise, and object-oriented way containing comments and following standard software development principles/procedures in order to achieve maximum business growth, scalability, profitability, and accuracy.
  • Work with Management, Development, and Data teams to develop, architect, implement, and deploy systems and processes supporting company goals listed above in the Position Purpose.
  • Build QA checks and planning into the development process and actively work with management, data, and QA teams in ensuring all releases are bug free and ensures quality and usability across the organization to drive growth and profitability.
  • Support Management in creating technical documentation and test plans to ensure features are documented and progress is communicated clearly and concisely across the organization to ensure cohesion and maximum effectiveness in testing, launching, and understanding newly developed/enhanced products and services in order to increase client satisfaction.
  • Actively participate in discussions with Management, Development, and Data teams to provide suggestions on direction and architecture of upcoming projects to ensure the team is communicating with each other in a professional matter in order to promote cohesion, comprehension, and a clear goal to work towards which would lead to increased productivity and accuracy.
  • Work effectively across various data systems (i.e., OLTP databases, Data warehouses, HDFS, etc.) using specific programming languages (i.e., Python, SQL and SPARK) to best support the team and provide efficient and productive strategies to ensure quality and accuracy.
  • Define data quality plans to address data quality issues; work with business users to build business understanding of data issues to promote accuracy, communication, and client satisfaction along with increased revenue.
  • Create well-written, clear, concise, and accurate document artifacts including functional specifications, mapping documents process diagrams test plans and test scripts for the purposes of data warehouse construction, report development, verification etc. to allow for consistency and proficiency within the organization.
  • Practice and demonstrate exemplary work ethics, self-motivation in learning and taking on new challenges to support sales for the company to strengthen team culture and promote growth and profitability.
  • Any special projects as requested by senior management that would aid the company in efforts to support one another as a team and provide customer satisfaction in a timely fashion.
  • Act as support and a liaison with sales team to provide insight on areas of possible growth and development for clients.
  • Any other tasks and projects as assigned by management.

Qualifications

Education, Experience & Skills:

  • BS Degree or equivalent work-related experience.
  • 2-3 years of experience in IT as an Engineer, Analyst, or SQL Developer using SQL, Python and/or Spark.
  • Familiarity with core technology required: Hadoop/HDFS, CLIs such as Bash, SQL, Python and Spark.
  • Solid understanding of SQL, stored procedures and ad-hoc query development in Postgres environments.
  1. UNION, INTERSECT, EXCEPT clauses.
  2. Nested SQL queries.
  3. Windowing queries (postgres has a distinct flavor).
  4. Pivot queries (crosstab function).
  5. INNER, LEFT, and OUTER JOINs.
  6. EXPLAIN and how to interpret query plans.
  7. Compound indexes and query optimization for compound indexes.
  8. Data loading and dumping.
  9. Foreign key constraints as they relate to data manipulation.
  10. Hashing for id generation.
  11. Understanding of triggers and functions.
  • Ability to write requirements to extract data from existing databases, develop transformation rules based on business requirements gathered from non-technical people and test them.
  • Comprehensive understanding of relational databases and the data in them.
  • Conceptual understanding of OLTP vs Data Warehousing.
  • Experience with Data Conversions, Data Profiling and Root Cause Analysis a plus.
  • Understanding of digital media analytics and reporting is a plus.
  • An understanding of Business Objects, Tableau, Excel pivot tables is a plus.
  • Demonstrates an ability to think outside the box and take initiative upon themselves in order to solve problems as they arise.
  • Flexible schedule with the ability to work some nights and weekends.

 

For more information about Luth Research, please visit our website at www.luthresearch.com. 

Luth Research is committed to the principles of Equal Employment Opportunities and to making employment decisions based on merit and value. We will comply with all Federal, State and Local laws providing Equal Employment Opportunities, as well as all laws related to terms and conditions of employment.

Additional Information

All your information will be kept confidential according to EEO guidelines.