Lead Data Engineer Job at WorkHQ, Los Angeles, CA

dm1yUHVqbUE4U3JBd2s2Umg5S2lGaW9CdVE9PQ==
  • WorkHQ
  • Los Angeles, CA

Job Description

Company Context

Series A, well-funded US startup in HRTech developing WorkHQ.com and an AI Recruiter product.

This is a US-only, Remote role (Mainland).

Role Overview

Lead data infrastructure architect managing billions of data points across 250M+ professional profiles.

Hire data engineers to aid you in that journey.

Core Responsibilities

  • Design scalable data pipelines processing massive record volumes

  • Architect ETL processes using PySpark on Amazon EMR (Open to shifting to other solutions like Data Bricks / Snowflake)

  • Distribute enriched data through medallion architecture across Postgres, Athena, OpenSearch

  • Integrate new data sources into the main pipeline

  • Implement advanced data matching using Splink

Technical Requirements

  • 5-8 years professional data engineering experience

  • Good proficiency in:

    • PySpark and distributed computing

    • AWS data services (EMR, Glue, Athena)

    • Docker

    • Pandas and DataFrame manipulation

    • Complex data format handling (JSONL, Parquet)

  • Strong background in:

    • Big data processing architectures

    • Data warehouse design

    • Performance optimization

  • Advanced Python, SQL skills

Nice to Have

  • Probabilistic record linking expertise

  • OpenSearch/elasticsearch technologies

  • Machine learning data pipeline design

  • Recruitment tech ecosystem knowledge

Technical Stack

  • Big Data: PySpark, EMR

  • Databases: Postgres, OpenSearch

  • Cloud: AWS

  • Containerization: Docker

  • Data Formats: JSONL, Parquet

  • Analytics: Metabase, Athena, Glue

  • Data Processing: Pandas, Splink

Other Considerations

While this role has specific requirements - if you lack a few technical skills, but motivated to learn and lead the platform, please apply for consideration.

If you are coming from Director/Head of/VP levels that is relevant to this job, you can apply as well.

You will need to apply directly on our platform.

Thank you for your time.

Job Tags

Permanent employment, Remote job, Shift work,

Similar Jobs

Buildofarm

Client Onboarding Specialist Job at Buildofarm

 ...Buildofarm is seeking a highly motivated and communicative Client Onboarding Specialist to join our team. In this remote role, you will be the first point of contact for new clients, guiding them through the onboarding process via Zoom and providing a clear breakdown of... 

The Roof Mentors

Canvasser Job at The Roof Mentors

 ...does not work hard for you, then come join a team that wants to see your success as much as you do. We are seeking a motivated Canvasser to join our team! Responsibilities: This is a door to door canvassing position Identify and point out damages to homeowners... 

Insight Global

Mac/ iOS Mobile App Developer (Swift, C, and C++) Job at Insight Global

 ...VPN Personal VPN Troubleshoot and resolve customer escalations. Minimum Qualifications Expert in Swift 5.0, Objective C AND/ OR C++ Expert in MAC/ iOS network extension framework and stack and device management Basic to good knowledge on Network stack... 

Accolade Technologies, LLC.

Journeyman Geolocation Operator Job at Accolade Technologies, LLC.

*APPLICANTS MUST HAVE AT LEAST a TS/SCI with Maryland Polygraph* Required Qualifications: Current certifications in basic operations, passive and active geolocation methods and technologies Possess formal training and/or demonstrated proficiency in relevant foreign... 

Pantaq

Independent Sales Representative (1099) - Dental & Private Clinics Job at Pantaq

 ...We are seeking a dynamic and self-driven Independent Sales Representative (1099 Contractor) to sell our premium medical and dental consumables to dental clinics, private medical clinics, and healthcare facilities in the Houston area. This is a high-commission, flexible...