Databricks Engineer Job at Conflux Systems, Phoenix, AZ

  • Conflux Systems
  • Phoenix, AZ

Job Description

Databricks Engineer

We are seeking a highly skilled Databricks Engineer to lead the design, implementation, and optimization of scalable cloud-based data platforms. This role requires deep expertise in Lakehouse architecture, Databricks, Apache Spark, and Delta Lake, along with proven experience delivering end-to-end enterprise data engineering solutions.

Key Responsibilities

  • Design and deliver end-to-end data engineering pipelines, including batch and real-time streaming solutions.
  • Lead implementation of:
  • Cloud-based data lakehouse platforms integrating diverse data sources.
  • Real-time data processing pipelines for operational and analytical use cases.
  • Develop scalable ETL/ELT pipelines using PySpark, Scala, and SQL.
  • Implement advanced data modeling solutions including 3NF, dimensional modeling, and enterprise data warehousing strategies.
  • Design and build incremental data loading frameworks and metadata-driven ingestion pipelines.
  • Establish data quality frameworks and governance standards.
  • Implement and manage Unity Catalog, including fine-grained security and access controls.
  • Leverage Databricks components such as:
  • Delta Live Tables
  • Autoloader
  • Structured Streaming
  • Databricks Workflows
  • Integration with orchestration tools (e.g., Apache Airflow)
  • Drive CI/CD automation, deployment strategies, and DevOps best practices.
  • Optimize performance of pipelines, Spark jobs, and compute resources.
  • Provide architectural guidance and technical leadership across cross-functional teams.
  • Engage with stakeholders and clients to translate business requirements into scalable technical solutions.

Deep expertise in:

  • Databricks and cloud-native storage/compute platforms
  • Apache Spark (batch & streaming)
  • Delta Lake & Lakehouse architecture
  • Distributed data processing systems
  • Strong hands-on programming skills in Python, PySpark, Scala, and SQL.

Job Tags

Similar Jobs

TruTeam

Insulation Installer Job at TruTeam

 ...distributor of roofing products; second-largest publicly traded distributor of lumber and building materials; and largest distributor of waterproofing products. QXO is the fastest growing company in the $800 billion building products distribution industry and plans to become the... 

Bruce's Tire & Auto Service

Senior Automotive Technician Job at Bruce's Tire & Auto Service

 ...mechanical repairs, such as brakes, suspension, and engine diagnostics. We are looking for good people that we can grow with. The Automotive Technician is responsible for effectively and efficiently diagnosing and repairing customer vehicles while adhering to the MAP... 

MaziCTools

Remote Transcriptionist: Entry-Level for Teens Job at MaziCTools

 ...entry-level. Candidates should have strong typing skills, attention to detail, and be...  ...scheduling, making it ideal for balancing work and personal commitments. Responsibilities...  ...confidentiality. Enjoy the convenience of working from home and gain valuable experience in the... 

Patriots medical transport

EMT-B Job at Patriots medical transport

 ...Job Description Job Description We are seeking an EMT-B to join our team!You will assess injuries, administer emergency medical care, and extricate trapped individuals. Responsibilities: Transport injured or sick individualsto medical facilities Determine... 

Freeport-McMoRan Inc.

Shovel Operator I Job at Freeport-McMoRan Inc.

 ...Shovel Operator I Requisition ID: 143603 Location: Bagdad, AZ, US, 86321 Category: Operations Share this Job Why You Should Apply For This Job: At Freeport-McMoRan, we are committed to providing an employment package that recognizes excellence, encourages...