Databricks Data Engineer
USA · Remote · Full-Time
Job description
Develop reliable data pipelines and analytics foundations using Databricks, PySpark, SQL and AWS. This role focuses on turning source data into well-tested datasets that engineering and analytics teams can use confidently.
Key responsibilities
- Build and maintain batch and streaming data pipelines in Databricks using PySpark and SQL.
- Ingest and transform data from files, databases and APIs into documented data models.
- Implement data-quality checks, reconciliation, exception handling and operational monitoring.
- Investigate failed jobs, tune distributed workloads and improve pipeline reliability and performance.
- Collaborate with analysts and engineers to define requirements, review changes and support releases.
- Maintain technical documentation, source control and repeatable deployment practices.
Skills and experience
- Practical experience with Databricks, Apache Spark, Python and SQL.
- Understanding of distributed processing, data modelling and incremental data loading.
- Experience working with cloud storage and data services on AWS.
- Ability to troubleshoot data issues and explain findings to technical and business colleagues.
- Familiarity with access controls, testing and version-controlled development.
Additional experience
- Exposure to Delta Lake, workflow orchestration and CI/CD for data pipelines.
- Experience optimizing cloud data workloads for performance and cost.
Career enquiries
Use the career enquiry form on this page and include the role title, your relevant skills and preferred location in your message.