ETRM Databricks Data Engineer

Houston
|10 years of exp
|Full Time

Posted: 1 week ago• Recruiter recently active

Job Location

Houston

Remote Work Policy

In office

Visa Sponsorship

Not Available

RelocationAllowed

Skills

Python

SQL

Git

Azure

Apache Spark

Pyspark

databricks

Azure Cloud Platforms

About the job

Job Description:

Position: ETRM Databricks Data Engineer

Location: Houston, Texas (5 days on-site)

Hire Type: Full Time

Experience: 10+ Years

ETRM Databricks Data Engineer to design, develop, and maintain scalable data solutions that support business intelligence, analytics, and data-driven decision-making with understanding of Data Domain & Data Assets in ETRM Domain. The ideal candidate will have strong expertise in Databricks, Apache Spark, Azure Cloud platforms, and modern data engineering practices.

Key Responsibilities:

Design, develop, and optimize data pipelines using Databricks & ADF.
Must possess good understanding of Trading Domain knowledge.
Build and maintain scalable ETL/ELT processes for structured and unstructured data.
Integrate data from multiple sources into enterprise data platforms.
Develop and manage data lakes, lakehouses, and data warehouses.
Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.
Ensure data quality, integrity, security, and governance across data platforms.
Monitor and troubleshoot data processing jobs and optimize performance.
Implement CI/CD practices and automate data engineering workflows.
Create and maintain technical documentation, data models, and process flows.
Support cloud-based data solutions and migration initiatives.

Required Qualifications:

Hands-on experience with Databricks and Azure.
Strong proficiency in Python, SQL, and PySpark.
Experience with cloud platforms such as Azure
Knowledge of data warehousing, data modeling, and ETL/ELT concepts.
Experience with version control tools such as Git.
Strong analytical, problem-solving, and communication skills.
Trading Domain experience related for Crude, Refined Products, Natural gas is a must.

Preferred Qualifications:

Experience with Azure Data Factory, Azure Synapse Analytics, Delta Lake, or similar technologies.
Knowledge of streaming data processing and real-time analytics.
Familiarity with DevOps practices and CI/CD pipelines.
Databricks, Azure, or related cloud certifications.