Avatar for Saksoft
Digital transformation and consulting services

Azure Databricks Platform Administrator

  • Raritan
  • |5 years of exp
  • |Full Time
Posted: 3 weeks ago
Job Location
Raritan
Remote Work Policy

In office

Visa Sponsorship

Not Available

RelocationAllowed
Skills
Python
SQL
Git
Authentication
REST APIs
Azure
Shell Scripting
PowerShell
Apache Spark
Jobs
Terraform
CICD
databricks
Azure Data Lake
Azure DevOps
Azure Databricks
Azure Key Vault
Delta Lake
Compute
Azure SQL
ARM Templates
Azure Monitor
Bicep
Azure CLI
Azure Virtual Network
Azure Policy
Unity Catalog
Databricks CLI
Microsoft Entra ID
Log Analytics
Azure Data Lake Storage Gen2
Azure Private Link
Audit Logs
SQL Warehouses
Databricks Asset Bundles
Cluster Policies
System Tables
Workspace Administration
Databricks Account Console

About the job

We are seeking an experienced Azure Databricks Platform Administrator to design operate secure and optimize enterprisescale Databricks environments on Microsoft Azure The role is responsible for platform governance workspace administration identity and access management networking security monitoring automation cost optimization and production support The ideal candidate combines strong Azure infrastructure knowledge with handson Databricks administration and works effectively with data engineering analytics security cloud and application teams.Key ResponsibilitiesAdminister Azure Databricks accounts workspaces clusters cluster policies pools SQL warehouses jobs notebooks repositories libraries and access controls across development test and production environmentsManage workspace provisioning and configuration using Infrastructure as Code and automated deployment pipelinesImplement identity and access management through Microsoft Entra ID single signon service principals managed identities groups rolebased access control and SCIM provisioningAdminister Unity Catalog including metastores catalogs schemas tables volumes storage credentials external locations privileges and dataaccess governanceConfigure secure Azure networking including virtual network injection private endpoints Private Link network security groups route tables DNS firewalls and controlled egressIntegrate Databricks with Azure Data Lake Storage Key Vault Azure DevOps or GitHub monitoring platforms and other approved enterprise servicesEstablish cluster standards runtime policies library controls tagging autoscaling autotermination and workload isolationMonitor platform availability performance capacity audit logs job health and security events build dashboards and actionable sLead incident problem change release and servicerequest management for the platform meeting agreed service levels and operational KPIsTroubleshoot complex workspace compute authentication authorization connectivity storage Spark and integration issuesOptimize platform cost through usage analysis rightsizing policy enforcement commitment utilization chargebackshowback and removal of idle resourcesPlan and execute Databricks Runtime upgrades platform changes vulnerability remediation businesscontinuity testing and disasterrecovery activitiesDefine and maintain platform standards runbooks architecture diagrams operating procedures support documentation and knowledge articlesPartner with engineering architecture security compliance FinOps and vendor teams to onboard workloads and drive continuous improvementMentor junior administrators and provide technical leadership for platform operations and governance.Required Qualifications And Experience5 years of experience in cloud dataplatform infrastructure DevOps or productionsupport roles including substantial handson administration of Azure DatabricksStrong understanding of Azure Databricks architecture control plane and data plane concepts workspaces compute jobs SQL warehouses Delta Lake and Unity CatalogHandson experience with core Azure services such as Microsoft Entra ID Azure Data Lake Storage Gen2 Key Vault Virtual Network Private Link Azure Monitor Log Analytics and Azure DevOpsStrong knowledge of cloud security leastprivilege access secrets management encryption network isolation auditability and enterprise governanceExperience automating Azure and Databricks administration using Terraform Bicep or ARM templates Databricks Asset Bundles Azure CLI Databricks CLI REST APIs PowerShell Python or shell scriptingExperience building and supporting CICD pipelines for infrastructure and platform configurationWorking knowledge of Apache Spark Python SQL Delta Lake and common dataengineering workload patternsStrong troubleshooting skills across identity networking compute storage security and distributed dataprocessing layersExperience operating production platforms using ITILaligned incident problem change release capacity and servicelevel management practicesBachelors degree in Computer Science Information Technology Engineering or a related discipline or equivalent practical experience.Technical CompetenciesDatabricks Account Console workspace administration compute cluster policies jobs SQL warehouses Unity Catalog Delta Lake audit logs system tables authentication and REST APIsAzure Entra ID ADLS Gen2 Key Vault Virtual Network Private Link Azure Monitor Log Analytics Azure Policy and Azure DevOpsAutomation and DevOps Terraform Bicep or ARM Git CICD Databricks Asset Bundles PowerShell Python shell scripting and com.SkillsMandatory Skills : AZURE DATA LAKE, Azure SQL, Databricks