Avatar for Cogniify
Cogniify
Actively Hiring
More jobs
Cogniify: Enterprise AI consulting and production system implementation

Databricks Solutions Architect - USA (Remote)

Posted: 1 week ago• Recruiter recently active
Hires remotely in
Remote Work Policy

Remote only

Company Location
Visa Sponsorship

Available

Preferred Timezones
Pacific Time
RelocationNot Allowed
Skills
Python
SQL
Git
Clustering
Data Architecture
Architect
ETL
Azure
Data Migration
Spark
Apache Spark
AWS
QUERY OPTIMIZATION
Pyspark
GCP
databricks
CI/CD
Azure Databricks
FinOps
Delta Lake
Medallion Architecture
Lakehouse Architecture

About the job

The Role
Cogniify is hiring a Databricks Solutions Architect (Databricks Architect) to plan and lead our Databricks lakehouse platforms. You will decide how our data is stored, cleaned, protected and used, and guide the team that builds it.
You will work with data engineers, product managers and business teams, from the first design to a live system.
You will also lead the move from older data warehouses to Databricks.
Much of our data work is in energy and utilities, such as smart meter, grid and outage data. Experience with this kind of data helps.
This is a remote role based in the United States, with some travel for workshops and key milestones.

Core skills for this role include Databricks, Apache Spark, PySpark, Delta Lake, Unity Catalog, Lakeflow, lakehouse architecture, AWS, Azure, Google Cloud (GCP), data migration, CI/CD, data governance and FinOps.

What You Will Do

  • Design lakehouse architectures on AWS, Azure or Google Cloud, using medallion layers, Delta Lake and Unity Catalog.
  • Set how data pipelines are built with Lakeflow, Apache Spark and Databricks SQL, so every engineer on the team builds in the same way.
  • Define data governance, security and access control, along with cloud cost controls (FinOps), so data stays safe and spending stays in check.
  • Lead the move from older data warehouses and ETL tools to Databricks, and make sure the new numbers match the old ones.
  • Set CI/CD and DataOps standards, including Git, Databricks Asset Bundles and automated testing, so releases are safe and repeatable.
  • Advise on machine learning and generative AI use cases, using tools such as MLflow and Agent Bricks where they make sense.
  • Mentor engineers, run architecture reviews, and explain designs and trade-offs to business leaders and other stakeholders in simple words.
  • Travel for workshops, planning sessions and key milestones when needed.

What We Are Looking For

  • A BS, MS or higher in Computer Science, Engineering, Information Systems or a related field, or equivalent practical experience.
  • At least 12 years of experience in data engineering or data architecture, with at least 4 years of hands-on work on Databricks and Apache Spark using PySpark and SQL.
  • At least 3 production Databricks implementations that you took from design to go-live.
  • Strong experience on at least one major cloud, AWS, Azure or Google Cloud (GCP).
  • Deep knowledge of Delta Lake, Unity Catalog, performance tuning and cluster sizing.
  • Strong leadership and communication skills, with a record of leading technical teams and working with business stakeholders in large, complex organizations.
  • Experience setting CI/CD and DataOps standards, including Git and automated testing.
  • Comfort working remotely and traveling when the work needs it.
  • Databricks certifications such as Data Engineer Professional, Machine Learning or Generative AI are a plus.
  • Databricks architect level or Champion level standing, or equivalent proven experience, is a plus.
  • Hands-on work with Lakebridge migrations, Lakebase or Genie is a plus.
  • Experience with streaming and real-time pipelines, and with data modeling, is a plus.
  • Experience with energy and utility data, such as smart meter, grid or outage data, is a plus.
  • Experience on large enterprise data programs is a plus.

Perks and Benefits of Working With Us

  • Unlimited PTO
  • Generous parental leave, well above industry standards
  • Medical, dental, and vision coverage for employees
  • Disability and life insurance
  • Mental health and wellbeing support
  • Annual bonus program
  • Employee Stock Purchase Plan (ESPP)
  • Yearly team-building experiences
  • Mentorship and sponsorship opportunities
  • Manager resources and support
  • Small teams where your work makes a real difference
  • Open communication with management and company leadership
  • An entrepreneurial culture where taking initiative is part of everyday work

Cogniify is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, veteran status, or any other protected characteristic.

Similar Jobs

GVOS  company logo
GVOS
An Edge Cloud for Autonomous Driving
Scale AI company logo
Scale AI
Accelerate the development of AI applications
tribe.ai company logo
tribe.ai
We embed elite AI engineers to ship real, production-grade AI for enterprises
Brex company logo
Brex
The AI-powered spend platform
Stuut company logo
Stuut
AI Co-Pilot for Accounts Receivable
Snout company logo
Snout
Affordable vet care, for every pet
Sigma Computing company logo
Sigma Computing
Sigma Computing is SaaS, next-generation business intelligence and data exploration platform