Avatar for 3Pillar Global
Product Development Services
  • B2B
  • Scale Stage
    Rapidly increasing operations
  • 4.1
    Work / Life Balance
    Employees rate 3Pillar Global 4.1/5 on Glassdoor for work / life balance

Data Architect with Databricks/Snowflake and AI experience

Posted: 4 weeks ago
Job Location
Remote Work Policy

In office - WFH flexibility

Visa Sponsorship

Not Available

RelocationAllowed
Skills
Python
SQL
Neo4J
Snowflake
Docker
AWS S3
AWS Redshift
Kubernetes
Terraform
Aws Kinesis
databricks
Pinecone
AWS EKS
GitHub Actions
AWS Glue
Faiss
OpenSearch
LangChain
Llamaindex
Chromadb
AWS Bedrock
ETL Processes
Embedding Pipelines
Chunking Strategies
LLM APIs (OpenAI, AWS Bedrock, Claude, HuggingFace)
Real-Time Streaming (Kafka, Spark Structured Streaming)
Batch Processing (Databricks, PySpark, Delta Lake)
Cloud Storage and Compute (AWS, Azure)
Data Quality /metadata Management
Hybrid Retrieval Layers Combining Semantic Search with Structured Queries

About the job

3Pillar is an AI transformation partner on a mission to help enterprises build the AI-native products and intelligent agents that will define the next era of business. With teams across North America, Europe, Latin America, and Asia, we work with the most ambitious companies in financial services, healthcare, media, and technology — helping them move faster, modernize boldly, and compete on their own terms. Our HelixAI platform and Helix Pods delivery model put our engineers at the center of real agentic transformation — doing work that is open, portable, and built to last. We are building the future of enterprise AI.Data Architect with DataBricks/Snowflake and AI ExperienceWe are looking for an AI Data Architect to design, build, govern, and evolve the single source of truth that powers every AI initiative in our organization.Requirements:Architect and own the enterprise AI data platform — the unified, governed layer that ingests, transforms, stores, and serves all data consumed by AI systems across the organisation.Design multi-domain data models (lakehouse, data mesh, event-driven) that are structured from day one to serve AI workloads: clean lineage, versioned schemas, well-documented contracts, and low-latency serving APIs.Strong exposure to different Data architectures, data lake & data warehouseDefine tools & technologies to develop automated data pipelines, write ETL processes, develop dashboard & report and create insightsTechnical SkillsPrimary Skills: Python, SQL, Snowflake/Databricks, AWS (S3, Glue, EKS, Bedrock, Kinesis, Redshift), Docker, Kubernetes, Terraform, GitHub Actions, LangChain, LlamaIndex, LLM APIs (OpenAI, AWS Bedrock, Claude, HuggingFace), (Pinecone, FAISS, ChromaDB, OpenSearch), knowledge graphs (Neo4j).Secondary Skills: MLflow, FastAPI, CI/CD pipelines, observability tooling (CloudWatch, Grafana, or equivalent), data lineage and metadata management platforms.Requirements:12+ years of hands-on data engineering and architecture experience, alongside building production AI/ML and LLM-era data infrastructure.Strong Experience with either Databricks or Snowflake; experience with both is desirable.Strong data architecture patterns & principles, ability to design secure & scalable data lakes, data warehouse, data hubs, and other event-driven architecturesExpertise in designing and writing ETL processes in Python / Java / ScalaOwn the full data stack: real-time streaming (Kafka, Spark Structured Streaming), batch processing (Databricks, PySpark, Delta Lake), cloud storage and compute (AWS, Azure), and data quality /metadata management.Drive modernisation of legacy pipelines (on-prem ETL, batch DWH) to cloud-native, AI-ready architectures with measurable improvements in cost, latency, and delivery velocity.Proven experience designing enterprise-scale AI data platforms that serve multiple AI consumers —not just one application or pipeline.Hands-on experience with vector stores, semantic models, knowledge graphs, and retrieval infrastructure in production environments.Working knowledge of LLMOps: model serving pipelines, MLflow, CI/CD for AI, automated evaluation, and production monitoring.AI ExperienceRAG, Vector & Retrieval InfrastructureDesign the retrieval infrastructure that powers RAG-based AI applications: embedding pipelines, vector stores (Pinecone, FAISS, ChromaDB, OpenSearch), chunking strategies, and hybrid retrieval layers combining semantic search with structured queries.BenefitsMedical Insurance benefits as per company policy.Dental insurance as per company policy.Vision insurance as per company policy.Employer paid Disability, Life, and AD&D insuranceUnlimited PTOPaid parental leave401KFlexible work policy12 Paid Holidays

About the company

3Pillar Global company logo
Product Development Services501-1000 Employees
  • B2B
  • Scale Stage
    Rapidly increasing operations
  • 4.1
    Work / Life Balance
    Employees rate 3Pillar Global 4.1/5 on Glassdoor for work / life balance
Learn more about 3Pillar Global image

Funding

AMOUNT RAISED
$25.9M
FUNDED OVER
4 rounds
Rounds
A
$12000000
Series A - Jun 2013+3

Founders

Shane Krider
President & Founder • 17 years
Phoenix
image
David DeWolf
Founder
Washington DC
image
View the team image

Similar Jobs

Enigma Technologies company logo
Enigma Technologies
Authoritative answers for your agents and workflows
Forus company logo
Forus
Accelerate the future of medicine
Clipboard company logo
Clipboard
Flexibility and high pay for professionals. Top talent and fast fill for workplaces
Blue Ink Digital company logo
Blue Ink Digital
Automated Marketplace for real-time exclusive leads
AARATECH company logo
AARATECH
AaraTech Inc. empowers businesses by providing innovative and tailored IT solutions and services
EquityZen company logo
EquityZen
Private Markets for the Public // Pre-IPO Stock Marketplace
Iterative Health company logo
Iterative Health
Pioneering the use of artificial intelligence-based solutions for gastroenterology