Software Engineer, AI Infrastructure

About Harell Data
AI has transformed digital industries, but progress in the physical sciences — drug discovery, materials science, climate modeling — is stuck. The bottleneck isn't compute or algorithms. It's data. The most valuable scientific datasets are locked away in silos, unstructured, and hard to access.
We're fixing that. Harell Data is a managed platform where organizations can securely share proprietary datasets and train models on high-performance GPUs — all in one place. Dataset owners can share their data for model training without giving up control of it — trainers get access to compute against the data, not the data itself. Think of it as the infrastructure layer that turns scattered scientific data into domain-specific foundation models.

About the Role
As the founding AI Infrastructure Engineer, you will report directly to the CTO and lead the development of our core compute and orchestration layer. This is a high-impact role where you will hold a significant ownership stake in the company and lead the 0-to-1 build of our infrastructure. You will work closely with our customers to translate their needs into a world-class platform, while simultaneously shaping our engineering culture and technical direction from the ground up.

What You Will Do

• Build the GPU compute layer. Design and manage orchestration for GPU workloads — resource allocation, cost management, support for large-scale training, fine-tuning, and inference.
• Design developer interfaces. Build SDKs and APIs that make complex ML workflows feel simple for researchers and data scientists.
• Own the ML pipeline end-to-end. Data ingestion, preprocessing, model serving, monitoring — build the systems that tie it all together.
• Work directly with customers. Debug fine-tuning jobs, build observability tooling, and track model performance and resource health in real time.
• Shape technical direction. Lead build-vs-buy decisions on infrastructure and security. Set engineering standards. Help hire the team you want to work with.

Qualifications

• 5+ years in software engineering, focused on ML infrastructure or backend systems supporting ML workloads
• Production experience with ML/DL training or inference pipelines (PyTorch, Hugging Face, or similar)
• Hands-on Kubernetes on AWS or GCP, ideally with GPU workloads
• Strong CS fundamentals and system design chops
• Comfortable with ambiguity — you've worked somewhere where the playbook didn't exist yet