ML Systems Engineer (Intern) — Model Serving & Runtime

ML Systems Engineer — Model Serving & Runtime

Compensation: Equity + Revenue Share
Type: Full-time

Remote: Optional

Own the end-to-end local inference stack: make models run fast and reliably on Windows (x64/ARM64) and macOS (Apple Silicon). You decide which engine (MLX, llama.cpp, ONNX Runtime, TensorRT-LLM) runs where, how models are quantized and packaged, and how the runtime talks to the Tauri frontend. You collaborate with AI Research on quantization strategy, kernel selection, and speculative decoding—and you ship the runtime that makes it all work.

Responsibilities

• Model Serving Layer: Build and maintain the inference runtime for Windows and macOS desktop apps
• Engine Integration: Integrate MLX (Apple), llama.cpp (cross-platform), ONNX Runtime, TensorRT-LLM, vLLM; evaluate emerging engines
• Hardware-Aware Dispatch: Metal/MPS on macOS; CUDA/DirectML/Vulkan on Windows; CPU fallback with optimal threading
• Model Packaging & Conversion: Own GGUF, MLX weights, Safetensors, Core ML pipelines; automated conversion and validation
• Quantization & Optimization: Implement GPTQ, AWQ, GGML, EXL2 pipelines; profile latency/memory/battery across device tiers
• Download & Update Pipeline: Sidecar binary delivery, checksum verification, staged rollouts, delta updates for model weights
• Cross-Compilation CI/CD: Multi-arch builds (rustup targets, Xcode, MSVC, clang), device-farm testing, regression detection
• Compatibility Matrices: Document supported models × OS × hardware; migration guides for new model releases
• AI Research Collaboration: Partner on quantization strategy, kernel fusion, speculative decoding, attention optimization
• Debug Platform Issues: Metal shader compilation, CUDA driver regressions, ARM64 Windows quirks, memory mapping edge cases

Qualifications

• 3+ years shipping native desktop applications on Windows and macOS
• Strong proficiency in Rust (Tauri backend), C++ (llama.cpp), Swift/Obj-C (MLX)
• Deep understanding of inference pipelines: tokenization, KV cache, sampling, speculative decoding, attention kernels
• Hands-on experience with MLX, llama.cpp, or equivalent local inference engines
• Experience with cross-compilation toolchains and multi-arch CI/CD (GitHub Actions, self-hosted runners)
• Fluency with GPU compute APIs: Metal/MPS, CUDA, DirectML, Vulkan
• Fluency with model formats: GGUF, Safetensors, ONNX, Core ML, MLX arrays
• Systems programming: memory mapping, shared libraries, dynamic loading, zero-copy buffers

Preferred

• Contributions to llama.cpp, MLX, ONNX Runtime, or similar open-source inference projects
• Shipped a local-first AI application or model runner
• Experience with quantization tooling (llama.cpp quantize, GPTQ, AWQ, EXL2)
• Familiarity with Tauri v2 architecture (Rust backend + WebView frontend)
• Background in compiler/runtime optimization (TVM, XLA, MLIR a plus)

Tauri - rust

(React 19, TS, Lexical, custom grid, Recharts). Own the rendering layer: SVG shapes, theme-aware charts, and one shared selection/interaction system across the canvas, diagram, and shape editors.

Work

• Shared drag/resize/rotate + keyboard selection for all editors
• Extend token-driven chart theming and the SVG shape layer
• Design pass on pages/blocks (spacing, type, empty states)
• Print/preview layout polish

Skills

• React + TypeScript; strong SVG/canvas rendering
• Design-systems experience (CSS custom-property tokens, light/dark)
• Built or worked deep in an editor (Lexical, Slate, Draft, ProseMirror)
• Comfortable with the UI detail work
• Nice-to-have: Recharts/D3, Mermaid, dagre, Tauri