Avatar for Boz AI
Boz AI
Actively Hiring
A personal AI platform

Local LLM / Inference Engineer — llama.cpp / Rust

Posted: 1 day ago• Recruiter recently active
Hires remotely in
Remote Work Policy

Remote only

Company Location
Visa Sponsorship

Not Available

Preferred Timezones
Pacific Time
RelocationNot Allowed
Skills
SQLite
Rust​
LLMs
Libraries: Langchain, LLama-Index, NumPy, Scikit-Learn, Pandas, Matplotlib, Seaborn,
Inference
LLM Frameworks (Langchain, Claude, LLamaIndex) RAG Technologies Embedding Models Vect
RAGs, ChatGPT, Hugging Face, LangChain, LlamaIndex, Transformers, VectorDB

About the job

We are an early-stage AI startup building a new kind of privacy-first personal AI platform designed to run primarily on the user's own computer.

We're looking for an experienced engineer with deep, hands-on expertise working with local, open-weight large language models. This is not primarily a model-training, cloud AI, or LLM API integration role. We're looking for someone who understands the practical realities of running LLMs locally and getting the best possible performance, quality, and reliability from them.

Our ideal candidate has substantial real-world experience with llama.cpp / llama-server and enjoys working close to the inference stack.

We're flexible about the working relationship and are open to full-time, part-time, or contract arrangements with the right person. Finding someone with the right expertise and enthusiasm for the problem is more important to us than the particular arrangement.

San Francisco Bay Area candidates are strongly preferred. We work primarily remotely, but we'd like someone local enough to collaborate in person periodically.

About the company

Boz AI company logo

Boz AI

Actively Hiring
A personal AI platform1-10 Employees
Learn more about Boz AI image

Founders

Brazos Donaho
Founder
image
View the team image

Similar Jobs

Albeado company logo
Albeado
Breakthrough causal AI predictions, optimizations and interventions - in real time
Scale AI company logo
Scale AI
Accelerate the development of AI applications
Highnote company logo
Highnote
Highnote is the world's most modern card issuance platform
tribe.ai company logo
tribe.ai
We embed elite AI engineers to ship real, production-grade AI for enterprises
IonQ company logo
IonQ
Building the world’s best quantum computers to solve the world’s most ecomplex problems