Avatar for Entelligence.ai
Building artificial engineering intelligence

Research Engineer

Posted: 6 days ago
Job Location
Remote Work Policy

In office

Visa Sponsorship

Not Available

RelocationAllowed
Skills
Python
Agents
LLM APIs
Batching
Prompt Caching
Token Pricing
Tool-Use Loops
LLM Inference Economics

About the job

Research Engineer, Coding Agent Optimization EngineSan Francisco, in person · Full-time · EntelligenceCoding agents ship more code every week, and they fail in repeatable ways: the same wrong assumption, the same missed test, the same review comment on PR after PR. Our Coding Agent Optimization Engine closes that loop. It mines agent sessions and PR review comments for failures, traces them to where the harness went wrong, and feeds the lessons back so the next session does not repeat them. It also routes every turn to the most efficient model that can solve it. It runs on live traffic from Claude Code, Codex, Cursor and API clients. We want a research engineer to make it measurably better at both.Key Focus AreasFailure mining. Captures coding-agent sessions and links them to the PRs they produced, then mines transcripts and review comments for errors, struggles and recurring root causes.Learning loop. Turns review findings into root-cause patterns and tested rules, and delivers them to the next agent session as guidance cards through CLI hooks.Pre-PR judge. A reviewer model checks an agent's change before the PR is opened, and the harness reports back what happened in each session.Cost-aware routing. Picks a difficulty tier per instruction turn and weights models by expected cost per solved task, with prompt-cache affinity, failover and per-org constraints.What you will work onFailure attribution. Given a session transcript and the review comments on its PR, find where the harness went wrong: missing context, a bad tool loop, a skipped check, the wrong model.Prevention that measurably works. Decide which lessons become rules, cards or judge checks, and prove with held-out data that they cut repeat findings without flooding agents with noise.Evaluation. Own the benchmark harness of record: replay real agentic tasks headlessly against competing harness configs and routing policies, and measure quality vs. cost honestly.Engineering world model. Build the comprehensive data layer connecting sessions, PRs, projects, errors and incidents. Leverage the world model to catch issues from happening in future sessions.Cost and outcome modeling. Turn measured usage, cache shares, latency and review outcomes into continuously refreshed model and harness parameters.Your Background3+ years building ML or LLM systems that run in production, or equivalent research experience with shipped code.Strong Python and comfort reading a large production codebase.Experiment design you can defend: held-out sets, variance, confidence intervals, leakage checks.Hands-on experience with LLM APIs, agents or tool-use loops, and their failure modes.Work on agent evaluation, failure analysis, LLM-as-judge or automated code review.Work on routing, cascades, bandits or learning-to-defer.Experience with LLM inference economics: prompt caching, batching, token pricing.Built or maintained public evals (SWE-bench, Terminal-Bench style harnesses).Published research or well-known open-source work in ML systems or developer tools.Compensation and benefitsBase salary: $170,000 - $210,000 plus meaningful equity. Final offer depends on level and experience. Healthcare and a generous compute budget for experiments.

About the company

Entelligence.ai company logo
Building artificial engineering intelligence11-50 Employees
Company Size
11-50
Company Industries
B2B · SaaS · Mobile · Artificial Intelligence / Machine Learning
Learn more about Entelligence.ai image

Founders

Aiswarya Sankar
Founder
image
View the team image

Similar Jobs

GVOS  company logo
GVOS
An Edge Cloud for Autonomous Driving
Deepgram company logo
Deepgram
AI speech API for transcription with human-level understanding
Archesys company logo
Archesys
Improving the government services that impact everyday lives
Scale AI company logo
Scale AI
Accelerate the development of AI applications
Pulley company logo
Pulley
Helping project teams break ground faster
Rocket Money company logo
Rocket Money
The Money App That Works for You
Tatari company logo
Tatari
Buy and measure ads for brands across linear & streaming TV