AI Reliability Engineer
My work sits between model capability and production consequence: evaluation harnesses, deterministic safety gates, fault injection, traces, replay, and the operational controls that make AI behavior inspectable.
I came to AI systems through manufacturing quality engineering, where correctness is not a style preference: a bad decision changes inventory, production, or accounting. That background shaped a simple standard for my work:
Build systems that fail loudly, preserve the evidence, and block unsafe action before a confident mistake reaches a system of record.
302
tests in FAULTLINE
400+
production invoices validated
50,000+
users served
99%+
platform uptime over 18 months
Flagship engineering work
Reliability workbench for document-grounded AI
FAULTLINE tests the machinery used to make reliability claims: deterministic subjects, independent oracles, labelled fault injection, crash-resistant traces, replay boundaries, detector scoring, and statistical comparisons. It deliberately does not claim that a synthetic stand-in proves a production agent is reliable.
Identical seeds produce byte-identical corpora and questions
Six fault classes use deterministic triggers and out-of-band labels
Required-source correctness is scored separately from answer shape
Trace reconstruction is attacked by deleting fields and whole spans
Secret redaction is verified inside captured failure messages
Rates carry Wilson intervals; paired changes use McNemar tests
Production evidence
LLM extraction feeds a deterministic validation gate before approved data reaches a live financial system. The pipeline was validated on more than 400 real invoices before cutover and now supports a business processing over 300 invoices per day. Unresolvable cases escalate to a human rather than becoming best-guess writes.
Outcome: reduced a two-person review process to a single reviewer.
A multi-stage deterministic pipeline that converts noisy engineering BOMs into ERP-safe manufacturing BOMs. Spec-based conflict handling prevents silent merges, collapses derivative errors into root causes, and blocks ERP execution until the dataset satisfies a formal safety condition.
Outcome: manual cleaning for a multi-level dataset fell from roughly 30 minutes to under 2 minutes.
As sole technical operator, I own monitoring, incident response, database and infrastructure performance, and day-two reliability. The platform maintained more than 99% uptime across 18 months, while response time for incidents fell from 45 minutes to under 5 minutes and page load improved from 3.2 seconds to 0.8 seconds.
Engineering standard
A detector never grades itself. I score failures against deterministic or human-owned ground truth and keep schema validity separate from semantic correctness.
Seeds, configuration, datasets, traces, and statistical methods travel with the result. If another engineer cannot reproduce it, it is not yet evidence.
AI output does not earn trust by being fluent. Writes into ERP and financial systems pass deterministic gates or escalate to a human reviewer.
I care about the system after launch: observability, incident reconstruction, performance, rollback behavior, and the cost of recovery.
Where I can contribute
I am looking for high-ownership work where I can build evaluation and debugging infrastructure, harden agent execution, turn ambiguous failures into durable fixes, and help teams ship AI systems with evidence rather than intuition.
Samir Sawarkar