I build systems where bad data can't hide.
Six years of data pipelines and backend systems in finance — reconciliation engines, forecasting services, the kind of infrastructure nobody notices until it breaks. The through-line: I don't trust my own input. Validate at the boundary, fail loud, and never let automation do something irreversible without a gate. Below is the work — every project runs, has tests, and has a write-up.
I learned this one the hard way: once bad data gets three steps downstream, nobody trusts anything the pipeline produces. So I check at the door — log what failed, say why, stop the line.
Automation is most dangerous right before it does something permanent. Every automated action I build gets a gate first — approval, confirmation, timeout — and only then the action.
The worst pipeline I ever inherited marked broken rows as “processed” and moved on. Weeks of quiet corruption. If something's wrong, my code screams. I'd rather get paged than get surprised.
Every reconciliation break has a reason. This finds it automatically.
Kafka streams transactions in, five passes match them — exact, timing, fuzzy reference, amount, then a local LLM for the stubborn ones. pgvector remembers past anomalies so the same surprise never happens twice. 94% of breaks resolve without a human.
auto-matched
94.2%
LLM cost
$0
passes
5
match breakdown, latest run
An AI wrote a trading signal. Who checks it before money moves?
A gate between strategy and broker. Five rules — position limits, restricted lists, wash trades, concentration, and EU AI Act human-oversight — check every proposed trade. It also reads new SEC filings and drafts candidate rules for human review. Every check lands in an audit trail that can't be edited.
rules
5
EU AI Act
Art. 14
audit
immutable
Everyone has a strategy that “would have worked.” This tries to prove you wrong first.
Signals are shifted a day before touching returns, so lookahead is structurally impossible. Every rebalance pays spread and market impact. Walk-forward validation exposes overfit strategies. Then the same weights go to Alpaca's paper API for real fills — the gap between backtest and paper is itself a measurement.
strategies
2
metrics
13
lookahead
impossible
What happens inside an exchange between “buy” and “filled”?
A limit order book that reads NASDAQ's real wire format and watches for manipulation in the same process — spoofing, layering, momentum ignition, quote stuffing. My first benchmark claimed 45ns per order; the real measured number was 673ns. Fixing the measurement taught me more than the code.
add
673ns
cancel
29ns
detectors
4
Every branch pads its vault order 15–20% as a buffer. This turns the buffer into a number.
Ridge regression per branch over 730 days, Isolation Forest to flag demand anomalies, a 14-day horizon with confidence bands. The guess becomes an order recommendation with a reason behind it.
avg MAPE
9.1%
tests
14/14
horizon
14d
2am. Service is down. The fix is buried in a 40-page runbook.
RAG over your own runbooks — answers cite the exact file and line, and the LLM only quotes what it found. Remediation pauses at an SNS approval gate; nothing touches production until a human says so.
vector db
FAISS
gate
SNS
workflow
StepFn
A truck breaks down. The service was six weeks overdue. The spreadsheet was the last to know.
An hourly scheduler catches overdue maintenance before anyone checks. Alerts are idempotent — the same event fires once, not on every poll. 25+ endpoints, role-based access, 16/16 integration tests.
endpoints
25+
tests
16/16
stack
Java 21
Kafka, PostgreSQL, pgvector, Redshift, DynamoDB, Airflow, Glue
Java 21, Spring Boot, FastAPI, GraphQL, Docker
AWS (Bedrock, Step Functions, Lambda), Terraform, Ollama, FAISS
C++20, order books, ITCH 5.0, backtesting, walk-forward
The projects above are the best representation of how I work. If that's the kind of engineer you need, my inbox is open.
karthicks399@gmail.com →