Multimodal code models with Juspay

Juspay’s Xyne team came to us with two problems that looked related but weren’t. The first was search: an AI-first engine that could answer questions across large engineering workspaces. The second was code completion for Hyperswitch, Juspay’s Rust payment infrastructure, where a model would sit inside the editor and suggest the next piece of code.

Both had to hold up in real developer workflows. A model that scores well on a benchmark and then guesses wrong inside someone’s editor is worse than no model at all, because engineers stop trusting it after the third bad suggestion.

Search That Moves the Way Engineers Do

An engineering workspace is repositories, documents, and tickets, all at once. When an engineer asks a question in plain language, they want an answer grounded in those sources, not a list of files that happen to look similar to the query.

So we built Xyne’s search as an agent rather than a lookup. Tool connectors let it retrieve from a workspace, read what it found, and move between files and repositories as the question demands. Underneath sit RAG pipelines with optimized vector search, which keep that first retrieval step fast and relevant enough for the agent to reason on top of.

Why the Suffix Mattered

Completion is a different shape of problem. An editor hands the model the code before the cursor (the prefix) and the code after it (the suffix), and the model has to write what belongs in between. That’s fill-in-the-middle, or FIM.

Many baseline models still predict strictly left to right. They read the prefix, start typing, and effectively ignore everything below the cursor. In practice that means suggestions that compile in isolation but clash with the very next line. We wanted a model that treats the suffix as a constraint, and an evaluation that rewards exact completions rather than code that merely looks plausible.

Teaching the Model to Fill the Gap

We built the training data from AST-parsed prefix and suffix snippets instead of full-file context. Parsing with the AST, the syntax tree a compiler uses, means every split point lands somewhere syntactically meaningful, so the model never learns from a gap cut halfway through a token. Special tokens mark where the prefix ends and the suffix begins, which teaches the model to generate a middle it has never seen.

For fine-tuning we used LoRA adapters at rank 32 and alpha 64 over a 4-bit NF4 quantized base model. LoRA trains a small set of add-on weights instead of the whole network, and NF4 shrinks the base model’s memory footprint, so we could iterate quickly without giving up the base model’s general knowledge. Training ran for 225 steps across three epochs with gradient accumulation on H200 hardware. The result was a model tuned specifically for FIM behavior, with stable training dynamics and a clear path toward RL fine-tuning in later iterations.

What the Numbers Showed

The fine-tuned model reached 52.3% exact-match accuracy on code completion, a 21-point improvement over the baseline. The baseline had the advantage of full-file context and still lost, which tells you the gain came from learning the task, not from seeing more code.

• 94%+ token accuracy on 1.8M+ training tokens, and 90%+ eval accuracy on 1.5M+ evaluation tokens.
• Loss improved from 1.6 to 0.25, with gradient norms holding steady in the 0.25 to 0.4 range.
• FIM-specific training behavior, built on special tokens for prefix and suffix context.
• Agentic search shipped for Xyne, with tool connectors and semantic retrieval across workspaces.

The work ran on LoRA, NF4 quantization, AST parsing, FIM training, PyTorch, RAG, and vector search.