Blog
I write about Java, software architecture, design patterns, applied artificial intelligence, RAG, AI agents and software engineering. Through my articles, I share feedback, technical analyzes and concrete approaches to design robust, maintainable and useful applications.
Comparing Two Versions of an LLM System
Identify and qualify behavioral changes in an LLM system when the model, the prompt, the tools, or the rules evolve, to tell an improvement apart from a regression.
Building a Representative Evaluation Set
Build an evaluation corpus that represents an LLM system's usages, edge cases, and invariants, to give version comparisons a solid foundation.
Detecting and Qualifying Behavioral Drift
Tell an acceptable behavioral drift apart from a real regression in an LLM system, by qualifying it through nature, frequency, scope, and impact rather than a single overall score.
Testing Decisions Rather Than Wording
Verify the decisions and invariants an LLM system upheld rather than the exact wording of its response, to build tests that survive changes to the model and the prompt.
Defining the Behavioral Invariants of an LLM System
Turn the behaviors an LLM system must preserve into explicit, observable, and verifiable properties, so we can tell what may vary from what must remain stable.
Behavioral Surfaces
Map the areas where an LLM system's behavior can vary, drift, or regress so that it can be observed, tested, and governed more effectively.
Decision Trajectory
Reconstruct the path followed by an LLM system to understand, compare, and govern its decisions. Why two similar responses can mask fundamentally different behaviors.
