Approach
An engagement runs in six steps. How long and how deep each one goes depends on the model’s risk rating and on the quality of the existing documentation.
- Scoping
- Information request
- Review in your environment
- Findings rated by severity
- Report
- Remediation follow-up
How an engagement runs
- Scoping. We agree on the model, its use, its risk rating, the scope of the review and the timeline. A confidentiality agreement is signed before any information changes hands.
- Information request. We send a specific list: model documentation, data description, the developer’s test results, code, monitoring reports. You know from the start what we will need.
- Review in your environment. Wherever possible, data and code stay with you. We work with the access you provide, on your systems, under your security rules.
- Findings rated by severity. Each finding states the issue, the evidence, the impact and a recommendation, and is rated high, medium or low. Findings are discussed with the development team before the report is finalized, so that factual errors can be corrected. Conclusions the evidence supports don’t change.
- Report. A written report, structured for the committee or person who approves the model, with an overall opinion on whether the model is fit for its intended use.
- Remediation follow-up. Once fixes are in, we check each finding and close it or keep it open, in writing.
What you receive
- A validation report: executive summary, scope and limitations of the review, findings rated by severity, overall opinion.
- Technical appendices: tests performed, results, data and parameters used, so the review can be reproduced.
- A findings log, ready to load into your own issue tracking.
- A closure memo after remediation is verified.
Independence and confidentiality
- We don’t validate a model we designed, built or helped build.
- Our fees don’t depend on the conclusions of the report.
- Before the engagement, we disclose in writing any relationship with the model’s developer or vendor.
- Data, code and documentation stay in your environment wherever possible. Anything we hold is encrypted, limited to the engagement, and destroyed at the end on the terms agreed.
- We never name a client without written permission.
Our report is an independent opinion. Approving the model, and responsibility for regulatory compliance, stay with the institution.