“"There is only one way forward. Evaluations and monitoring activity go from an afterthought QA task and become the forefront of how management gains trust and confidence in their systems and processes."
Jonathan ShertokGlobal AI Group Product Manager and Chief of Staff @ EY
GenAI is breaking every IT playbook. As systems become less predictable, traditional methods of ensuring trust and control no longer hold up, and xperts in the enterprise stress it's time for a new approach.
Jonathan Shertok, Global AI Group Product Manager and Chief of Staff at EY, has spent years deep in the weeds of internal controls. Now, he's calling for a fundamental reset in how organizations build trust in AI.
- Trust or bust: As organizations race to adopt AI, many are skipping the hard part: building real trust. Shertok argues that assurance can’t be an afterthought anymore. "There is only one way forward," he says. "Evaluations and monitoring activity go from an afterthought QA task and become the forefront of how management gains trust and confidence in their systems and processes."
- The ghost of Big Data past: We've been here before. Shertok compares today's AI buzz to the Big Data boom, when hype outpaced control. "When Big Data became in vogue, it washed over mainstream media, and reshaped companies. There was a lot of early investment there, and a lot of wasted cost," he recalls.
- Back then, it took time to realize that digital didn't automatically mean accurate. Now, AI demands that same reckoning—especially when it comes to ICFR and SOX compliance.
- Testing, testing: Old testing methods don't work here. Traditional systems can be validated by checking every outcome. But with GenAI, "You can't test every calculation alternative because the whole system is generative," Shertok explains. "GenAI and agents flip that on its head." Instead, assurance depends on evaluating behavior: stress-testing known patterns, edge cases, and defined performance criteria to see how the system responds.
AI systems don’t show their true colors until they're in the wild. Real behavior emerges post-deployment, as people interact in ways no test environment can fully predict. "This is where I foresee all of ICFR that touches AI going—it's an inevitability," Shertok says. "This pattern of evaluation and monitoring could scale to all non-deterministic or probabilistic systems."
