Meet Egiziago Cioffi, the IT and Enterprise Architect, and CEO of SynSphere Italia, a Microsoft partner based in Milan. He recently shared his experience building an AI assistant using Azure OpenAI, and the challenges he faced along the way.
According to Cioffi, his Azure OpenAI email assistant successfully auto-resolves about 60% of inbound customer emails. However, during a detailed evaluation, he discovered a critical flaw in the retrieval pipeline.
Running a test with low-privilege and high-privilege accounts revealed that the assistant was returning SharePoint content that should have been inaccessible to the requesting user. This discrepancy highlighted a potential security vulnerability in the system.
Further investigations into similar production deployments revealed that 91% of successful attacks on productivity agents resulted in silent data exfiltration. This alarming statistic underscored the importance of robust security measures in AI systems.
In response to these findings, Cioffi implemented a query-path filter that checks the requesting user’s permissions before retrieving content. This filter helped restrict the assistant’s access to sensitive information, improving overall security without compromising functionality.
It’s crucial for organizations deploying AI systems to consider both identity governance platforms and retrieval-time entitlement checks to ensure comprehensive security measures are in place.
By asking critical questions and conducting thorough tests, security teams can identify and address potential vulnerabilities before they are exploited. Cioffi’s experience serves as a valuable lesson in the ongoing effort to enhance the security of AI systems.
