Introduction
Hello, and welcome to this SPK and Associates vlog. My name is Michael Roberts. I’m the Vice President of Sales and Marketing with SPK.
Now, everybody knows that artificial intelligence is transforming the way organizations develop products, write software, and make decisions. But while many companies are racing to tools like ChatGPT, Microsoft Copilot, or Claude, far fewer have taken the time to establish the governance needed to use them safely and efficiently.
So, in today’s talk, we’re going to talk a little bit about AI governance, including what organizations need to do so that AI becomes part of everyday engineering and business operations without a lot of concern for security and those things.
From protecting sensitive data, managing access, and reducing the risk of things around shadow AI, which is another thing that you don’t want to do, there’s a lot for leaders to consider, which is why we’re on this topic today.
And luckily, I’m not here alone. I’m here with Mike Sullenap. So, Mike, please feel free to introduce yourself.
Thanks, Michael.
Mike Sullenap. I’ve been with SPK and Associates for close to 19 years, and I lead up our Cloud and Infrastructure practice here at SPK. More recently, we’ve been heavily involved with AI solutions and customers leveraging AI tools, and that’s really what we’re here to talk about today.
Yeah, Mike and his team have seen a lot in the last couple of years, and this is why this topic is so important. The capability is so easily there, but the structure and guardrails are so important.
Security and Data Privacy When Using Public LLMs
So, Mike, many organizations are rushing to adopt all these tools, right? But they’re forgetting the governance, how to put that in place, and what that even looks like.
What are the biggest security and data privacy mistakes that you’ve seen companies make, or hopefully you’ve helped prevent them, especially when employees are starting to use public LLMs?
Yes, I would say that the first thing companies need to look at are the privacy policies behind each of these providers.
Within those privacy policies, you’ll want to look specifically at how your data is being used, whom it’s being shared with, and for what specific purpose.
For example, is it being used to retrain and refine the model? Because that could be a problem depending on what type of information you share with it.
Within subscription types and subscription tiers, there are also certain differences in terms of how data is collected and how it’s being used. So, you want to look into the privacy policies across subscription tiers as well.
What you’ll often find is that with paid subscriptions, data is not being shared or it’s not being used for training purposes. And that’s really what you want so that you can protect your proprietary data or any other critical pieces of information that may have been uploaded to the LLM by your employees.
But I think, regardless of what those privacy policies say, best practice is always, and we would recommend instituting this at a company-wide level, providing training or learning sessions for your employees.
There’s a lot of new behavior that you may not want them taking on with these products and LLMs.
For example, that includes things like cleansing code snippets. You may have code snippets that contain proprietary algorithms or things of that nature, or maybe those pieces of code reference internal resources. That might give hackers some additional information about your key pieces of infrastructure.
Then you also have more general things like company proprietary documents or even email conversations.
Basically, what I recommend is if it’s a piece of information that you’d be hesitant to share with somebody off the street, then you should be wary about sharing it with an LLM.
Yeah, I thought I was scared before with the internet and security, and now I’m even more scared with all the AI components. So, good recommendations there and best practices.
The Risk of Shadow AI
So, one of the other growing concerns, and we’ve heard a lot about this, is shadow AI, where employees are connecting AI tools to company systems or potentially even uploading sensitive data without IT oversight.
This is much like the shadow IT problem.
So, what risk does that create for engineering organizations, or really any part of an organization? And how should leaders think about access control and governance around that shadow AI component?
Yes, so part of the problem here is that employees are moving really quickly, right?
Part of their ultimate goal is to get that one deliverable created, get that email created and polished up by AI, or have that document or presentation created. Ultimately, they want to save themselves some time, and that’s why we’re drawn to leveraging AI in the first place.
But in employees trying to find shortcuts, they might also be trying to find shortcuts with the integrations and the tools around AI. Oftentimes, that means giving very little thought to things like permissions.
Typically, what you’ll want to provide is least privilege. We’ve heard that term before, but it really lends itself well to AI and AI tooling.
For example, if an AI agent doesn’t need write access into one of your line-of-business tools that you’re giving it access to, then we shouldn’t take that risk.
The problem is that properly designing application integrations takes a lot of planning, testing, and some reiteration or refinement. That’s a big investment.
But I would say that those investments and efforts are well worth the costs.
Aside from permissions risks, we’ve also seen cost be a big factor.
With no centralized governance, companies could suffer from severe sticker shock as token usage is either not monitored or limited in any way.
So those are kind of the two big risks that we see.
Operationalizing AI With the Right Guardrails
Got you.
Now, I want to shift from that component to a little bit more of how things are being operationalized.
Organizations are moving from experimenting with AI in different pockets to, you know, some are even starting to operationalize it with these AI agents in their workflows now.
So, what practical guardrails should companies put in place around identity management, the least privilege access that you mentioned, approvals, and human oversight in order to reduce risks while they can still enable innovation and utilize the AI components?
Yeah, I have a recommendation there, and this may be kind of a new concept. It’s essentially implementing a private LLM.
The Case for Private LLMs
The reason I mention that is because it’ll really give companies a lot more control.
You’ve got a ton more levers at your disposal in terms of things like connectivity or managing budgets, the way the data is stored and where the data is stored.
You can do things like content filtering if you have acceptable use policies within your company and you want to enforce them in terms of your employees interacting with models, as well as being able to control privacy and kind of the future of your own data, so to speak.
Most companies today are already familiar with things like private compute resources and public clouds.
Likewise, we can move our LLM usage into private areas in public clouds.
But that’s not an easy thing to do, and that’s really where we’ve been helping a lot of our customers, by helping them build their own custom solutions to maintain that governance, also keep costs in check, all the while maintaining security and privacy.
Yeah, and I love the combination of doing that private LLM.
It actually provides some governance and gives you the ability to control that, but also the cost component, which I think is another whole angle in terms of the way that these LLMs are going, which is that consumption model, right?
So, this will give you the ability to kind of control that.
Mike, thank you for sharing those recommendations. Really appreciate your time.
Yep, no problem.
Key Takeaways on AI Governance
So, I think at a high level here, we’ve talked about how AI has a lot of potential to improve productivity and accelerate innovation, but there are also a lot of guardrails that need to be put in place too.
And as Mike shared, effective governance shouldn’t slow that innovation down, but it should provide you some cost control, some protection around your organization’s data and intellectual property, and obviously give you some ability to maintain your compliance obligations as well.
How SPK Can Help
So, if your organization is evaluating AI adoption or looking to establish a governance framework around your AI implementations that balances security and innovation, our team can definitely help.
We have a lot of experience with that.
You can reach out to the folks through our contact information on our website, which is linked in the description of this video.
Closing
Thanks again for watching.
If you found this discussion valuable, be sure to like this video, subscribe to the SPK and Associates YouTube channel, and click that notification bell so you don’t miss any future conversations on engineering, AI, cloud, and digital transformation.
We’ll see you in the next video.
Thanks.
