Introduction
Hello, and welcome to this SPK and Associates vlog. My name is Michael Roberts. I’m the Vice President of Sales and Marketing with SPK.
Now, everybody knows that artificial intelligence is transforming the way that organizations develop products, write software, and make decisions. But while many companies are facing and racing to to get to tools like ChatGPT or Microsoft Copilot or Claude, far fewer have taken the time to establish the governance needed to use them safely and efficiently.
So, in today’s talk, we’re going to—we’re going to talk a little bit about AI governance and, including in that, what organizations need to do so that AI becomes part of the everyday engineering and business operations without a lot of concern for security and, and those things.
So, from protecting sensitive data, managing access, and reducing risk of things around shadow AI, which is another thing that you don’t want to do, there’s a lot for, for leaders to consider, which is why we’re on this topic today.
And luckily, I’m not here alone. I’m here with Mike Solinap. So, Mike, please feel free to introduce yourself.
Thanks, Michael. Mike Solinap. Been with SPK and Associates for close to 19 years, and I lead up our Cloud and Infrastructure practice here at SPK. But more recently, we’ve been heavily involved with AI solutions and customers leveraging AI tools. And that’s really what we’re here to talk about today.
Yeah, Mike’s—Mike and his team have seen a lot in the last couple years, and, and this is why this topic is so important because it’s not—the, the capability is, is so easily there, but kind of the, the structure and guardrails are so important.
Security and Data Privacy Mistakes
So, Mike, many organizations, they’re—they’re rushing to adopt all these tools, right? But they’re—they’re forgetting the, the governance and, and how to put that in place and what that even looks like.
So, what are the biggest security and data privacy mistakes that you’ve seen, either companies make or hopefully you’ve helped prevent them, especially when employees are starting to use public LLM?
Yes. I would say that the kind of the first thing that companies need to look at are the privacy policies behind each of these providers.
So, you know, within those privacy policies, you’ll want to look specifically about, you know, how your data is being used, whom it’s being shared with, and for what specific purpose.
For example, is it being used actually to retrain and refine the model? Because that could be a problem depending on what type of information that you share with it.
And then, you know, within subscription types, sub—in subscription tiers, there are also some certain differences in terms of how data is collected and how it’s being used. So, you want to look into the privacy policies across subscription tiers as well.
What you’ll often find is that with paid subscriptions, data is not being shared or it’s not being used for training purposes. And, and that’s really what you want so that you can protect your proprietary data or any other, you know, critical pieces of information that may have been uploaded to the LLM by your employees.
But I think, you know, regardless of, you know, those privacy policies, whatever they say, best practice is always, you know—and we would recommend instituting this at a companywide level—is, you know, providing training or, or learning sessions for your employees because there’s a lot of new behaviors that, that you may not want to have them, you know, kind of taking on with these products and, and LLMs.
For example, that includes things like cleansing of code snippets, where you may have code snippets that are, you know, proprietary algorithms or things of that nature, or maybe those pieces of code reference internal resources, and that might give, you know, hackers some additional information about your key pieces of infrastructure.
And then you also have more general things like comp—company proprietary documents or even email conversations.
Basically, what I recommend is, if it’s, you know, a piece of information that you’d be hesitant to share with somebody off of the street, then you should be weary about sharing it with an LLM.
Yeah. I—I thought I was scared before with the internet and security, and now I’m even more scared with all—all the AI components. So, good, good recommendations there and best practices.
Shadow AI Risks
So, one of the other kind of growing concerns, and we’ve heard a lot about this, is the shadow AI, where employees are connecting AI tools to company systems or potentially even uploading, like, sensitive data without IT oversight.
You know, this—this is much, much like the shadow IT problem. So, what risks do—does that create for, you know, engineering organizations or really any parts of the organizations? And how do leaders think about—how should they think about access control and governance around that shadow AI component?
Yeah. So, part of the problem here is that employees are moving really quickly, right? So, part of their ultimate goal is to get that one deliverable created, you know, get that—get that email, you know, created and polished up by AI, have that document or, or presentation created because, ultimately, they want to save themselves some time.
And that’s why we’re drawn to leveraging AI in the first place. But in the employees trying to find shortcuts, they might also be trying to find shortcuts with the integrations and the tools around AI. And, and oftentimes, that means giving very little thought to things like permissions.
And typically, what you’ll want to provide is lease privilege. We’ve heard that term before. But it really lends itself well to, to AI and AI tooling.
So, for example, if an AI agent doesn’t need write access into one of your line-of-business tools that you’re—you’re giving it access to, then, you know, we shouldn’t take that risk.
And, and the problem is, is that properly designing application integrations, it takes a lot of planning, it takes a lot of testing, and, and some reiteration or, or refinement. And, and that’s a big investment.
And—but I would say that those, those investments and efforts are well worth the costs.
And aside from permissions risks, we’ve also seen costs be a big factor. So, with, you know, no centralized governance, you know, companies could suffer from severe sticker shock as token usage is either not monitored or limited in, in any way.
So, those are kind of the two big risks that we see.
Gotcha.
Operationalizing AI
Now, I want to shift from that component to a little bit more of how—how things are being operationalized.
So, organizations are moving from, like, experimenting with AI in, in different pockets to, you know, some are even starting to operationalize it with these AI agents and their workflows now.
So, what practical guardrail should companies put in place around identity management that you mentioned, that lease privilege, access approvals, human oversight, in order to reduce risks while they can still enable innovation and utilize the, the AI components?
Yeah, I have a recommendation there, and this may be kind of a new concept, and it’s essentially implementing a private LLM.
And the reason I mentioned that is because it’ll really give companies a lot more control. You’ve got a—a ton more levers at your disposal in terms of things like connectivity or managing budgets, the way that the data is stored and where the data is stored.
You can do things like content filtering if you have acceptable-use policies within your company and you want to enforce them in terms of your employees interacting with models, as well as being able to control the privacy and kind of the, the, the future of your own data, so to speak.
So, most companies today, you know, they’re already familiar with things like, you know, private compute resources and public clouds. So, likewise, we can move our LLM usage into private areas and public clouds.
But, you know, that’s not an easy thing to do, and that’s really where we’ve been helping a lot of our customers, is by helping them build their own custom solutions to help maintain that governance, also keep costs in check, all the while maintaining security and privacy.
Yeah. And I love the, the combination of, you know, doing that private LLM actually provides some governance and, and gives you the ability to control that, but also the cost component, which I think is another whole, whole angle in terms of the way that these LLMs are going, is that consumption model, right? So, this will give you the ability to kind of control that.
So, exactly.
Mike, thank you for, for sharing those recommendations. Really appreciate your time.
Yep, no problem.
Final Thoughts
So, I think the, the—at a high level here, I think we’ve talked about how AI is, you know, there’s a lot of potential to improve productivity and accelerate the innovation, but there’s also a lot of guardrails that need to be put in place too.
And as Mike shared, effective AI governance shouldn’t slow that innovation down, but it should provide you some cost control, some protection around your organization’s data, intellectual property, and give you, obviously, some ability to maintain your compliance obligations as well.
How SPK Can Help
So, if your organization is evaluating AI adoption or looking to establish a—a governance framework around your AI implementations that balances security and innovation, our team can definitely help. We have a lot of experience with that.
You can reach out to the folks, to our—our contact info on our website, which is linked in the description of this video.
Closing
Thanks again for watching. If you found this discussion valuable, be sure to like this video, subscribe to the SPKN Associates YouTube channel, and click that notification bell so you don’t miss any future conversations on engineering, AI, cloud, and digital transformation.
We’ll see you in the next video. Thanks.
