The BCN Podcast

Not All AI is Equal Part 2

• BCN

Share episode

Following the latest episode of the BCN podcast, "Not all AI is Equal Part 1," where AI experts Fraser Dear, Head of AI and Data Innovation at BCN, and Andy James, AI and Power Platform expert delved into the nuances of AI, breaking down its various categories, the team takes it a step further in Part 2 as they explore Microsoft's AI ecosystem. Discover the key differences between the general Copilot, a versatile free assistant, and the Microsoft 365 Copilot, which integrates seamlessly with productivity tools like Word, Excel, and Teams. Learn how these AI tools can enhance your business operations with features like AI summarisation in Teams, and gain a clear understanding of the pricing tiers to make an informed decision for your business needs.

Explore the groundbreaking capabilities of Microsoft's Copilot Studio and its AI-driven chatbots, which are reshaping enterprise solutions. We also spotlight advancements in data processing that unify diverse data to prevent automation breakdowns. Additionally, the team provides examples of broader AI applications, including data science and cognitive services that transform telephony data into actionable insights.

Peter Filitz:

Hello and welcome to the BCN podcast. My name is Peter Phillips and I'll be your host today. As we started off this year discussing 2024 and what it brought from an AI perspective, we also looked to the future and we discussed the trends coming up in 2025. Just following on from our second episode this year, where we discussed not all AI is equal, we thought we'd create a second installment to the series, because there still seems to be a lot of confusion and hype around the different types of AI and what they need to be used for. So, with that being said, I'd like to invite Andy and Fraser back on today's conversation. Thanks so much for joining us. Fraser is the head of AI and data innovation here at BCN. Fraser, for those who've not met you before, do you want to quickly introduce yourself?

Fraser Dear:

Yeah, sure, Hi everyone. My name's Fraser. I look after all of our kind of AI and innovative data challenges here at BCN.

Peter Filitz:

With Fraser on today's conversation. We've got Andy James. Andy, for those who have not met you before, do you want to give a?

Andy James:

quick intro to who you are Sure. So me and my team use the Microsoft platform, predominantly the Power Platform, to try and make tools and solutions for business processes and just make people's lives easier.

Peter Filitz:

Excellent. So I think the objective of these conversations is really to drive awareness to both our audience and clients who are listening to these conversations around what AI's capability is and understanding how they can use this in a practical sense in their everyday business as well as their personal lives, because it seems to be playing a more prominent role in both those areas. Now, as we've said before, not all AI is equal and it's making sure that you use the appropriate AI option and service for what it is that you actually want to do. So, fraser, I guess one for you. Do you want to maybe give our audience a high-level overview around the differences? And I think, just to focus it a little more, we're talking about the Microsoft AI services, so possibly the co-pilot for 365 and then the general co-pilot. Do you want to give our audience a little understanding around that?

Fraser Dear:

Yeah, sure. So let's break this down into two themes. One will be functionality, the other will be price. So first of all let's talk about Microsoft Co-Pilot functionality. So this is a general AI assistant, where it's intended to be. A broad assistant helps with various tasks coding, writing, generating images, answering general knowledge questions and, for those of you that have been around recently in the space, this is also known as Bing Chat previously.

Fraser Dear:

Now, availability-wise as far as Microsoft Copilot is concerned is that it's available for free and it's available now. It's available for everyone. Concerned is that it's available for free and it's available now. It's available for everyone. There is a premium tier called Copilot Pro, which again adds a little bit of complexity to the waters, which adds some extra features, of course for a fee, but it provides kind of more generative credits. It provides higher levels of access at peak times. It gives preferential access. If there's a lot of volume going through the free layer, the premium layer gets addressed first and some other light integrations with some of the Office Suite. But it shouldn't be confused with the Microsoft 365 Copilot because it is limited. So right now we're just talking about Copilot. The ProLicense can do integrations with Office 365, but nothing compared to the M. Talking about Copilot, the Pro license can do integrations with Office 365, but nothing compared to the M365 Copilot.

Fraser Dear:

As far as the Microsoft 365 Copilot, that is very much focused around productivity. So Microsoft 365 Copilot is specifically designed to enhance productivity within Microsoft 365 suite of applications that's Word, excel, powerpoint, outlook, teams, that suite. The target audience for the Microsoft 365 co-pilot is the professional enterprise environment, which helps streamline workflows, automate tasks and provide that intelligent recommendation based on the real-world usage patterns that were captured when they were creating the M365 co-pilot. Now, unlike Microsoft co-pilot, the 365 co-pilot is deeply embedded into the 365 apps. So it provides contextual assistance with tools, using a semantic index and other tools to provide that integrated and deep understanding of the end user estate access to data and activities that that person has then done.

Fraser Dear:

Now, to add a little bit more gray into that model, you've also got overlapping product co-pilots. So, for example, teams has a platform called Teams Premium. Now, if a a user wants to, let's say, get an AI summarization of a meeting, maybe create some insights, maybe get some actions, there's actually a number of different ways to achieve the same thing. So you could use a Teams Premium license which will give you those insights, but you could also get those insights through the m365 co-pilot in teams to achieve a very similar result. So actually it's a bit of a combination of different functions, features and also platforms available within each product, and it's about understanding what use case you're really trying to drive the way to think about. It is both co-pilot and M36.

Fraser Dear:

Copilot leverage advanced AI capabilities. Copilot is a versatile assistant for general use. M365 Copilot is tailored for the M365 ecosystem, which is Word, office, excel, powerpoint and Teams. So that's the functionality side. Licensing-wise, it's dead easy. Copilot is free. Coalpilot Pro is around £19 per user per month and the M365 CoalPilot is £23 per user per month. Does that help or does that hinder?

Peter Filitz:

I think you explained it really well, I mean, but just given the length of that explanation shows the complexity to it and I guess, again, it's understandable why there is a degree of complexity to it because, again, you know, there are so many different products and services available within the Microsoft Cloud stack. There is a certain degree of segregation and limits to different services needed, which is why we've got so many different co-pilots right, because not everyone is using everything within the 365 stack or within the cloud stack, should I say so having a co-pilot for each feature makes sense. Now, that's really useful and I think just touching on the different AI options for the different business use cases. So you've obviously talked about personal productivity there and those co-pilots you discussed and mentioned I guess fall within the personal productivity category. What about the organizational-based AI and how does that feature in part of the co-pilot family?

Fraser Dear:

so to speak. I think Andy's probably got a good insight with regards to how to customize co-pilot for probably got a good insight with regards to how to customize Copilot for personal productivity and did you want to maybe talk through some use cases that we've done or are doing here at BCN.

Andy James:

Sure, sure, sure, sure, yeah, just to throw another Copilot in the mix, there's Copilot Studio.

Andy James:

That's the Microsoft chatbot that uses generative AI to scour through the knowledge bases you assign to it, you give it access to in order to give you an answer. Those chatbots, those Copilot Studio bots, can also trigger automations and access different data sources. They're incredible and they're growing all the time. With those, we find that they're really really good for surfacing information. So if you had policy libraries and documents and HR processes that you needed to know, you know what your annual entitlement for annual leave is or the maximum you can have for lunch allowances, whatever that might be Querying those kind of document libraries to find that information. Copilot Studio is kind of amazing. That's where it's found its kind of bread and butter. It's really quick and easy to implement. Very cost effective Different cost model to the ones that Fraser just mentioned, but very cost effective. With those bots, you still have to talk to them, so you ask them a question. It takes that information, understands your intent, looks for the right knowledge, searches that knowledge, gets an answer, structures it with generative AI and gives you it back. And, as I say, it can also trigger automations if we need to, so you could ask it to send an email or raise an order or do a thing. They can trigger an automation to do that.

Andy James:

Recently, however, we've also had additional capabilities with them where you can have autonomous chatbots or autonomous agents, and the agent kind of nature of AI was something we touched on, I think, in our last conversation. But with these they just kind of sit there in the background waiting to be asked to do something. We can trigger them from automations. It can come from emails coming in, from records being created in SharePoint or from other chatbots, so you could have an autonomous agent that looks at your diary, identifies when you're available and curates a list to send out wherever that might be for your availability. Well, it might be that when an email comes in that says hi, andy, when are you free for a chat, that can trigger that bot.

Andy James:

It might be that in Teams we could have it as a plugin for Copilot, so anyone with an M365 Copilot license could use that and they could ask the bot when Andy's free and that same agent does his thing. Equally, it could be another Copilot chatbot that you ask the question and, through conversation orchestration, that chatbot knows to ask this agent. The agent does its thing, returns the answer, so it's all interlinked and to make sure that those processes are as smooth as possible. In addition to that, we're doing a load of pieces of work with AI, looking at restructuring information. So a really interesting use case is that an organization needs to capture survey results from a whole raft of different platforms and we want to ingest all of this information and then we want to tidy it up and we want to put it into a single data source, in a single data structure.

Andy James:

Or If we had five different platforms feeding us information.

Andy James:

That information comes in five different forms so historically we'd have to do some kind of data manipulation, either in Power Automate or in data pipeline factories in Azure, to get the data, remap it and then feed it into that single data source, knowing full well that if one of those structures changes then that automation breaks and we have to go back and kind of fix it.

Andy James:

But now I can just extract all that information, give it to a bot, say please tidy that up for me and put it in this structure, and then it does all of that. I can then take that structure in a JSON format and put it into my single data source and then if the structure changes from one of the clients or one of the providers, it doesn't really matter, because that generative AI is going to go through, understand what I want, understand the structure it's got, match the two together and give me an output. So there's all kinds of different opportunities that we can find with these bots just to try and, like we say, make life a bit easier, smooth things over and more robust in how these tools and solutions work.

Peter Filitz:

That's amazing. I mean, the progress from where we were a year ago to now is just astounding, and all aimed at driving productivity and alleviating the day-to-day mundane tasks from people, allowing them to focus on the more strategic elements of their job. Absolutely, fraser. Do you want to talk us through how some of the organizations in the enterprise space is taking advantage of this?

Fraser Dear:

Yeah, I mean, I think what Andy's just gone through there is kind of the beginning and beginning of the middle of kind of the sliding scale of complexity and sophistication of some of these AI tools that we now can implement. Because if we start to think about some of the more say complex, it doesn't need to be complex but if we start to think about, perhaps maybe data science workloads and cognitive service workloads, these are also AI, but they're not kind of classed under that generative AI banner that Andy's just walked through. So let's think of another use case here. If we think about an organization connected to, perhaps like telephony services, and so they record and capture all the calls that have been handled, An example of like an organization-wide solution might be to analyze all these calls, maybe using Azure Cognitive Services. We would then maybe extract the sentiment of the call, all the actions, insights, identification of, maybe keywords or features like personal identifiable information or maybe something that matches an organization's product catalog. So you would then get insight as to why calls are coming in and which products customers are actually asking about. Each of those elements that I've just described could be considered as separate data insights and capturing that, or it could all be brought together into kind of one master repository and then enable your business users then to query, understand marketing perspective, understand the operational expectations. You know which members of your team are giving the best technical responses and which of your team's callers don't responses and which of your team's callers don't call back because they actually got the information they needed from that call in the first place. What Andy's kind of just gone through might be a bot that replaces a telephony person, but when we do have to then connect through telephony, what can we do with that information?

Fraser Dear:

The other area that we might want to consider then would be like the data science workflows, and here we're talking about things like predictive analytics, forecasting of business demand based on, maybe, market data or macroeconomic information, and kind of machine learning and cognitive APIs. And what do I mean by that? I mean APIs are the things that pass data between business systems. We've now got the ability to have cognitive APIs so they understand a little bit like what Andy was saying. Data comes in and actually our APIs can then actually process that into something more meaningful for our requirement, rather than just pass data blindly.

Fraser Dear:

Then, of course, everything I've just said there could all be wrapped up with a generative experience of the top of it. So, for example, the end user just types in a question over maybe a SQL backend. Allow the AI to undertake all of those maybe calculations, drawing insights, leveraging information from across all of those elements that I've just described to give the actual answer. And actually the key thing here is not about the AI necessarily. The key thing here is about the data. So to achieve all of this, the data is critical and that underpinning architecture that an organization has will enable AI. That's the key piece there.

Peter Filitz:

So true, yeah, and these are conversations that I see every day businesses wanting to understand where and how to get started on this journey. And, as you quite rightly point out, it all starts with that data journey guessing that data in a structured format, in a way that AI can really leverage it, because ultimately, the performance and the output of your AI is only as good as the data that it runs on. Right Now I guess just again, one of the underpinning requirements and concerns a lot of businesses have is what security aspects of using OpenAI models within Azure do they need to understand and review. Could you talk us through at a high level what security measures are in place and what reassurances we can provide our clients when investing in these services on the Microsoft platform.

Fraser Dear:

Yeah, absolutely. This is a question that comes up quite a lot, because when we talk about the words open AI, immediately customers go oh, does that mean open source? And if it means open source, does that mean that my data is sloshing around on the internet somewhere? And I think we touched on this last time. Actually, web-based generative AI small star is generally open source. So if you're copying and pasting your business data into a web platform that is, for example, like a free chat GPT service, it's going out into the world. So what's the difference between that and Azure OpenAI? So there's a whole raft of measures, but the top three things from this kind of aspect of the question is data isolation. So your data, including the prompts, the generated content, isn't shared outside. So it's not shared with other customers, it's not shared with the model creators and it's not used to improve those open AI models. And when I say an open AI model, I might mean something like ChatGPT 3.5 Turbo. That would be an example of a model that we'd implement into an Azure workspace and that's the thing that's doing that generative transaction.

Fraser Dear:

The second piece would be data storage. So here, data stored and processed within your Microsoft Azure environment, ensuring that it doesn't interact with any other services operated by an OpenAI provider. And then, of course, the last piece is compliance. So Azure OpenAI adheres to all of the Microsoft Cloud security benchmarking and, as part of your Azure subscription, it will also provide recommendations on securing your cloud solution. Don't want to panic any customers. There are always ways to make your deployment risky and, of course, within Microsoft there are lots of ways to ensure security as far as your deployment is concerned. So I'm not saying here that just because it's in Azure means it is completely locked up and secured. That's not what I'm saying. However, of course, azure by definition provides you all of those security measures to lock it down as standard. You have to go some to expose all of that out into the open world again.

Peter Filitz:

Thanks, fraser. That's super useful and, as you quite rightly say, I think again, microsoft provides the platform and the tools in order to secure it. But you know, ultimately it we've got the services there to ensure that our data is protected. Talk us through at a high level how we're supporting businesses on that security journey, so to speak.

Fraser Dear:

Sure. So data protection is. I mean, it's a whole series of podcasts just on its own right, so I'll try and not get carried away here. So I'll try and not get carried away here, but if we're thinking about data protection as a whole, the most complete solution we're working with our clients on at the moment is Microsoft PowerView. So Microsoft PowerView is a suite of tools that help secure and protect organizations' actual underpinning data. Now, yeah, I mean we could talk about this for a few hours. So at a really high level.

Fraser Dear:

It all starts with data classification. So what we do is we take the data and we apply sensitivity information around it, and PowerView can automatically do that, but we can also then use code and examples to identify data within an organization. The second layer is around data loss protection policies. The second layer is around data loss protection policies. Microsoft PowerView essentially enables users and businesses to classify their data and apply sensitivity information types. So here we can put built-in or custom expressions, keywords and confidence levels about what data is Is it sensitive or is it not. What then happens inside PowerView is we start to create these data loss prevention policies or data loss prevention arguments, which are then tied together with those sensitivity labels and scenarios. Powerview then enforces data protection by using those sensitivity labels in concert with what the activity is actually happening.

Fraser Dear:

So that was a whole load of big words. What does that really mean in practice? Well, breaking it down at a user level, if someone tries to copy data and paste it into ChatGPT, for example, the copy function will still operate as normal, because guess what? You need to copy and paste stuff in your Word documents and in your daily activities all the time. But when they move to a web interface, a DLP policy will trigger and the user will essentially get a pop-up saying this activity has been blocked. Microsoft PowerView has identified that this is sensitive information and a risky activity. Now, it's a trivial example, and all of this happens in the background and there's a huge level of sophistication to all of this. But essentially that's the outcome when we start to think about surety for organizations on accidental and also malicious data exposure.

Peter Filitz:

Yeah, and I was just going to say I mean putting the whole AI conversation aside. If businesses have sensitive data, which every business rightfully has, microsoft Purview is certainly something that they should be looking at for data governance, security and compliance. It's a tool that's part and parcel of the Microsoft 365 service. A license upgrade might be required, but if you've not looked into it, it's certainly something that you should, because it does just revolutionize the management of security across your entire cloud estate through a simple to use dashboard right, which previously was extremely cumbersome when trying to do it at sort of folder or user-based levels. Thanks, fraser, that's super insightful.

Peter Filitz:

I think that sort of brings us close to the end of this week's conversation, just to sort of recap. You know, again, this was intended to provide our audience with insight around. Not all AI is equal. I think from our guests' comments and insight, it's fair to say, we can see why we've chosen that as a topic. There are a multitude of options and services out there, but don't be overwhelmed, you know. This is why bcn, as a trusted partner to our clients and prospects, exist because we help bring sense to these conversations and we can work closely with you and your businesses to help identify which AI services are appropriate and ensure that, obviously, your data is locked down and adequately secured, whilst also empowering your staff in leveraging these new products and services. Don't forget to visit our website, bcncouk. There you'll find a wealth of information and knowledge around the products and services we've also just discussed today.

Peter Filitz:

Fraser, andy, thanks so much for joining us this week. It's always a pleasure having you. We'll have you on the next AI podcast. We've got a whole series coming up this year. Thank you, gentlemen. Thanks everyone, cheers, don't forget to tune into our next podcast and don't forget to like and subscribe. Thank you so much for tuning in.