Microsoft Copilot and the AEC

Joe Giegerich: 

Hi, everybody. Thank you for joining today’s ProjectReady podcast. On the panel today, we have myself, Joe  , Shaili  , who’s our head of development, and we have Lou  o, a long-term veteran of Microsoft, and what we want to discuss today is to try to get a handle on what AI technology can do for the industry, its potential, but with a specific focus on Microsoft Copilot. 

Now, I’ve always been big advocate of if you have commercial off-the-shelf solutions, try to bring them together rather than a whole bunch of proprietary stuff, and in fact, even Procore is now starting to use Copilot as part of their stack, and that’s something that we’ll be rolling out shortly. But there’s an awful lot of buzz about AI, and we had done a previous podcast called Garbage In, Garbage out, which our contention was that if you’re just using unstructured data, if you don’t have a scalable taxonomy, the answers you’ll get via AI are as good or as bad as the underlying data. 

And in fact, when Shaili had sent me this great video, we can send it out to you guys. We’ll post it when we post this podcast. It shows you how to set this thing up. In fact, the ability to point to specific data sources, I’m stealing everybody else’s thunder, and now on the topic of AI and the AEC. So half the industry is starting to look at AI, robotics, obviously in the construction side, and so you have these huge stats about people wanting to adopt AI, but then you have reports from KPMG that they’re in the very early stages, and that’s reasonable. Another thing that I have mentioned was that there’s a lot of hype about AI, so let’s just get to the truth and the value of it. It’s not putting everybody out of work, it’s just letting you work a whole lot more efficiently. 

And then the final comment about the industry itself is I have seen no industry, and I’ve been in tech since I was a teenager and I’m long in the tooth now, I have never seen an industry with more data that it has to make sense of and where the importance of data can really impact real world profitability and real world disasters, right? The built world is the foundation literally upon which we stand. 

So Lou, why don’t I turn it over to you, and why don’t you tell us a bit about Copilot and the studio, and the different things that it can do and that you think has value out there. 

Lou Gucciardo: 

Absolutely, and thank you for having me. So you mentioned before the concept of build versus buy, and Microsoft, we offer both, and Copilot is essentially our buy solution. What Microsoft has always been very good at is democratizing technology, making technology easily accessible to large swaths of personas within an organization, and so that’s essentially our Copilot strategy. So whether you are an information worker, you’re in customer service, you’re in sales, whatever that may be, we have a Copilot that allows you to do your job more efficiently, and those Copilots are growing and growing. We have now over a hundred and we are looking at expanding into more functional areas. As an example, we’re in beta, something called Project Galea, which is a Copilot for frontline workers. We just introduced a Copilot template for manufacturing operations. And so Copilot is that SaaS solution. You buy it, you do slight configurations, it’s running and you interject AI into your organization. 

The other side of the coin, the building is something we call Azure AI Services. Azure AI Services are not just generative AI, it’s also traditional AI. And what’s traditional AI? That’s machine learning, that’s vision, that’s speech translation. AI, traditional AI has been around for I’m going to say a dozen years. Probably my first AI project was with a very large New York City hospital probably about 12 years ago, where we took a look at heart data from all of their patients, tens of thousands of patients, and used machine learning to predict who was at a greater propensity to have a heart attack. 

Joe  : 

And I would argue machine learning goes back a lot earlier too. My cousin, she’s in her mid-seventies now, she was a programmer in the eighties working on machine learning. It’s just this constant progress, so back to you. 

Lou  : 

Absolutely. In the scientific community, it’s been around, so Turing is older than you and I. 

Joe  : 

Right. 

Lou  : 

It’s in the- 

Joe  : 

The 1950s, right. 

Lou  : 

Yeah. But Copilot is that SaaS solution that you can purchase and enables you to have AI within an afternoon. 

Joe  : 

Well, let me ask you this, because Copilot comes out, there’s this huge buzz. The markets have clearly responded to this. What makes today’s iteration of Copilot… So if machine learning has been around since Turing and just keeps evolving, what’s the big breakaway that Copilot represents? Why is the industry so excited about its introduction? 

Lou  : 

It is the generative AI component. So people love having conversations with bots, and now the bots are more intelligent and more interactive, and really, truly able to interact in natural language. They weren’t able to do that before. Before, you knew, and we’ve had these types of interactive bots for years. You knew when you were talking to a bot and you would say, “Get me to a person.” Now, you’re interacting with them all day long and don’t have an idea. 

The other concept is Copilots are now grounded against the data that you have in Microsoft Office. So it’s grounded against the Microsoft Graph, and that graph includes all of your interactions with people and data. The graph has knowledge built into it and concepts like social distance. So the graph is not just data. You made a very good point. The quality of data is important. The graph isn’t on your data, your graph is intelligence about the data. So when I ask it a question, it knows that my boss is more important to me than someone I speak to or chat with once every three months. It knows that customers are more important to me because it understands, reading through my emails and reading through my chats, what’s of value to me. So it’s that grounding of this information that I use every day and being able to ask questions of it. 

Joe  : 

Yeah, I had heard a term called data deficit recently, and data deficit, it was actually from Microsoft, and that data deficit is, great, I have email and data everywhere, but… No, sorry, data debt, that was it. And that there’s a price to pay for the sheer volume of information we have now, which goes to your thing of if it can tell you which is more important and find it more quickly, this becomes a big part of the payoff. I was incredulous at first, and then I’ve started using it in my just general sales and marketing practice. I’m sold. 

Lou  : 

It’s amazing and I use it every day. This really just appeared kind of out of the blue, November of last year, but we at Microsoft have been using this for years, and I think Joe, you and I have been talking about it for years as well. I think one of the really important things to understand is this is not search, and so you as a consumer of artificial intelligence have to approach it a bit differently. And we go back a long way, Joe, when we started talking with SharePoint and Microsoft bought Fast, a search engine. 

Joe  : 

Yeah, it was 2007. We’ve known each other quite some time. 

Lou  : 

So the average number of words put in a search engine back then was 2.5, so you would just do the shortest amount of prompt possible. With artificial intelligence, you really have to treat it as a personal digital assistant, underline personal. So you have to describe to it as if you were talking to your assistant, someone you’re working with, and give as much detail as possible. So it’s no longer two words. It might be two paragraphs. 

Joe  : 

Right. And actually, when I was seeing that video that Shaili had shared with me, and I want to turn over to Shaili in one second, one of the things I found fascinating when I was watching the how to do it was, and who are you speaking to? And it will actually take on the persona of the type of digital assistant that you should be speaking to, which influences the way it understands and responds, which I found fascinating. 

But if I may turn it over to Shaili for a moment. So Shaili, we have every intention of rolling out a Copilot integration with ProjectReady, but can you give me some insight on your side or some views of where you see this as assistive, both within our product and in general within the AEC? Some of what it solves and what some of those challenges might be? 

Shaili Modi Oza: 

I think all great points we’ve been talking about already, but I think as you mentioned, definitely about the graph data which is accessible through Copilot now is very exciting, that we can access everything in SharePoint and your Teams, your Outlook, all the data that lives in Microsoft, we can access that. And as we start to integrate it into ProjectReady, I think what makes it that much more powerful is we have data of course in M365, we have data in a lot of other systems that- 

Joe  : 

Autodesk, Procore, correct. 

Shaili  : 

… we bring together. Yeah, Autodesk, Procore, all of the workflows, all of this information is in our data warehouse in the database. That comes in together with that as well. There’s data from the internet. We can put your sites of your organization, and anywhere basically that data lives, you can bring it all together in the model that runs Copilot. So I think that makes it very powerful where there’s just so much data to sift through, and as you mentioned, it’s intelligent enough to know what is important to you. And the same tool, if I add something versus if you type in something, it gives a different response because it knows I’m looking for more technical related data versus you are looking for something else. So that is also a pretty good way as users try to start using it on every daily use, that each user will have their own customized response, which is I think very powerful. It’s very assistive. Based on all the data that already exists, it’s going to create these intelligent responses and give it back to you. 

Joe  : 

And this is the thing I always stress about our data warehouse. So you’ll have Copilot for Procore, Autodesk is rolling out their thing. I don’t know if they’re OEMing as well, but with our database, the ability to now extend it into SharePoint, all that stuff comes into our database, all the workflow statuses, content and flight, all of that. And because of that taxonomy, one of the things that I found really intriguing was you can point to disparate data sources and then rank their validity. So one thing to get ahead of is everybody knows about ChatGPT’s hallucination, but that’s because it’s taking in information from the internet, which is basically troll information. It’s not really quality stuff, and the fact that you can throttle that quality is really quite profound. 

And then another thing I want to add to this is a recent use case that came up that we’re solving for one of our major clients. They have as part of… I’ll give you an AEC use case where I think Copilot is going to be really powerful. They do program management so their customer will have a hotel, a casino, public park, amusement park. They manage the contractual deliverables, so they’re supposed to get a certain number of documents and models that exist in Autodesk and M365, and we’re talking about tens of thousands of documents. So the ability to go, oh, okay, how many documents do I have in flight? Can you compare that to this schedule that said my deliverables would due on this date? That kind of capability is huge because to sift through 10,000 documents, sometimes in a week, to work on a billion dollars worth of assets, that is just so inaccurate and so wrought with peril and misinformation. That one use case alone I think just really kind of drives that home. 

If you could ask Copilot, “Tell me where I am on my deliverable schedule and the documents that we’re contractually obligated to receive, by system, by number by date,” and have that conversation, “Well, why is that late? Which vendor is it?” rather than going back and forth, that’s really transformative. 

Shaili  : 

Yeah, definitely. And I think that comes back to what Lou was saying earlier. Why is it so different from search where search is just going to return all the documents for you or all the data for you, but to intelligently sift through all of this based on your question and what you’re looking for, it gives you that correct information. So that way, any of these repetitive tasks of going through a whole bunch of documents and data reduces a lot, for sure. 

Joe  : 

Yeah, and it allows you to compare that information in other Systems. when the internet was new, the fact that I used to do research in the public library when I was in university meant I knew how to ask questions, but not everybody knows how to ask the question correctly. So you go to a Power BI report, you have to have all sorts of stuff built, but then what ifs outside of it, what do you do? Same thing with search. You have to have the right query, but it’s not going to think outside of the most narrowest of scopes. 

So Lou, you mentioned, and Microsoft has, a whole bunch of pre-built stuff, obviously for their M365 stack. It just makes sense. But for the AEC, which is dealing with a whole bunch of disparate systems and the like, how does Copilot help work on what is more industry specific or proprietary data? 

Lou  : 

One of the most powerful features of Copilot, and I’m using the term Copilot generically. 

Joe  : 

Yeah, that’s the problem with Microsoft terms, it encompasses too much. Yes. 

Lou  : 

It’s a very overloaded term, is you have this ability to ground the model, the AI model to your own data, and we refer to that as RAG pattern or retrieval augmented generation. So in essence, what we do is we go out to Microsoft 365, gather whatever information is there. We then go to your local data source, whatever that may be, gather that information together. Combine the two, do governance, do security trimming on that, and send that back to you. That’s all within the confines of your tenant, so it’s completely secure, but not only would you have access to the graph, but by using a RAG pattern, you can have access to any of your own internal proprietary data. 

To take it even a step further, you can actually train an AI model with your data. So if you have some proprietary intellectual property that you feel can be a competitive advantage, you can train your AI model against that data. Again, this is within the boundaries of your tenants so no one will see that, and now you can begin using that to generate even better and more intuitive answers. So there’s a whole realm and a whole world of grounding and model types that we can explore. 

Joe  : 

And you can rank it too, is what I had seen. Zero to five or whatever it was in terms of, eh, trust it and don’t trust it, which is important to ensure quality. But what about on-premise information? 

Lou  : 

The on-premise information is very, very difficult to get to. We just recently began releasing Azure search services, and so rather than actually getting to the information on-premise, you would index that information, then you can get to the index, but really, AI is built for the cloud. 

Joe  : 

Yeah, everything these days, let’s face it. 

Lou  : 

Everything’s built for the cloud, and it makes sense. You need a tremendous amount of horsepower, you need the elasticity. You need to be able to access many, many, many different sources of information. We have another product called Microsoft Fabric, and the idea behind Fabric is we understand how difficult it is to bring together many, many different sources of information, and this would be one approach to doing that. But in your case, you’ve already solved that problem. You’re bringing all those sources of information together. 

Joe  : 

Yeah, because Shaili, you can talk to this, we have what we call as the SQL connector, and so if one of the challenges is your data is kind of sloppy, inconsistent, and some of that is on-prem, I would think there’s an opportunity to start to just fetch the data that you know you need and start to replicate that to Azure. Do you want to pick up on that, Shaili? 

Shaili  : 

Yeah, definitely. I Think the whole concept of, we call it a SQL connector, but we have clients who have data that’s in an on-prem database and using a connector. We basically, as you mentioned, all the important data points relevant to projects, we would basically bring it into the Azure SQL database where all the other data resides. So on a schedule, it brings all of that data and then it has a lot of possibilities. You can mash up that data with everything else that we have up in our Azure SQL database. So we have data coming in from all the cloud systems, we can bring the on-prem data points in there as well, and we are already logging and tracking workflows from all of the different systems in ACC or BIM-360, Procore. So we have that architecture in place where all of these systems, they’re all putting their data into this one scalable data warehouse, and that makes it that much more powerful that now we can just connect our Copilot and use that as a data source that has information which is already organized. 

We also take care of the security because we manage security as well. So we would know that, okay, these users would have access to this data, and that is also a very important aspect, that you would only need to see the data that you’re supposed to. When you basically try to get these answers from Copilot, the security aspect of it is also very important, so that’s one thing to keep in mind as we are setting up projects or M365, to make sure that the teams are set up correctly, the users have the correct level of access to the sites, and that will ensure that all the data is secure when they try to access this information. 

Joe  : 

And that cascades across the different data that we bring in. They can ask something, but if they don’t have access to Autodesk data as part of our warehouse, they don’t get it. Correct? 

Shaili  : 

Yep, correct. 

Joe  : 

And the way I look at it is my hat look. Sometimes, it’s easier trying to move things progressively into a new room than it is to straighten out a room, right? Because at the end of the day, most enterprises are comprised of hoarders, and the AEC in particular, I would argue they save everything from time immemorial. So the ability to selectively go, “Sage on-prem, this is where I have my project, accounting information and information, what RFIs I had to pay for. Those RFIs originated in Autodesk or Procore’s cloud and was answered in documents inside SharePoint.” Now, you’re bringing together all these disparate data sources so you can say, “How many RFIs from this customer has cost me what?” And analyze all that data and get a clean start, somewhat pun intended, to clean data. Is that a reasonable supposition? 

Lou  : 

Absolutely. And it’s not so much the quantity of data. So depending upon the type of AI, quantity is important, but it’s the quality of the data. And I always use the analogy, I’m going to clean my garage before I move from one house to another, right? I’m not going to move all my garbage with me and then later. 

Joe  : 

Exactly. 

Lou  : 

And it’s a very, very good exercise and a good process to go through. And typically, I work in every industry. Now, I’m focused more on manufacturing, but when we typically do this, we see as much as 40 plus percent, 45% or so of data being removed, because data is both an asset and a liability. 

Joe  : 

Correct. And you mentioned manufacturing. What we, I’m proud to say, Autodesk is publishing, I think we just announced it today but we have it out there for a while, a case study we did which is around our client, CRAF, who’s a major manufacturer, but they have to deal in CAD and they have to distribute documents, and they use SharePoint and M365, so please do look it up. But they’re a manufacturer, and you go, but manufacturers are owners, and in their case, they do design. So the AEC, just as a related note, it’s a lot more than architecture, engineering and construction. It’s the owner, it’s manufacturers, anybody who has to deal with a built asset gets lumped in. Microsoft is an owner. I think you own a few buildings in the world right now. 

Lou  : 

Yes. Yeah, we are spending $10 billion a quarter on new data centers and improving our data centers, so we’re continually buying. I wouldn’t be surprised if we become one of the largest real estate owners, private real estate owners in the world within the next couple of years. 

Joe  : 

Right, and there you are, a software company building data centers and office spaces, and that goes right back to the built world and the same challenges. Because for Microsoft to manage its campuses and its data centers, my God, the millions of documents running around, to build them, maintain them, understand them, retrofit them. Well, again, through Copilot and the ability, and we would argue we’ve already solved it but there are ways you can solve it on your own, to bring all that data together intelligently really will drive Copilot’s bang for the buck, as it were. 

Lou  : 

One of the things that I don’t think many folks realize is that because we have so many different properties, whether it’s Xbox, whether it’s Live, whether it’s Microsoft 365, we have more data centers than our next two competitors combined, so it’s a massive, massive footprint. We are always in the building phase. 

Joe  : 

I think Microsoft should look us up, but that’s a different issue. 

Lou  : 

Sure. Last week, even President Biden was on television talking about the new data center we’re building in Wyoming, a $3 billion AI-focused data center. 

Joe  : 

Yeah. Well, it’s all about infrastructure at the end of the day. 

Lou  : 

Yes. I think this is a potentially waterfall, watershed moment for us, and the companies that adopt this technology and adopt it early, of course, Shaili made the point about security and we don’t want to go run down the hall with scissors. We want to do this at a market pace, but I think this is an opportunity to gain advantage on our competitors. It’s a chance to be able to gain productivity, but translate that into… As you mentioned, Joe, maybe people aren’t going to lose their jobs, but maybe over the next 18 months to two years, maybe not having to hire. Being able to expand my business, being able to increase my revenue without having to increase my cost. 

Joe  : 

Well, and in the case of the AEC, with the avoidance of rework. That is a huge pain within the industry because, I don’t know. Is that right? Is that right? Because just of the reams of data, not because people are silly like that. 

Lou  : 

I could tell you a funny story about the cube on Eleven Times Square. We have this massive cube that’s part advertising and part design of the building, and that cube was six inches off, so the cube had to be redone. 

Joe  : 

Right. 

Lou  : 

Can you imagine? There’s a cube there on 8th and 42nd Street, seven stories in the sky. Any change to that- 

Joe  : 

Right, and that could fall. And back to that built world thing, there is more than enough for us to do, right? I’m very big on carbon capture. I think that’s one of the most important things being looked at now, but with data centers being built, how do you do cleaner energy? How do you accommodate all the infrastructure required? There is no end to the need for skilled professionals in the built world to build more, to build better and to innovate more. But if they’re spending half their time dumpster diving for information and then coming up wrong a third of the time, nobody loses their job, everybody gets a bonus. That’s the way I look at it at this point. 

Lou  : 

Yeah. 

Joe  : 

And the world will be a better built place, no pun intended. 

Lou  : 

One of my colleagues say there’s going to be two types of people, those who adopt AI and those who don’t. The ones who adopt AI will continue to work, will continue to thrive. Those who do not are the ones who are at risk. 

Joe  : 

Right. And this is Moore’s law in action, kind of, but you can say the same thing of people who were in a room with a green visor and a chalkboard, and then, oh, what happened? Mac came out and Lotus 1-2-3. And so you either learned how to use a spreadsheet or you no longer worked in the stock exchange. Fair enough. But the stock exchange has grown since, so there’s clearly been upside to the adoption of Tech. Shaili, is there anything else you would add? 

Shaili  : 

No, I think we summarized pretty well. 

Joe  : 

All right. Well, I hope this was informative for folks out there. There is so much hype, and Wired had a piece out there that this is the year of disappointment for AI. It’s only disappointing if you don’t understand it and know how to adopt it. If you do, it’s the best thing since sliced bread. It will be the most impactful thing to profitability and efficiency ever known, basically. So we are hoping to help you get past that hype, understand the practicality of it. Yes, it’s a really smart chatbot, kind of, but it’s a lot more than that and the ability to target what the information behind that conversation is is really the game changer. So anyway, I thank you again, folks. Talk to you next time, and thank you for joining. 

Lou  : 

Thank you. Be well.