Breaking Through Manufacturing Challenges with DxOps Transformation

Speakers

Samantha Willins

Project Engineer

RoviSys

Drew Daff

Lead Software Engineer

RoviSys

Anh-Tuan Tran

CPG Group Manager

RoviSys

Learn how to combine consistent processes with novel concepts to break through challenges in downtime tracking and OEE visibility. In this session, RoviSys will share how their DxOps Transformation approach, used at Nice Pak, helped overcome high variation between production lines and facilities, lack of data connectivity, extensive turnover, and data integrity gaps. Learn how to standardize integration methods for better scalability and real-time tracking and see how these solutions can enhance efficiency in your facilities. Don’t miss this "how to" guide on transforming challenges into opportunities for breakthrough success!

Transcript: 

00:00
Jesse Oehler: How's it going? So my name is Jesse Oehler and I'm a Technical Recruiter, Inductive Automation. And welcome to today's session, which is with RoviSys, titled, "Breaking Through Manufacturing Challenges with DxOps Transformation." I'll be the moderator for today's presentation. To start things off I'd like to introduce our speakers. Here's Samantha Willins. She's a Project Engineer with RoviSys in multiple vertical markets. She started using Ignition in 2018 and now leads the RoviSys Global Ignition Technical Committee, which is focused on novel platform solutions and information sharing. She ensures RoviSys resources around the globe stay up to date on the latest Ignition features and functions, promotes participation in forums, and drives innovation by identifying project accelerator opportunities. We also have Drew Daff. He's a Lead Software Engineer at RoviSys and he's part of the Digital Transformation and MES division. He has experience in designing and implementing advanced MES solutions across entire industries and regions.

01:08
Jesse Oehler: His involvement in numerous Ignition projects has addressed customer and configuration challenges, driven consistency, and improved performance that results in faster customer ROI. We also have Anh-Tuan Tran, who's a CPG Group Manager at RoviSys. He has more than 18 years of automation experience tackling customer challenges, ranging from DCS, PLC, HMI, SCADA systems layers to complex MES integrations. Since discovering Ignition in 2017, Anh-Tuan's been a continuous advocate for Ignition, enhancing data usability for manufacturers through both out-of-the-box and customized platform solutions. As a manager at RoviSys, he leads a team specializing in information data systems and OEE configuration for CPG companies. Please help me welcome our three speakers today.

02:05
Samantha Willins: Alright, well thank you for the introduction and thank you for having us at ICC. We're really excited to be here. I know I am. So, I just wanna say that we all know the theme is "Breaking Through," right? We're gonna talk to you about how we break through hurdles on our projects, and specifically we're gonna talk about a project we did with Nice-Pak and how we ran into a lot of hurdles and how we broke through those challenges to deliver a successful project. But first we wanna tell you a little bit about our company.

02:33
Anh-Tuan Tran: Hey everyone. I'm Anh-Tuan. I'm from the North Carolina office. A little bit about our company, if you haven't heard of us. We were founded in 1989. We're based in Ohio. We were founded by our CEO John Robertson, so he makes up the "Ro" in RoviSys, if you guys don't know how we came about our name. Really when we started, we started in the process automation space. But through the years as we hired different people with different expertise, now we actually offer a number of different capabilities in a variety of vertical markets, and then a variety of offerings in terms of implementations.

03:10
Drew Daff: Hi everyone. I'm Drew. I'm based out of our Ohio office. So, like Anh-Tuan mentioned, today at RoviSys we still do a lot of process automation work. But we also have a dedicated group that helps deliver turnkey solutions for other types of projects such as MES implementations, electrical design, setting up OT network infrastructures, integrating artificial intelligence or documentation services.

03:40
Anh-Tuan Tran: And as you can see here, we are a global organization. We have about 17 locations worldwide. We're just strategically positioned so that we can service our customers within that region. One of the things that makes us really unique is that across all the different locations, we share resources and work on projects together. This just allows us to put the right resources on the types of projects that we're implementing for our customers. The goal is really to give them that small company feel when we're working closely with them, but by being strategically positioned like this, we can reduce costs and improve the efficiency of implementations.

04:19
Anh-Tuan Tran: Alright, I'll talk a little bit about Nice-Pak. This is the customer that we did the project for. Right after the pandemic, a colleague of mine in our Georgia office introduced me to Nice-Pak. When they came to him, they actually had a Industry 4.0 initiative. And so, they presented a vision to me and asked me if I can help them on this journey, right? So we've been talking about Digital Transformation journey and Nice-Pak, if you don't know about them, they invented the disposable wet wipes. And so, right after the pandemic, their business was booming and they needed a way to really maximize the efficiency of production because the demand was so high. And they came to us and said, "Hey, what can we do? Where can we start?" And one of the things that we did was try to sit down with them and figure out a good starting point within their vision, 'cause there was a lot of stuff going on that they wanted us to help them with in terms of getting them along on the journey.

05:16
Anh-Tuan Tran: So, in this presentation, we'll talk about the project goals and deliverables for the project that we did for Nice-Pak, but one of the things that we're gonna do a little bit differently is beyond just talking about how we use Ignition to deliver a successful outcome, we're gonna focus more on the obstacles and the surprises that we encountered. So, it's more focused towards strategies for overcoming these challenges. And then what we're gonna do is we're basically going to give you guys our secret sauce. And as a system integration company, we don't actually have a product. A secret sauce is really just the proven strategies that we've developed through the years as we encounter multiple projects like this. What we've done is we've actually packaged up all of that knowledge and all the lessons learned that we've done from the project. We packaged it up into documentation and a game plan for us to approach these projects so that we can deliver a better solution. And really our strategy is just to mitigate risk and then create successful outcomes.

06:19
Samantha Willins: And this whole strategy is called the Digital Ops Transformation, which is where the "DxOps" comes in the name of the session.

06:28
Anh-Tuan Tran: Alright, a little bit of background to the project. I'm gonna briefly kind of touch on the different things that we did. But for Nice-Pak, we started with an OEE and downtime tracking project. And that was really just a good starting point for them. They were tracking OEE in downtime, in Excel spreadsheets. It was really bad. Like their dashboards were all in Excel. There were a lot of tabs and they were like, "Hey, it's gonna be really straightforward just to digitize this guys. Why don't you just take all these paper processes for us and then digitize it?" So, they had a paperless initiative. They call it the paperless shop floor goal, right? So, what we did was we used Ignition to digitize these process, but one of the things that we had to look at was, hey, if we don't really take the time to understand their future goals, if we come in with the technology and implement it, that might be the only project that we do with them. So our strategy was, take a different approach so that we can figure out what is the right starting point for them.

07:38
Samantha Willins: Oh, did I hit the wrong button? Sorry.

07:41
Anh-Tuan Tran: Yeah. So like for us, since we are an independent integrator and we're technology agnostic, when we go into these projects, we don't actually think of like what platforms to use right off the bat. The way that we approach it is that we have a architecture assessment process and we basically get all the stakeholders together and sit down, figure out what their business drivers are, and then from that understanding, we start building a picture of, what are the right platforms for them? So, for us, we basically have a scorecard, right? And we have a number of technology stacks that we look at, and then we start checking off the boxes and give them ratings. And in this instance, as you can see here, Ignition just checked off so many boxes for this customer, a lot. And so, they ended up becoming the solution that we chose and when we're in this process, we basically give them the pros and cons of all the different technologies, and then we also try to give them as much clarity as possible as to why we recommend this technology to them.

08:37
Anh-Tuan Tran: And then this allows the customer to see the transparency and the independent mindset of like, "Hey, we evaluated all these platforms, but this one fits the best for you guys and here's why." And they really liked that. And one of the things about Ignition that was great was that it was very, very scalable. And that was something that they were very focused on, was like, can I take this solution and then port it to other sites and scale it out, and then can we templatize all this stuff? And Ignition just fits the bill perfectly.

09:07
Anh-Tuan Tran: So, when we started this project, Nice-Pak is a global organization so they had multiple sites worldwide. It wasn't in their budget to do it at all the sites. They wanted to implement the solution at all the sites, but really they only had the budget for two sites. So we picked two sites that were similar but also vastly different. So the first site that we concentrated on was in their Mooresville, Indiana plant. This was an older site. They had aging infrastructure, a lot of their equipment weren't even connected or automated. So, in contrast to that, we also picked the Jonesboro, Arkansas site, which was a new facility for them, very modern, everything's automated, everything's connected. And this was kind of the two sites that we had to implement a solution that was global enough and templatized enough that we can normalize that data and show them that we can take this solution. If we can scale it at these two vastly different sites, then they knew they could take that solution and scale it out to all of their other sites globally.

10:03
Samantha Willins: Awesome. So that's a little bit of background on the project, so let's give you the secret sauce, right? Alright, so it may sound a little bit backwards, but it's been mentioned even in the presentations you've seen so far. We always start with the outcome of how operations will utilize the solution to get value. We have all the appropriate stakeholders in the room and we really like to have everyone down from operators up to the people that are handling the money. We like to have everyone involved because they're all using it. This is what we call our operational scenarios. So, for this solution, operations is gonna be using dashboards, user graphics, and reports to drive improvements in behaviors. And then we also do this whole process through our functional design specification design process. And that's creating user stories, mockups, and identifying those edge cases that can kind of trip you up if you don't look out for them in advance.

11:00
Samantha Willins: So once we start doing that, we start looking up at information flow, right? What data is available and how can we use that to design a structure? This is where you're gonna create your I/O lists and you're gonna start really just standardizing your tag naming conventions and things of that nature. Once you have that structure, you're gonna start developing base configuration and templatization and that's gonna be used across all the customer sites, right? So we always templatize the plant model to ISA-95 standards. And then we use built-in templatization and Ignition to build out the UDTs, the graphical templates, the lookup tables, and the global objects that are gonna be utilized across the enterprise. So if anyone ever moves locations, it'll be pretty obvious there. For enhanced features, those are gonna come into play for those edge cases we identified in the design phase. This is where we're gonna start really honing in on the equipment where it acts differently or just isn't connected at all. Then after that, we use our master data-driven configuration to normalize all that different data coming in and connect it within the Ignition tag database or an OPC server or MQTT broker or anything like that to just make sure that we have all the data where we need as we need it.

12:18
Anh-Tuan Tran: So keep an eye out for these icons on top of these slides. These are our strategies. And so, this is a strategy for having a scalable model. You can take this and implement it in any company and if you follow this approach, this is an easy way to make sure that your solution's gonna be scalable.

12:36
Samantha Willins: Yep. So, let's talk about the network architecture a little bit too, 'cause that's kind of a starting point, right? So their architecture was fairly simple. They had Ignition Edge gateways down here at the bottom that connected to the plant field I/O. Then it fed up to the corporate network. And I know it sounds a little scary, you normally don't want to do that. However, all the I/O was configured as read only. So, it was only going up, it was never coming back down, so they didn't have a worry there. With that, they also installed Ignition clients at each of the equipment stations for downtime reason entry, as well as they had giant 75-inch Samsung TVs with industrial PCs on the back as monitors for around the plant for metrics monitoring. So big goal about doing the network architecture was we really wanted it to fit into their existing structure and complement it, right?

13:31
Samantha Willins: You don't want it to be kind of standoff or be noticeable that it's an addition. You want it to merge well and flow with the existing architecture. And that was the goal here. So, let's kind of look at what the solution was. So, as Anh-Tuan mentioned, they had Excel sheets that were very complex and a lot of tabs. Anybody looking at these, they're going to, it's not intuitive. They're gonna sit here and try to figure it out for days at a time. So it was very complex, just not ideal for what you want to do. So this is what we made. We did a more intuitive theme, dark-themed application where you can look at this and hey, right, there's my OEE, I know it's right there, it's very clear. The good part about this also is it had multiple value streams. So this template dashboard was used for all of their value streams at both sites.

14:22
Samantha Willins: So their downtime, guess what, it was also tracked in Excel. So we also brought this into a more modern age with that as well. So, they were actually having people go down and count the downtime minutes and try to figure out what data was actually useful for the downtime. And it was a very tedious and long process. So we transformed it to this in a way that's easier to digest. It's very simple. You can see it. I mean, they already have the data, so why not put it in a way that makes it easier on them and saves them time. So we're actually, if you click these buttons up here, I hit the wrong button, buttons, anyways. So we actually have the ability for them to create downtime reason entries from either selecting from the current or even creating a new entry for the reason of the downtime. This is gonna help them identify the issues that are causing the most problems, ways to fix things if they notice that certain issues are arising, what they can do to save money and time in the long term. So, this is the solution but we're gonna have Drew talk to you about how we even got to this point.

15:36
Drew Daff: Thank you, Samantha. So, the room here is filled with people that know how to develop solutions. We all know that. What we believe at RoviSys is the true test is navigating the unexpected. So, the roadblocks, the hurdles, and the unforeseen challenges that arise along the way. So at RoviSys we anticipate these issues. So our real story is about our process to consistently break through these challenges to achieve success.

16:02
Drew Daff: Like Anh-Tuan mentioned before, the two plants we worked with had some similarities, but they also had a lot of differences. We had shifts that were different. We had operator processes that were different and the process to import schedules was also different. What our process allowed us to do though was capture these differences so that we could plan out how to deliver a consistent look and feel with not-so-consistent data. As we all know, customer processes change and what our process allows us to do is reanalyze the differences and figure out how to work around them and get the results that we need. For example, during our in-process design reviews, one of the plants saw what the other plant was doing and recognized it was a better process. So what they wanted to do was basically do exactly what that other plant was doing.

16:48
Drew Daff: Even though things weren't the exact same between the plants, like I mentioned, we adjusted and were able to give them very similar outputs. We also ran into very challenging edge cases and for these types of challenges, we have a process for our team to bring challenges that they're stuck on and go through them with some of our experienced developers that have gone through those experiences. We call that the RoviSys Architecture Review Board. And so, what that is, is a group of developers from, you know, all over the world at all our sites that bring in a lot of different experiences. And what they did was help us brainstorm solutions for functional gaps that we faced on the Nice-Pak project so we could rework solutions and avoid potential adoption issues once we were finished.

17:29
Anh-Tuan Tran: And this has been incredibly useful for us just to kind of identify SMEs within certain topics and then once we bring a problem to them that our team couldn't come up with, that team had a whole host of solutions that they could provide to us. And that made it so much faster to solve problems, especially when you can leverage that expertise and kind of have an organized group that did that work.

17:56
Drew Daff: Yeah.

17:57
Anh-Tuan Tran: So one of the earliest challenges that we had whenever we got into a Nice-Pak project was that they really had their own language or jargons that they used. And I think you guys, when you're working with customers, you're probably running into the same situations, their own acronyms. The things that they say when you interpret it is really not what they think, 'cause things are interpreted differently. So, when we start these projects, one of the things that we have to do is really sit down and kind of build that glossary so that we understand exactly what they're talking about. And then that allows us to synchronize with the team so that our team understands it as well.

18:31
Drew Daff: Yeah. And a big part of getting things right is, as we all know, learning what the heck the customer actually wants and what they need. So, often overlooked in projects like this is the voice of the actual users, that are living in the application day in and day out. A typical functional requirements document only outlines the technical requirements. So, but what it does is often misses how the system is really used by the people, like I mentioned, that live in it day to day. So, what we've done is we've built a process which helps developers understand the situations that operators and the users are in, so they can design and develop with those situations in mind. So, just a guess, a show of hands here if I can see. Who here has heard of SIPOC and KDD processes? Okay.

19:20
Samantha Willins: Yeah. A few.

19:21
Drew Daff: Yeah. A few. Good. So, for those who haven't, SIPOC stands for Suppliers, Inputs, Process, Outputs, and Customers. So what this is, is a structured high-level map that helps explain the start and endpoints of a process and what's needed to go into that process. So, the inputs and the outputs. And this is all from the user's point of view, which is the important part. It's extremely helpful, not only in design, but like I mentioned earlier, the adoption of the finished product. So, moving on to KDDs. KDD stands for, Key Design Decisions. So, this is a process for documenting decisions and basically the why behind them. The why is very, very important because it helps understand the rationale behind the decisions that were made. A good example of this would be if someone forgot something, you have these decisions documented so you can go back and figure out why those decisions were made and move on.

20:21
Anh-Tuan Tran: And we didn't invent those terminologies. So, if you guys don't know it already, make sure you guys read up about it and learn about it, 'cause as we're implementing it, it's working really well for us in projects. So, it's just knowledge that we're trying to share, you know?

20:36
Samantha Willins: Yeah. These are not our acronyms, it's not just random letters for you.

20:43
Anh-Tuan Tran: Another obstacle that we encountered on this Nice-Pak project was that you know, when I sold this project, the customer told me that, "Hey I'm gonna give you all the tags that you need to check downtime." I was like, "All the tags, I don't have to put a control guy on this project?" And they're like, "Yeah, yeah, yeah, yeah. We got a guy and he's gonna give you all the tags. Don't worry about it. So you don't have to scope that piece." And we're like, "Alright, you give me all the tags. I can configure this whole application for you guys." That did not happen on this project.

21:16
Drew Daff: So, yeah. So just a quick word of advice. Don't always anticipate your customer's gonna give you exactly what you need. We all know how that goes.

21:25
Anh-Tuan Tran: Yeah. So, one of the things that we had to do on this project was that, when we realized the customer wasn't going to give us the PLC tags that we needed to give us the triggers for the downtime events or the counts to calculate OEE, the way that we staff the project is that we don't force or have a specific team in mind. We staff it based on the needs of the technology skill sets that we have at the duration of that timeframe. So when we identified that we needed a PLC person to come onto the project because they needed to pull up some Rockwell code to look into the Ladder Logic and figure out these triggers, we just created a staffing need for the duration, and we were able to staff resources from other sites. And they were able to come on for that short duration, got all the tags for us, built out the I/O list, and then we were able to continue with our development. And this is really important because it allows you to be very agile in terms of implementation, make sure that you don't lock in the team so that if you run into a scenario, you can adjust and pivot.

22:31
Drew Daff: So, another challenge that we really faced, and it was fairly big, as we all know, cyber threats restricted all network access for our customer. But fortunately for us, our implementation plan that we've gone over so far, meant that backups were available and stored in our local repos, allowing us to restore the system. What we did was we used this downtime to build our readiness checklist to make sure everything looked good to go when we were ready to get the system back up and running for our customer. During this time, like Anh-Tuan mentioned, our staffing model also allowed us to offload resources temporarily to stay efficient. And once that access was restored and everything was back up and running, we worked with our customer to get a fresh list of requirements that were needed to help get the system to optimal status.

23:17
Samantha Willins: And a good part of this too, is when we offload those resources, it saves the customer money. So, they're not paying for those resources to just sit. We're able to utilize them on other projects momentarily, to make sure that they still get the value for their money.

23:32
Anh-Tuan Tran: Yeah. And we're downplaying this event a little bit, but when we're hitting stage gates, we always do a major backup of everything that we develop. And that's just something that we do. And we have our own repository and in return that has saved us in a lot of scenarios when especially an event like this where you can take everything that you've saved, restore them, and get them back up and running within a day. So, it was very beneficial to get them back into production with a very short amount of time. And that kind of helps build into the cyber resiliency model that we have.

24:06
Drew Daff: So, I do have another question for everyone, and I don't totally expect an answer. So here we go. What do you guys do when your customer asks you for something important and you can't find it? So, this is an unfortunate situation we had to deal with on this project. Turnover dig at the best of both sides. So, the Nice-Pak side and our side, both the project managers on both sides of the project left and everyone that was involved with and executed SAT also left. So, we were kind of just dead in the water. So what basically happened was we couldn't find our executed copy of the SAT. The thing about RoviSys though, in this situation is we always try and do the right thing with our partnership mindset that we've gone over, our team adapted and did what was right. We broadened our customer success team and we were honest with Nice-Pak and let them know that we would be re-executing the SATs at our cost. This transparency and honesty is always something that is important to us and always will be. It's big with our partnerships. What helped to make this tough situation a little better is the fact that Nice-Pak was able to help us out while we were executing the SATs remotely. They were helping us out on site.

25:15
Anh-Tuan Tran: Yeah. And the key here is just work with integrity. Like if you don't know a situation, if you're just honest with the customer and you can kind of go and just tell them, hey, so and so worked on it and so and so worked on it and they're both not here in the company. We'll just re-execute the SAT and give you guys signed SAT that way you guys can have a more validated system.

25:37
Samantha Willins: Yeah. And those hard conversations are always hard to have, and no one looks forward to them. But I have known on multiple projects, the more transparent you are, the better the results overall. So...

25:52
Anh-Tuan Tran: Yeah. It's, show of hands. Who dealt with like turnovers or having to pick up from where someone left off or cleaning up other people's mess.

26:01
Anh-Tuan Tran: Alright. Definitely not unique just to us, right? So in this project, the craziest thing happened. We got the solution to the "go live" state. Now it's in the production floor, they're using it. And at the Mooresville plant there's a champion, at the Jonesboro plant, there's a champion, on our side, we have a project manager. And I don't know what happened, but at the same time, everybody turned in their two weeks. And that whole entire team was disbanded. Nice-Pak's like, we gotta assemble a new team. So when that happened, we were like, alright, what do we do? Because there's still a period where they're evaluating the solution and telling us whether it's working well, are the counts accurate? Are we doing the calculations? Are the reports working?

26:46
Anh-Tuan Tran: So, what we did throughout the project was that we just documented everything really well. So our function design specification, when we developed it, it was our engineering cutoff. But as we made and implemented the solution, we took screenshots of what we had and we plugged it into the FDS so that we have snapshots as we went through each stage gate to capture the as-built state that we were developing for the customer. And that really saved us in this scenario, because when they put on a new team and we assembled some folks to come in on our side to take over the project, that document allowed them to really understand exactly what was going on with the project, what are the deliverables, what our goals are, and how are we doing the calculations? What tags are we looking at to pull in these triggers and then what we're doing. So by dumping all that into one document, one central location, the onboarding process was a lot easier. So by having good documentation standards through your project, whether it's part of scope or not, we recommend that you always do it because it can save you a lot of time and headaches later on in the future.

27:54
Samantha Willins: Alright. So you heard a lot of challenges here. So, big thing here is all these challenges, I mean, they would typically result in a failure of a project. It would typically be very bad. After all the statistics show that Digital Transformation projects do not have very high success rates. I don't know why our image did that up there.

28:19
Drew Daff: But by adhering to our proven methodologies that we've gone over, we've transformed a high-risk project into a resounding success. And you can see we have some quotes up there from our customer representing that success. So, like Anh-Tuan mentioned before, keeping an eye on those icons on the screen up there. So throughout these challenges and obstacles that we've gone over, this is our secret sauce. What they represent is, like I said, and Anh-Tuan mentioned a couple of times, our secret sauce, digital operations transformation. So, what that is exactly is a set of scalable, repeatable processes that when followed consistently help us navigate, avoid, and overcome challenges on even the most complex Digital Transformation projects like we've gone over today.

29:06
Drew Daff: So, all these icons you saw are a specific set of service processes that we've standardized at RoviSys. Our team has decades of experience in Digital Transformation and all the scars that go along with it. So, what we've done is we've taken all that tribal knowledge that we have and expertise for delivering these successful solutions and packaged them up into these offerings that you see up here on the screen. By following some of the same standards we use, you can also de-risk your own projects and break through your challenges to success. There is a lot more behind these these standards here than just some pretty cool-looking icons. And we're happy to connect you with our digital operations team. And we also put a QR code up there for everyone to scan. It'll take you to our Digital Transformation website. And yeah.

29:56
Anh-Tuan Tran: Yeah. So, the big takeaway here is, as you are encountering these projects and you're working with customers, this strategy works guys. If you've kind of follow the format that we have, we know that this is a proven strategy for Digital Transformation projects. If you implement kind of that structure, it will mitigate the risks on those projects and then make you successful. So, if you have any questions, come find us.

30:20
Drew Daff: Yep.

30:21
Anh-Tuan Tran: And then you can ask us about these cool shirts.

30:23
Drew Daff: Yep. Cool.

30:28
Jesse Oehler: I can come on again, thank you three. Appreciate it.

30:29
Drew Daff: Yeah.

30:30
Jesse Oehler: We do have, we've got a little time left in our session, so we could have a little Q&A if people have questions, go ahead and raise your hands and we'll make sure you all get a mic. If we have more questions than our time allots, then these three will hang around for some additional questions afterwards.

30:50
Samantha Willins: You should be able to identify us.

30:52
Jesse Oehler: Yep.

30:56
Audience Member 1: First I wanna say thank you. That was a great presentation. I really appreciate your transparency and willing to share information. Could you talk a little bit about the initial part of the project, like the RFP process do... Did you find that the request for proposal had all of the necessary information contained therein to accurately estimate the scope of the project at the two different sites?

31:21
Anh-Tuan Tran: Yeah. So during the RFP process, actually first what they gave me was a PowerPoint that had their Industry 4.0 goals. And their goals were like really simple. They're like, "Oh, we want an ERP system that's totally integrated with MES. We want a digital CMMS or a digital QMS system also feeding information to our MES. And then everything's automated, everything's connected, and then everything's funneling up through the DMZ to the enterprise." I was like, okay, that's crazy 'cause you guys don't have any of that stuff right now. So we looked at it and we're like, "Okay, what is the biggest pain point for you guys right now?" And they're like, well, we're doing OEE and downtime tracking in Excel, and we print it out and fill it out, and then we go on the shop floor. And that's the easy starting point for us.

32:07
Anh-Tuan Tran: And they gave us these calculations of how they wanted to calculate OEE. They gave us their reason tree, and at both sites they also gave us their production schedule. So we looked at it and then we were scoping it out and it allowed us to kind of understand a good starting point to implement. But the challenge for us when we were scoping out was that they really thought they had all the data to give us to feed the calculations for OEE, and that was a gap. But what they did was they said, "Hey, let's grab some of our OPE budget on some of the screens and monitors that you guys were gonna buy and move that to labor, so that we can cover the cost of the controls engineer that you're gonna bring on site and work directly with our plant floor."

32:53
Anh-Tuan Tran: "And then we're gonna reduce the amount of screens that are on our line." And so we kind of like just worked with them. We were very, very flexible because it's money, like I don't care where you pull it from. If we can move it in and staff the resources and have the hours to do it, we did it. So we just made it so that we were in collaboration with the customer. And our goal is like, "Hey, if you can make their lives easier, they're gonna recognize that and they'll continue wanting to work with us." So that's our strategy.

33:20
Samantha Willins: Yeah.

33:20
Audience Member 1: Okay. Great. Thank you very much. And again, great presentation.

33:22
Anh-Tuan Tran: Thank you.

33:23
Samantha Willins: Thank you.

33:24
Drew Daff: Yeah. Thanks.

33:27
Anh-Tuan Tran: Right front...

33:31
Audience Member 2: I think one of the other presentations this morning was about the value of the Ignition community. And I'm really interested in OEE. Are you guys gonna publish any of that stuff to the [Ignition] Exchange?

33:41
Anh-Tuan Tran: We are trying to publish anything that is very reusable to the Exchange, to share the community. We created like Pareto charts that we've used in the past. I think some of the downtime screens that we've developed are at a level where it is customizable enough that we can share that, so that people can create their own downtime reasons. But we actually have a board and a group that are trying to build these reusable resources that we can share to the community. I don't know exactly which ones are selected because it kind of have to be standardized in a certain way... Yeah.

34:14
Samantha Willins: Yeah. So we have multiple different side projects going on, especially as we have projects that want things that are reusable, like so. So if we notice that there's a lot of a need to it, like PlantPAx and Ignition is huge. And we have lots of projects that want pretty much the exact same display but in Ignition. So, we actually are working on ironing out the last few minor details of that so that we can actually share those things, and we share them throughout the company as is with those alterations. But...

34:46
Anh-Tuan Tran: Yeah. But short story is we're Premier Integrators. We will contribute stuff to the community and downtime, especially downtime reasons entry is a great, it's a great use case that we can supply that.

35:04
Audience Member 3: You mentioned that one of the two plants recognized that one of the other plants was doing something really useful and they wanted to adopt it. Can you be more specific about what that was?

35:14
Anh-Tuan Tran: Yeah. It was really in the way that they were doing their shift management. The plants had different shifts, and they actually have a dedicated planner that was generating the shifts. And one planner was more sophisticated in terms of how they organized that information. And the way that we did it was we were able to pull in an Excel spreadsheet into Ignition, so that we can build the shifts because we had to correlate that data back to their downtime reasons and their OEE metrics for specific trips or shifts or specific SKU. And so, when the other facility recognized that, wow, your spreadsheet's so much better, we're gonna make our planner sit down with the other planner and learn how they're using their spreadsheet. And then that allowed us to really just make a few iterations of that spreadsheet so that we can import it back into Ignition. And then we actually pulled in their production schedule, so that when we display it on the dashboard templates, they knew what SKU they were producing, they knew what their targets were, they knew how much they needed to produce that day, and then how much one shift was doing against another shift. And then each value stream used the exact same template. Everything was plug and play. The UDTs just worked perfect and it made it really, really easy for us to implement and scale.

36:29
Audience Member 3: Did the old plant emulate the new plant or vice versa?

36:34
Anh-Tuan Tran: Yeah. It was actually different. The new plant emulated the old plant, 'cause the old plant was so outdated that they made a lot of creative solutions in Excel. So they were way more sophisticated in Excel than the new plant. So, yeah.

36:51
Audience Member 4: Hey, thanks for the presentation. I think at the start of the presentation, you had a slide there on the architecture. I'm just wondering, is that a, like from looking at the architecture, it kind of seems like maybe there was some work done in terms of Digital Transformation. I'm just wondering, was that something you guys did or would you come in at a later stage and finish off the project?

37:10
Anh-Tuan Tran: Yeah. So when we came in they wanted everything to sit on their Wi-Fi network, because we had to give them, so we use Elo tablets that allow them to have this touchscreen capabilities to kind of just enter in the downtown reasons. And then the dashboards were just Samsung TVs, but we needed a way to funnel that information but also protect the data integrity at the various plant. Because our Ignition gateway set on a VM on the corporate network. So we use Ignition Edge and we use OnLogic, which already had Edge built in it. And so we installed it there in different size of plant. The plant's pretty big, and it funnel all of the devices to Edge. And I think it was like 10 devices per Edge gateway. And that just funnel it through their firewalls up into the corporate plant. Everything's configured, it's read only. The only interactions really with the... The operators will enter downtime, but at the same time, we had a lot of trigger tags so that we could automatically generate downtime events as well in the background and categorize those. So...

38:15
Samantha Willins: Yeah. So everything we added is in orange. The blue was already existing on their previous system.

38:22
Anh-Tuan Tran: Yeah.

38:23
Audience Member 4: Yep. Thank you.

38:24
Samantha Willins: Yep.

38:27
Audience Member 5: Yep. So you did not decide to use the Sepasoft modules, right?

38:31
Anh-Tuan Tran: Right. It was out of their budget and at the same time the way that they wanted to do this was very, to them, very straightforward, they said. So when we were pitching it to them, they weren't really interested in it and they saw a solution that we've developed in the past that looked good, and they're like, that's exactly what we want. And ours is very similar. But then when we got to the design phase, we found a lot of differences, but the implementation is very templatized in the structure that allows you to really pull in any sort of reason tree that you want in the background, and then we will dynamically build out how many drill downs they need based on the plant model that they have. So, but yeah, this is, this is like, you can definitely build your own MES within Ignition if you wanted to, so that you didn't have to manage another module.

39:25
Samantha Willins: Yep.

39:26
Audience Member 6: What's the format of your guys' architecture review board? Like how does an engineer engage that board?

39:33
Anh-Tuan Tran: So, it's cross platform. So we don't actually, we bring in folks that have Digital Transformation initiatives. So it depends on the types of project that you're looking at. So like, if you have a question that is related to the OT/IT conversions, like we call it business IT. There's a different group there that have the networking expertise to kind of coach you through. In this instance, it was the MES group with Digital Transformation based projects. The challenge that we ran into was that we had one machine that could not give us counts. So, what we were doing was we were looking at the case packer to calculate the count, but the case packer needs to reset the count at a very specific interval. And the whole calculation became really complicated. So, we were just asking for some ideas or some options of how to implement this without having to spend a great deal of time or having to add in additional sensors. And so, they came up with a really good idea. I mean, you were part of that group, right?

40:32
Drew Daff: Yeah.

40:33
Anh-Tuan Tran: So, they came with a really good idea on how to implement it. We presented to the customer, they were satisfied with it, and then when we roll it out, it just worked. And they kind of gave a count that was very close to how they were already calculating it anyways in their spreadsheets. And after a couple weeks they started trusting that count and then they just shifted to using that count for the quality metrics.

40:56
Audience Member 7: Just curious if you could share how big the team was. And I may have missed the first couple of slides, and if possible, what the total duration of the engagement with the client was.

41:06
Anh-Tuan Tran: Sure. So the sales cycle is very long. On these projects, I don't know why it just takes a long time to get a PO. The team was very straightforward. We had a project manager who was responsible on the RoviSys side. We have a lead developer and then we had two software developers working underneath the lead. And so, the makeup was four people on the team. On their side, they had a champion, like an overall champion that was overseeing this solution for their global organization. And then each site had their own champion that was overseeing the production data and how to calculate the OEE and how to transform some of these manual-based process into more digitized process. And then each site had one controls person, at the Jonesboro site, we had a really good controls guy. He knew PLCs, he understood and gave us a lot of tags. At the Mooresville site, that wasn't the case.

42:00
Anh-Tuan Tran: A lot of the equipment we found out was so old we couldn't get the tags. And so we had to come up with a new strategy or approach on how to calculate or pull in that tag. And there were a lot of edge cases where, hey, if you can't give me this data, just enter it here and I'll capture that timestamp and I'll use that for now, until you can add a sensor and then we can just replace it with the tag. So, we had a lot of stop gaps like that where we just create, we created manual-based entry system to allow them to fake the tag and use that timestamp so that we can actually continue to build the OEE solution.

42:37
Audience Member 8: Hi. So you talked a lot about OEE as your MES deliverable. Do you have any other MES aspects that you gave for the customer or is it more driven by what the customer wanted?

42:47
Anh-Tuan Tran: So, scheduling was a huge one that was just inherently pulled in. We needed a way to understand the shifts and how to split up that data so that when they are generating the report, they can look at what shifts, what the OEE for that shift was. Or they had specific teams that they wanted to categorize, like there's a team A or team B to see who was performing better. So, what we did was we just treat it as master data, right? So if they gave us a spreadsheet and we understood the columns and the rows, then we can build it in as master data and then we can contextualize that on the reporting side. So, once they gave us the schedule and we built all that in, we can tie it to the OEE data that we were capturing, so that they can see exactly who was working on what and at what time and what was their efficiency. And so, what team was doing better and then what value stream was operating better than another. And so they had a couple of value streams that had very similar equipment, but some produced the same SKU, but they had older equipment and they can see real quickly what was the efficiency between one versus the other. And then we kind of broke it down so that they can see what piece of equipment was running really bad and then they can like upgrade it, so...

44:00
Samantha Willins: We typically do it based on customer wants and needs or things that we identify will benefit them in the long run. Like, "Hey, you didn't mention you want this, but this is something that might benefit you in the MES space."

44:14
Audience Member 8: Thank you...

44:15
Jesse Oehler: Yeah. I think that that wraps up the time we have available. I believe you three will hang around for a little while?

44:20
Anh-Tuan Tran: Yeah. We're hanging around. If you guys have questions, just come through, we're happy to share.

44:23
Jesse Oehler: Thank you all for joining...

44:24
Samantha Willins: Thank you guys for coming.

44:25
Jesse Oehler: These presenters.