Deeper Dive into 8.3: New Features

Speakers

Matthew Raybourn

Sales Engineering Manager

Inductive Automation

Reese Tyson

Sales Engineer III

Inductive Automation

We have even more exciting Ignition 8.3 features to show you! Join us in the second of two sessions as we continue to share what’s new with 8.3. This time, we’re looking at some project-level resources and other features available through the designer, including new Perspective features, changes to the Tag Historian Module, and the brand-new Event Streams resource.

Transcript:

00:13
Reese Tyson: Hi, welcome to the second session of 8.3 Deep Dives. Honestly, we're super thrilled to be sharing some of the newest features of Ignition that will make... Honestly, this is going to be the best version yet. So, we're super excited about all these features. As you heard from some of these previous presentations, obviously there's a lot of great things coming out. And so, we're just excited to see how these features and dive into these features and see how they can really expand the platform, make them super dynamic and scalable, and honestly, the most flexible platform yet. I'm Reese Tyson. I'm a Sales Engineer here at Inductive Automation.

00:52
Matthew Abran: And I'm Matthew Abran, Sales Engineering Manager. And you might be wondering, what the heck is this giant chameleon for? What does it have to do with Ignition? And if you weren't here in the previous session, if you didn't have the right context, you know that's a valid question. So, yeah. We're doing a deep dive, essentially, into all the features of 8.3. In the previous session, we talked a lot about the platform features and some various things that Carl and Drew went over. So, now we're going to talk about all these next incredible features that Ignition, it's gonna take Ignition to the next level. But so we're not done yet. Let's go ahead. Let's bring out that whole basket of features.

01:40
Reese: Yeah, Paul. Paul doing the heavy lifting over here. All these features in here. All right. Well, so let's get into it. So, right, we're diving into some of these features. So, the first feature that we have here is actually going to be associated with these crayons. So, I am a father of a two-year-old. I'm very familiar with crayons, right? My daughter, she makes some beautiful drawings. Whether the drawing stays on the piece of paper or whether it gets onto the wall sometimes is another question. But these crayons represent the first feature that we'll be talking about in this session, and that is drawing tools. So, drawing tools, of course, allows you to create incredible visuals inside of Ignition. It's a first class, fully embedded SVG editor for Perspective inside the designer. For anyone who's been paying attention, this has certainly been a long time coming, right? So, this is the most requested feature on the ideas page, as you can see. You all asked, right, our community, we listened, and 8.3 we delivered. And it turns out that building a fully featured SVG editor is not really an easy task.

03:04
Reese: There's only a handful of folks that have actually done this out in the wild. And then, marrying the fact that we also wanted to make this really seamless to integrate with the designer and this SVG editor. And so, you can imagine there might be a little bit larger task than what you might expect. But, as you can see here, Drawing Tools is a way that you can create shapes, lines, merge objects, specify design details, and so much more. In fact, for those of you that have used Inkscape, I know a lot of folks have been creating visuals in Inkscape. For those of you that have been using Inkscape, this will look familiar to you, right? It's a similar experience. You'll be pleasantly surprised because we actually used Inkscape as kind of our inspiration for this new feature. So, let's dive into the details of what this really actually looks like inside the designer, right? As you can see on the right-hand side here, we have a brand new Perspective component. This is called the Drawing component. And so, you drag that onto the screen, just like you typically do with Perspective components, and it gives you a brand new, a blank SVG that you can, of course, build from scratch, anything that you want, right?

04:28
Reese: But you also might be asking, "Well, Reese, I have a whole host of other SVGs that maybe are existing or that I want to bring in a motor faceplate" or those types of things, and you might be asking, "Well, can I use those as well?" And the answer is, of course, yes. We can import SVGs into the designer as well, and then it's as simple as going to that SVG and right-clicking on it and converting that to a drawing. And so, this example here actually shows a motor, and so any SVG-based components inside a Perspective also have this capability, right? So, whether it's an SVG that you bring in and import or if it's an existing SVG component, we can convert both of those to a drawing component. Once it's a Drawing component, then you can just simply right-click on that component here and edit the drawing.

04:28
Reese: It's that simple. It's that easy to access the drawing editor there. So, here's a quick snapshot of some of the different functionality within the drawing editor. So, just to kind give you a lay of the land here, across the top you have, of course, your undo, your redo, your merge, your bring to front, all those common things you'd expect out of an SVG editor. On the left-hand side, we have the pointer tool, the line. You can create different types of shapes, triangles, rectangles, those types of things, text boxes, and then on the right-hand side, right, you have the ability to tweak these components and change how they look, whether that's the width or the height, the color, the stroke, all these types of things.

06:11
Reese: So, you can really dial the components down into exactly what you want to craft and how exactly you want that to look. One of my favorite features, and this was highlighted briefly in the tech keynote there, but one of my favorite features here is actually the bindings and how they interact between the designer and the drawing editor. So, you can see here, I just created a basic binding inside of the drawing editor, and then as you click through the elements of this SVG, it's showing up there inside of the designer, and this actually works both ways, right? If you make a binding on or inside the designer, it's going to be copied over and available for visual in the drawing editor. And so, this kind of goes back to we wanted to make this experience as seamless as possible. But really, that's just one of the features, right? There are so many different features that are available. We definitely don't have time to go into all of them, but some honorable mentions here: rename elements, browsing the elements, bringing icons and components from front to back, and so forth using the gradient tool, right? A lot of things that you would expect out of an SVG editor. And then just one more example here of things that are capable within the platform, right?

07:32
Reese: I was able to spin an image of this, of the Drawing Tools up here, and I wanted to just create a windmill, see how it worked and how easy it was. It took me about just a few minutes here to create a windmill that looks like this inside the Drawing Tools and just kind of duplicate and replicate that across your project. So, there you have it, folks. Drawing Tools, it really gives you a way to make custom visuals all within Ignition inside the designer. All right. Well, let's see what we have next in here.

08:18
Matthew: Yeah, let's stick with the Perspective thing. Okay. How about that? Yeah, that triplicate paper.

08:22
Reese: Triplicate paper.

08:24
Matthew: What is this for? What could this form be used for here? So, of course, we're talking about the Perspective Form component. So, yeah. So, for anyone that saw in the keynote, you saw a little bit of that there, and we're gonna do a deep dive into what is the Form component. Before we do that, let's kind of backtrack. Can't we already do that in Perspective? Well, of course, as mentioned, we certainly can. You can build forms. You can build really nice forms. They can be very complex.

08:52
Matthew: They can handle validation. You can submit data to all kinds of different processes. But the way they're built today is it's a little bit more cumbersome. You have to manually place all of your elements. Usually they're nested inside multiple different containers. Your views component tree looks a little bit like this. And then if you need to go back through and maybe make some changes, that could end up possibly breaking some paths, which is important when you start looking at how you have to deal with validation and submitting this data to whatever end point that you're looking for. So, of course, this very simple example doesn't even cover 80% of what you might typically do here. But you can imagine if any one of those paths changed, you'd have to go back and redo that, kind of rethink that. So this new component is really built specifically for forms of codes. That is the Form component. But it has all of those concepts in mind. So instead of specifying one individual element as part of the normal component tree, you actually specify just rows and columns and different widgets from a list of standard input fields. And then it also includes, just by design, it's used as CSS Flex. So that means it's already responsive. So you don't have to go back through and figure out that styling.

10:18
Matthew: It just does it for you. But you can also go through, of course, and customize it. You can do customized styling. So let's go ahead and take a quick look at the basics here. So we drag out a Form component just like that. You can see in the component tree on the left, it is just that one component. So it simplifies that part. It's also similar to maybe dragging out like a Menu Tree component. You drag it out, and then you can configure the items in it as a property in Perspective's property tree over there on the right side. So new fields are just simply added in different rows, different columns, and you select which widget that you want for that. It also works, of course, with standard styles, as I mentioned, and it uses a standard GUI for that. So you can go in here and you can see we can change the style pretty easily, just as you would expect, but we can even apply it, say, to the entire form pretty easily. So overall, it definitely simplifies the design aspect, but not only does it simplify that visual design part, it also simplifies the functional side.

11:19
Matthew: So validation support is included and has many different field types just automatically included that help with that. It also can populate fields and even hide or disable fields automatically just based on a little bit of configuration, which we'll go through as well. So you can see here, here's an email. It is incorrectly formatted. We can immediately see what is wrong with it. As an end user using this form, I can clearly see that something is typed in wrong, and we didn't have to create this style and create this feedback. At the same time, you can see that submit action is already disabled. So it's very clear, and that was just all part of the Form component. So to configure that, it's really just a set of standard properties, like I was saying. It gives you the control but keeps it simple. So we can go and just enable some sort of maybe pattern or a range. It can't be above a certain number. It can't be below a certain number. It all depends on the field type that you're using, the input type. But again, it is all just configuration, and so it is rather simple. So let's take another look at some more of the example fields here. This isn't even all of them, but it has all the expected ones that you would think of.

12:43
Matthew: So text, email, URL, phone numbers, passwords, telephone, radio buttons, check boxes, everything that you would typically throw into a form, and it can certainly do even more than what's listed here. And as just a quick note, I was playing around with this form to create this demo, and I had originally created it as one single column, and I decided it doesn't fit quite right. So all I had to do was right-click, copy a section of the property tree, paste it in a new location, and now I have the two columns. So I didn't have to redo anything, change any of the validation. It's all ID-based, so it worked out pretty well. And here's an example of showing that that same Form component, that Form component that I configured, is just automatically responsive. If I shrink down the screen, you can just see that the fields shrink down, but also radio buttons, check boxes, everything wraps. So I didn't have to do anything extra, anything special. It just handled it right away for me. Another cool feature of this that makes it really powerful is that the data can be populated ahead of time through the properties.

13:49
Matthew: You can see a data property on the right side; various ID-based kind of input components are listed there. So those actually go all the way back into the property tree that reference some sort of field, some input field that you have specified, and it's all ID-based, so even if you move things around, it will show up in the right place. So I can go in, play around with it in the preview mode, make sure it's set up the way I want by default, and then that's it. So it doesn't matter if you move things around. Data is just collected in one place, which makes it a lot easier to work with. The other feature I was talking about earlier, about the disabling and hiding different properties here, that's, again, just configuration. So you can see all I have to do is specify that. When do I want this field to be enabled? Well, when a certain other field is in a certain state. So if this toggle is off, then I want these disabled. So the checkbox and radio buttons are disabled. If I turn that toggle back on, then it's all enabled again. And a quick thing to point out here as well is a lot of this is just all clients, all browser side.

15:01
Matthew: So that's why it works well overall is because it's not firing a lot of different bindings that you have to control. It's all browser-side. It helps remove kind of that isolation of the individual components that you would normally have to configure. So it all goes into one place, which does make it easier to handle that submission action, which you can certainly do through standard actions like you'd expect. So if you originally were building forms and you put a submit button on there or a cancel button and you do an action event for that, you can still do that. You have an on submit, an on cancel as well. And so you can just take that data from one place and be able to write your script around that. If you have to make changes later, you don't have to worry about differences in locations and paths. But besides that, there's another option for the Form component. So you can actually specify that you want it sent through the submission event system. So there's a process flow that will go through. It will queue up that event and actually send it. And if you are offline temporarily, it'll queue up. Once you're back online, it can send it.

16:17
Matthew: And there's even more to that as well. And all this really does is end up in a session event handler like you would expect. And it provides the parameters that you would need to get the context around what happened here. And a lot of this is very important for the offline mode. So the form component certainly works with Perspective's new offline mode. So let's see if we have something else here in the basket.

16:37
Reese: Got another item? All right, well, looks like we got a mangled Ethernet cable that we've scrounged around from the IT department.

16:48
Matthew: Yeah, certainly. So, yeah, with the offline mode here, sometimes you're not always hardwired. You might be wireless. You might even be in a remote location. So what happens if you're in a situation where you need to open up your Perspective app to input some data and you have no connection at all? Well, that's where the offline mode comes in. So it's a new mode for Perspective projects. You can make a project actually launchable inside or in this offline mode. So not many people know this, but certain features of Perspective will still continue to work while you're offline if you already have the app up and running. But if you did not have that, then you can't get back into Perspective.

17:35
Matthew: So you can imagine some different scenarios where this would be quite useful. And so it actually just downloads the resources to your device, keeps them up to date, and that's how the form component also still works. It works because it works off of a lot of browser side and through those resources. So that means if you create certain views in your project and you enable it for offline mode, you can still go through, and just like this example here, be able to access those pages. So maybe you need to pull up a diagram or something that you're looking at, and you don't have an Internet connection or a connection to the gateway at least, and you can still do that. Maybe pull up a static line diagram for a piece of equipment that you're looking at. And so it's pretty useful for doing this kind of somewhat offline remote data entry, those kinds of scenarios.

18:32
Matthew: It does require the Perspective mobile app, so it works on Android and iOS. It's an opt-in type of model for a project, so not all projects are automatically exposed for this offline mode. You can, as the developer, decide what project goes into offline mode. And then users on their side, they get an option to go and download it, and then it will download it and keep it in sync, as long as they stay connected. And so there's an indicator provided to provide feedback there. So just like what we're showing here, I'd go in, find the project I want, click Make Available Offline, and then I'm good to go. So what we can look at here is kind of what's the use case? What's the user story around this? And I kind of alluded to it earlier. If you're driving out some remote location, maybe there's no network at all, no cellular, nothing, and you still need to be able to take advantage of Perspective and go through that Form component, especially, input your data, and then just keep going. You're not slowed down. You don't have to make notes in paper and then come back to the gateway and do it later. It's all done, and it's all right there. And so it makes it far more easier. So, yeah, I think, Reese, it's time to bring out the next item.

19:56
Reese: The next item? Well, honestly, I've been looking at this, and I'm not sure what this is or how it made it in here, but how this is... Before your applause happens, this is actually not a part of 8.3. It was completely a joke. So, we're just getting ahead of it, right? We knew you were gonna ask, so we're just, like, addressing it here in the keynote, all right? The next feature is actually an atomic model. So this is atomic model, of course, is represented by a nucleus and electrons and protons and how they revolve around one central place, right? How they all interact with each other. And so this next feature actually is Event Streams. So Event Streams gives us a way to receive events, filter them, and transform them out of the platform. So it's a brand-new module in 8.3 that allows you to do this functionality. So you might be saying, "Well, Ignition is already a place where I manage all my data. It's a central place to manage the data," right? So, yeah, that's absolutely correct, right? In fact, our first module, our first product, Factory SQL, did exactly that. And used Ignition as the hub to push that data to the database.

21:25
Reese: And so over the years, right, that has grown to include the different items that you see up here. Modbus, Omron, these different devices, DMP3, polling data from a SQL database, bringing in data from web services via Git requests, right, MQTT, alarm notifications, tag event chain scripts. There's a lot that goes on inside of the platform. And so it's become a little bit tricky, right? Especially if you're brand-new to a project, you're getting your feet wet. It's kind of hard to tell where these events originate and how they flow through the Ignition gateway. And so that's the first thing that Event Stream solves is a centralized event management within Ignition. The second thing is getting event data via subscription and not polling. You might look at these devices or at these different ways we get data into the platform and see that we're polling data out of the devices, bringing them into Ignition via tags. We're polling the SQL databases via SQL queries, web services with Git requests, right? But it's all these different polling ways of bringing that data into Ignition. And it hasn't really been until OPC UA and MQTT protocols that we start to get into subscribing to that. For example, with MQTT, you're publishing to a broker, and then we're subscribing to that to bring the data into Ignition.

22:56
Reese: And so that's great, but that was always external to the platform. Event Streams is a way that we can do this internal in the platform as well. And the third thing here is gaining flexibility in how the data is actually used, right? We know that folks out here are using enterprise message busses like Azure Service Bus and Amazon SNS to subscribe to data around their whole enterprise. And we know if we can just get the data up to those layers, then they can do the IT folks, and cloud folks can really do whatever they want with that in a familiar environment. And so just zooming into Event Streams a little bit more here, right, there really are four main parts to Event Streams. So the first is the sources, right, where the events originate. Second would be filtering, where we can filter the events that are processed through this event stream. And then the transform stage is where we can augment that data and provide more context. And then the handler would be where we stream that data out of the platform. So an 8.3.0, the initial release of 8.3, these are the three sources that will be available.

24:26
Reese: Event Listener, which is essentially a way that we can create a script and send events to this Event Stream all around the whole gateway, or around the designer and in the gateway. HTTP, which is a way that we can basically create this event stream as an endpoint that other systems can hit, and then we can process that information. And then finally, Kafka. And I'm gonna get a little bit more into Kafka for those of you that aren't familiar. Essentially, it's a large-scale enterprise messaging kind of platform that allows you to publish and subscribe to data. So that's the sources. So the second stage is the filter, where we can, of course, like I mentioned, filter the events that are coming through this Event Stream. So take this example. So if you have, say, five lines in your manufacturing environment, and you maybe only want to receive events for line one, right? I don't really care about the other lines. I only receive events for that line. Filtering is a way that you can do that. Once that line one is the only one receiving that event, then we can also transform that event flowing through this via live tag data or querying the database, whatever you wanna do within the designer, right?

25:33
Reese: And then finally, the handlers. So the handlers are going to be a way that we can stream data out of the platform, like I mentioned. And upon 8.3.0 release, these are the ones that will be available. HTTP, which of course allows us to post information to another endpoint. The logger, which allows you to log data or the event to the gateway logs. Database, for those of you that have used SQL Bridge at all, right? You know that you can map data to certain columns in the database. Very similar concept here that allows you to stream or receive an event via the source and stream that into a database. And we have scripting, gateway events, gateway messages, and then, of course, Kafka. So those are some of the handlers and sources that will be available upon release here. But there's a whole host of other sources and handlers that we do have planned for this, right? As you can see, there's a whole host of tags: MQTT, Ignition Events. These are just a handful of them, right? But really, what I wanna drive home on this slide is that there are limitless possibilities.

27:03
Reese: We're viewing this as a framework for you all to manage your events inside of one umbrella, right, and then utilize this platform to extend more and more handlers onto your Event Stream. So say you already have an existing event stream, you just plug another handler into that, right, when and if that comes available. And so you can see it's extensible. We can mix and match these sources and which ones come in, which ones go out of the platform. It really allows you to integrate with so many different systems. So, that's Event Streams with a lot of words, right? But you all wanna deep dive into the designer, so let's give the people what they want here. The first thing that I wanted to show in the designer is really just what this looks like when you open it up. As you can see on the left-hand side, there's the event streams. I'll call it a workspace, right? And creating a new event stream is as simple as right-clicking, or there's a little GUI as well. You might have seen that earlier in the tech keynote. But it's very similar with how you create new other items inside of the designer.

28:10
Reese: Then we're met with the source, right? This is where the event originates. So in this specific example, I'm subscribing to a Kafka topic, right? Bringing that event in. Then we flow into the encoder stage. This didn't really make the four main stages I was talking about earlier, because it's a smaller one, right? It's just a way that we can transform the data type. So say you have a JSON or a string; you can convert that into maybe a different data type very easily within this stage. The filter stage here, again, is where we can filter that data that's being processed through this Event Stream. And so, again, going back to that line example, right? Line one or two, or three, you can specify that here, just using the string language that you're already used to within the designer. Then we go to transform, right? Where we can transform this data, as I mentioned, whether that's reading a live tag value or polling data from the database, whatever you want to do with that, you can augment that data before then sending it out to the other handlers there. Once the transform stage is done, then we're going back into yet another encoder stage.

29:24
Reese: So, again, you can encode that data type into something else, and then we get to the buffer stage. So the buffer stage is really a very unique feature to Event Streams. There's really no way natively to buffer data unless you kind of create a custom application yourself. And so I'll give you an example of what I mean. Say you have a thousand tags, right? Each of those tags maybe has an event change script on them; maybe you are doing something on that change; you're gathering other data, logging that to the database. Each of those inserts to the database is going to be a separate insert statement. So you can see, as you scale, that can become not very performant. This buffer stage kind of allows you to batch all of those together, and it's really controlled by two different properties here. There's a couple other ones here, but the two primary ones are the debounce and the max time.

30:27
Reese: And so, imagine you have event streams coming in. We're going to continue to receive those until we hit the debounce time. And so what the Debounce time is, essentially, it's a rolling window. So every time we see a new event, it resets that rolling window. Receive event, reset. Until we receive all of these events, and then if we hit the Max Time, then we wholesale batch that into the database with one single insert statement. So you can see, this is really great for handling spikes of data, and you can tweak these properties exactly how you want them to be tweaked. Finally, the last kind of portion of the Event Streams here, there's a couple auxiliary UIs I'm about to show too, but the last kind of portion of the event streams kind of flow here is the handlers. So I just have a couple examples, but as you can see, there's a little plus icon there on the right-hand side where you can add more handlers.

32:06
Reese: So you've created this whole stream, and now I can send it out to ten different applications if you want. But just an example here with the logger: you can send the event.data, which is how this event is handled within each of these stages. This is just a JSON structure, so you could absolutely do a square brackets, right, key to grab the value out of that payload. So you could log the whole payload; you could log individual parts of it, up to you. That's a logger, and then the database, again, like I mentioned, similar to the transaction groups, right, where you can see you're mapping essentially the value expression.

32:09
Reese: Event.data and curly brackets item count, or square brackets item count, to column name one, or column one. So that would be how you can kind of interact with the handlers there. Now, like I mentioned, there are a couple different auxiliary kind of UI spaces here. This one's really nice to be able to handle errors gracefully. So this thing can handle thousands of events in subseconds. So if there was an issue that happened to one of those last night, like, how do I get to be able to find that information and debug that? So error handlers will capture any errors that happen within the handlers, yeah, any errors that happen within the handlers, and you could then, of course, log that to the database; you could send out an alarm; you can do whatever you want with that. The status page is going to be exactly what you imagine a status page has. Status is on the different stages, right? Events received over the lifetime of this, actually since the last time that we saved this event stream. Max times, min times, averages, average times of how these stages have executed, and so forth.

33:28
Reese: And then finally here, the test mode. And this is actually my favorite feature because, coming from a development background, this gives you a way to actually test this whole event stream without actually having to connect to a source to begin with. Okay. And so what happens here is you can see in that Test Controls there on the right-hand side, you have can specify JSON object and hit run all at the bottom, and you have the test results there across the bottom going through it's a good, through the source, through the encoder, the filter, et cetera. And if any errors were to happen, they would show up here. So that is kind of Event Streams. It's a great way where we can manage all of our events within the designer, and it's one central place and be able to kind of have a cohesive spot to put those.

34:22
Reese: So I mentioned Kafka earlier. And so Kafka is actually a brand new service connector in 8.3 that we'll be releasing. Kafka is an enterprise event streaming platform that's really, it's used by thousands of companies for high-performance data pipelines, streaming analytics, data integrations with IT tools and mission critical applications, really so much more. But by connecting to Kafka, it really opens up a huge ecosystem of tools that have Kafka connectors already. It's a very common enterprise streaming platform. And so we know that, again, if we can just get that data to that level, there's so many other tools that folks can utilize to do what they need to do. So what does this look like in the designer, right? This is what the handlers and the sources, which we've already discussed, looks like. So within the Kafka source, there's actually two different ways that we can subscribe to a topic.

35:31
Reese: And so that would be done by using either a consumer group or a partition. As you can see, there's a couple other settings that you can specify. The partition specifically, the max records that you're pulling at a time from Kafka, and so forth. And then on the handler side, right, on the handler side, you certainly have the ability to publish to a topic as well. And so that's Kafka. That's Kafka as well as Event Stream. So, yeah. I think we're ready to look at another feature there.

36:04
Matthew: Yeah. Yeah. Let's take a look at the last feature we're gonna be talking about today. So, yeah. Is it in there, Reese? Or did you leave it off to the side, maybe?

36:14
Reese: You know what? I do not see anything in here, and that's because I was playing with the next feature here.

36:23
Reese: What do you got there, Reese?

36:32
Reese: We got a super fast car. This thing's got so much power in it. It's got frickin' scissor doors. It's got everything, man. Honestly, as a kid, I wish I had this. Kids are so spoiled these days.

36:48
Matthew: It's a powerful machine. There you go, Reese. All right. So, yeah. So, with this powerful machine, we've got the Power Historian. So, what is the Power Historian? So, we've kind of talked about it earlier, but it's essentially a new time series database-backed historian for Ignition. So, it uses QuestDB as the backend, and it is embedded in Ignition. So, that means no separate installs required. You just go create a new history provider, and it's taken care of for you. It's also built on the new historian API, so it has a lot of those features that were mentioned previously in the last session about native aggregation and all of that.

36:54
Matthew: There's a lot of features, of course, that we'll continue to add as well in the future, but certainly it has a lot already going for it. So, taking a quick look here, it certainly supports partitioning and pruning as you would expect, but it's all handled by the database itself. It uses write-ahead logging, which helps with performance and robustness. It has a native aggregation capability, so the Ignition's no longer having to do that. It's all pushed onto the database itself, which is far more efficient. And as mentioned previously, renaming your paths does not break history, which is pretty critical for anyone that's gone through and had to rename things. It can be a little bit painful, but the new API and the Power Historian certainly address that.

38:24
Matthew: It also supports automatic archiving, where you can detach a partition and move it to a new storage location, which is pretty key for larger setups there. You need to think about your long term; what is your data storage going to look like, and how do you take care of that? But you still want to be able to keep that data somewhere. So archiving it certainly helps out there. But enough about all that. Let's talk about performance. So I went through, and I was able to do some preliminary testing with the current system. And so I threw together something that was just throwing only 4,000 data points per second continuously into it. And of course it was handling it just fine, and I was measuring query time at the same time while running a Perspective session looking at the tag history data, so all in one system here, and then comparing it to what the current internal historian option is, which is SQLite-based, as well as the dedicated Postgres database just for comparison.

39:20
Matthew: So first question, of course, is, was that heavy on the resources? If you have an internal historian running alongside Ignition, does it take up too much? Is there any kind of issue there? And no, the system actually in this test was using just 4% CPU and 700 megabytes of memory. Is actually really low to be able to store millions upon millions of records in a partition like that. So it was actually performing extremely well and still being able to keep up with everything. The next question you might be thinking of is, does it use a ton of extra storage? If we're storing all this data next to Ignition, what are we gonna have to do with that? And no, it doesn't store a lot of extra data. It's historical data, of course, but it's comparable to all of the other options.

40:13
Matthew: So you just have to again, plan ahead for how much storage you need, and it will take care of it for you. And you have now the options to do the automatic detach and migrate the data to an archive location. So it actually helps out there. So you can offload that older data, for example, to something maybe a little bit more slower, discs, or something that's just more efficient for that type of data. But the real question is how performant is it? You know, was it performing well while all of this was happening, even though it's all local to this one in process storing? And of course it was great. Yeah, it significantly outperforms, of course, the alternative, which is SQLite. If you use SQLite for anything more than even 100 data points a second, you'll probably be running into, you know, performance issues because it is not going to be able to keep up. The Power Historian certainly can.

41:07
Matthew: We're talking like milliseconds worth of query time here, querying a tag for raw data over say an hour's worth, so thousands of points, and it was able to just do that. It certainly is comparable to a dedicated database, but yeah, it significantly outperformed SQLite so much that SQLite in the end was taking two minutes just to query data and ultimately failed. It couldn't actually keep up. It had to stop inserting. So if you're looking for that local historian option, but something that will actually be able to keep up and perform, then Power Historian is certainly great with that. And we are constantly adding onto it. So even just this morning I received messages that we've added even more to it. So this is all preliminary, but one of the improvements we've made already, and this was actually asked, I believe, in the last session, is: can we connect to the Power Historian and get data out of it externally?

42:07
Matthew: And so they added the ability to connect to it using a standard Postgres connection, and so you can query the data right out of it. So yeah, you certainly can do that, and we will continually add more improvements like that and improve query times and all of that moving forward. So there's a lot that it does today already, and there's a lot more that it will be able to do. So there's questions about can it store things like engineering units, and the API certainly supports that, the metadata, but as well, Power Historian will be able to support that as well. Support, eventually storing UDTs, things like that, that you would really, really want. So yeah, overall, the Power Historian, it's plug and play; it's designed to be that way. It is performant, and we're gonna add even more functionality in the future, of course. So stay tuned for all that. So at this point, we can open up to any kind of questions for everyone. Mic runners should be running mics down. Yeah. It looks like we've got one at the top already about...

43:19
Audience Member 1: Yes. Cool. So if we take a look at this Power Historian, how can we relate to that, if I compare it with a normal SQL database? Is it capable of storing terabytes of data?

43:40
Matthew: Well, yeah. It's a time series historian. It can record gigabytes, terabytes of data over time, just like you would expect a time series database. It all works as a tag historian. So yeah, are you asking for anything else specific? I mean, the storage size is purely gonna be based off the size of the hardware that you're running, right?

44:03
Audience Member 1: Of course. Yeah.

44:03
Matthew: So that's certainly like the size of the historian.

44:06
Audience Member 1: Yeah. There are no limits, like 1 terabyte or whatever.

44:10
Matthew: Well, yeah. We're not gonna impose a limit. What would actually happen is your partition size would matter, and there's only so many rows you should put into your partition. I believe it's, you know, hundreds of millions. And so as long as you configure the partition size, then those are going to go into new files, and so you can keep storing and storing until you run out of space.

44:29
Audience Member 1: Cool. And you can also set it then that you will not store all that data on your install drive. Your C Drive, for instance?

44:37
Matthew: Yeah. So right now it stores into that local. If you've seen the collections there, it stores into local, but I believe we'll be able to store it elsewhere, but we'll have to get back to you on that one.

44:51
Audience Member 1: Alright, thanks.

44:54
Matthew: The worst case, I was able to do it myself by just mounting the directory to somewhere else, and then I didn't have to worry about it.

45:01
Audience Member 2: Hey. Is Ignition Edge gonna get Power Historian and receive all like full features?

45:08
Matthew: Yeah. So we're definitely testing that right now to make sure it can perform on what your typical edge device might be, like a small embedded PC, but that would certainly be possible. Yeah. And it outperforms its SQLite option, so it's great.

45:24
Audience Member 2: Thank you.

45:27
Audience Member 3: How does Power Historian handle redundancy? Does it synchronize while it's running?

45:33
Matthew: So it currently does not do that. That's definitely something we can evaluate, but yeah, it currently does not.

45:47
Audience Member 4: For the new Forms component, will users be able to submit photos when they submit a form, and for the validation support feature, will that work offline?

46:00
Matthew: Yeah. So the...

46:00
Audience Member 4: Yep. Sorry.

46:01
Matthew: Oh yeah, yeah. Yeah. So there's File Upload components; I believe there's more field options as well. But yeah, you could usually with the File Upload button; it allows you to snap a photo if you're looking for that with your device. But yeah, it would support that. And then the validation, all of that is browser-side, so it doesn't require the gateway to perform that action.

46:21
Audience Member 4: Nice. Is the offline function just for forms, or you mentioned it could work with static views?

46:29
Reese: Yeah. You just have to imagine when you design a project, bindings and scripts transforms. Those are on the gateway side, but the form component has certain features that work, you know, on the browser side, as mentioned. Other components, they're just resources, and so we would download those onto the device so you can navigate around; you can see the images text that you have, but you're not going to get live tag updates while you're offline. So you have to keep all of that in mind.

46:53
Audience Member 4: Understood. And would there be an indicator for whether the app is offline or online to let the user know that their changes are gonna be queued and then forwarded?

47:04
Matthew: Yeah. So with the offline mode specifically, you actually launch into it. It's not something where you would temporarily revert to offline and then it would auto-reconnect to online and back and forth like that. It is, you would go launch it in offline mode and you are in offline mode, or you go launch it online.

47:24
Audience Member 5: For the Kafka streams or for the Kafka integration, is there gonna be any support for consumer groups to like ensure that a certain Kafka message is only processed once by a consumer?

47:38
Reese: Sure. Yeah. So right now, the way that you do that is by the offset of the Kafka topic. There are definitely plans right now to for... Basically, what you're saying is an acknowledged message for that source, right? So right now that is not something, but it's definitely something that we are planning on putting into that.

47:54
Audience Member 5: Okay. Also, what about schema registry?

47:57
Reese: I'm sorry?

48:00
Audience Member 5: Schema registry?

48:01
Reese: Schema registry? I'm not really familiar with what you mean by that.

48:06
Audience Member 5: Okay. It's basically a framework for Kafka to define what a payload looks like. Like, basically, if you define a proto buff for a certain topic, it's a way of having a centralized store of saying, you know, what payload should be produced on a topic.

48:24
Reese: Okay. Yeah. I couldn't answer that one off the top of my head, but I could certainly connect up with me, and we can chat about that for sure. Do we have any up, up in the balcony there?

48:34
Matthew: There's one up there.

48:35
Reese: Yeah. Looks like one over there.

48:37
Audience Member 6: Yes. One more. So with the Forms component, does it also support adding additional views in there? Your own created views?

48:45
Matthew: So the Form component is a component inside of it. It doesn't have an embedded view field type. The problem with that is then we would have to figure out what your specific view would be doing, and so it would probably break the functionality of it.

49:01
Audience Member 6: Right. Makes sense.

49:02
Matthew: But you could always put a view alongside it and still... It's still a component.

49:08
Audience Member 6: Yeah.

49:14
Reese: Yeah, right here.

49:15
Audience Member 7: In a previous keynote session, it was listed that you can poll up to like 1.8 million record in one second or something. So how was the performance on a GUI side if we are pulling that kind of data on a history trend?

49:32
Matthew: So that I believe was ingestion rates. I think it was like 1.4 million a second. And so that is literally what the database could support on a performance system. Querying that back, I mean, 1.4 million data points, it's... That's a lot of data, but that's not what that graph was showing, but you can certainly query. I was querying millions of data points, and it was taking maybe a hundred millisecond, maybe 200 milliseconds, to query through all of that. So it is performant, but again, it is still preliminary, so we'll have more numbers, of course, later. All right.

50:07
Audience Member 7: Any change in a trending package?

50:14
Matthew: Any change in like trending overall? This is all standard historian. It all works with the same components, so there was no change with the trending there. You do get the native aggregates, but the functionality should be the same. The end result's the same.

50:25
Reese: We'll do one more question here. Yeah. Right over here.

50:33
Audience Member 8: Yeah. Alright. So talking about offline mode, when going online, when two or more upper people are, make changes to the same, let's say, talking about a database field, what about conflict resolution?

50:53
Matthew: So the...

50:53
Audience Member 8: Last in wins, or is there gonna be a mechanism to solve these conflicts? And on top of that, what if let's say that I'm talking about a database table and what about... What if the schema changes when the device was offline? How are those conflicts are gonna be resolved?

51:18
Matthew: Yeah. So what would happen in that case is you have your package, your data there that would get sent up, it ends up in your session handler, that special Form component submission event handler. And it's essentially gonna be a bunch of data that's gonna be provided to you. So that form component doesn't send you; I have, you know, this column of data. It sends you, here's my essentially JSON package. Your handler at that point, if the schema had changed, it would be up to you to fix your script. And at that point, the new data coming in later, as long as you can interpret it and understand it, if that part of the data didn't change, you could send off a query to insert that and that would be fine and it would be in the order that it is received. So if you insert and then insert or if you do an update and update, that last update would override it. So that would be part of your handler logic.

52:10
Reese: Alrighty. Well, thank you everybody for coming to the second session here. Really appreciate you coming out.