RSS .92| RSS 2.0| ATOM 0.3
  • Home
  • About ArchestrAnaut Blog
  • About Our Authors
  • Email Subscription
  •  

    Some Random Thoughts on Securing the Environment

    February 22nd, 2012

    Like many others I’ve been following the salacious developments around Cyber Security and our dear SCADA systems.  If you’re involved with any kind of SCADA chances are you have some kind of somewhat valuable target that an outside entity might be interested in either destroying or just messing with.  Maybe it’s nothing more than a building management system controlling the HVAC.  If nothing else they could crank the temperature way or up way down to make people really uncomfortable.  Yes, no harm no foul, but I wouldn’t want to explain that to my boss how someone hacked in a took control of the environment.

    So that got me to thinking.  Why, fundamentally, do most environments really suck at being secure.  Is it for lack of desire to be secure or lack of capabilities?  I’ve certainly been in a number of places where the operators “won” and the HMI either had no password or it was posted on a sticky note.  I guess you could say the sticky note method is somewhat “cyber secure”, but terrible nonetheless.  Chances are that password hasn’t been changed in 5 years so it’s really pretty worthless.  Say we get past the desire to be secure and we’ve handed out passwords to the operators and engineers who work on the system.  The next step is the capability of securing the environment.

    Read the rest of this entry »


    Who’s Behind? Who thinks they are Secure?

    February 6th, 2012

     

    I had a recent issue with a customer where their InSQL instance just stopped recording data to disk.  It was reading the data coming in just fine, it just wasn’t getting to disk.  I go through the basics but can’t quite get it to come back.  Place a call to tech support.  Turns out the customer is running InSQL 9.0 Patch 01.  My friendly tech support person quickly informed me that my customer was way out of support. 

    She said she’d be willing to help just a little bit but she couldn’t really do much until I was at least at Patch 02.  Realistically, she said, I needed to be on Version 10 to give me a fighting chance to get adequate support.  Well, this customer is in an FDA regulated environment so upgrade means months and $$$.

    Read the rest of this entry »


    VSphere Support …. It’s here

    January 24th, 2012

     

    Looks like the word is official, VSphere 5.0 is finally officially supported!  Check out the announcement

    https://wdn.wonderware.com/sites/WDN/Lists/Article/Article.aspx?List=10839c88%2D47d4%2D43fb%2D8699%2D1b9ce20313ca&ID=367

    (You’ll need a WDN login to access).  This also includes a link to a really big PDF detailing lots of considerations and detailed instructions on virtualizing your system.  I highly recommend reading it cover to cover.

    A brief summary from a recent communication I had lists the following features that are supported.

    VMOTION
    DRS ( Dynamic resource allocation)
    HA (High Availability)
    DR (Disaster Recovery)
    FT (Fault Tolerance)
    And Snapshots (Although we recommend not using this on production systems)

    Not totally sure about why snapshots aren’t supported.  One speculation is that a VSphere snapshot can stun the VM for a couple seconds.  There was an issue a while back with ESXi 4.1 that caused a 30 second freeze on NFS datastores.  I’ve seen a 2-3 second freeze on my low end ISCSI SAN. 

    Either way this is a really big deal on the support front. I know a pretty good list of customers now who went ahead and took the plunge a while back and have yet to experience an issue related to the fact that the system was virtualized.

    A little aside is that I’m going to work on a white paper that goes really deep into the considerations when choosing storage for your environment.  If you’ve ever seen one of my presentations or chatted with me about virtualization I will beat you about the head and shoulders about how critical it is to get your storage right.  If you screw up your servers that’s really easy, and relatively inexpensive, to fix.  If you screw up your storage you’re in for a long expensive process to get it fixed.  Keep an eye out for a white paper on WDN sometime in the future.

    - Andy


    A bump to the top and Scott Whitlock is a BadA**

    December 12th, 2011

    First off, a post to let you know we’re still here and alive.  I’ve been buried on a startup for the last few weeks on nights.  Inspiration is hard to find at 4:30 in the morning in a cave.

    Anyway, a couple things I wanted to get out for our reader’s consumption.

    First, if you aren’t reading Scott Whitlock’s blog over at ContactandCoil.com you are really missing out.  He spans the gamut from hard core PLC’s to deep dives in .Net all the way over to garden scale trains.  Anyway, he’s got a really neat idea (at least he wrote it up, don’t know if it’s his idea from scratch) on securing communications to your PLC networks.  The basic idea is that instead of having machines from outside the network actively connect to the PLC’s, do it in reverse.  Make the PLC actively connect to something on the other side of a one-way firewall.  Sure there are some limitations to the approach but as a start it’s a really neat idea.

    http://www.contactandcoil.com/automation/industrial-automation/safer-data-collection-from-a-plc/

    Second, got a lengthy comment from Roger Smith at Invensys on an older post that I thought had some great nuggets in it so I’m reposting it here for all to consume.

    I stumbled across it while Googling for something else and saw my friend Howard’s name on a post.  I just HAD to see what he was up to.  After reading Andy’s post, and the responses, I thought I’d chime in on a couple of the topics discussed.

    @Andy: I’m aware of the requirement for DCOM with A2 communications, but never would have thought to check to see if it had been disabled.  Thanks for posting this, I’ll try to remember it for future (re)use.  There’s a long line of people that would love to see DCOM replaced with something more firewall-friendly, like WCF, in a future release.

    @Dan: I’m curious if you working with Operations 4.0 or newer?  With that version Wonderware updated the MES Client API and middleware to support WCF, in part to get some relief from DCOM heartburn.

    @Howard:
    1) The new virtualization guide is included on the System Platform 2012 installation image, available on the WDN support website.  Most of the content is built around discussion and examples of Hyper-V.  This is likely because it’s a feature of Server 2008 R2 OS, rather than a 3rd party application, and perhaps due in part to Wonderware’s close relationship with Microsoft.
    2) The requirement to disable UAC for Vista and newer OS was introduced with App Server 3.0 and InTouch 10.0 in 2007.  It has been documented in the ReadMe.html file on the installation media for these products ever since.  Perhaps because adoption of Vista and Server 2008 OS was slow, it seems that many users didn’t discover this requirement until working with Windows 7 and Server 2008 R2 more recently.  Unfortunately, like the DCOM issue above, leaving UAC enabled results in a problem where the symptoms don’t necessarily point to the solution.
    3) It was great to see you at OpsManage in Nashville!

    -Roger

    That’s about all for now.  Hopefully once the startups die down David and I will be back in the saddle again.

    - Andy

     


    First Round of Goodies from Ops Manage

    November 17th, 2011

    I’ll apologize in advance for the slightly scattered nature of this post.   This is my brain dump of all the really cool stuff I saw and heard at Ops Manage this year.

    Before I begin, some of these are items that were publicly discussed in canned presentations while others I picked up in conversations with some of the powers that be.  Anything that wasn’t part of a public discussion I’ll mark with ** so don’t go asking around when feature XX might be released, you may get a denial the particular feature ever existed or has been discussed.  Also, the screenshots I’m including are from a beta release so if they change slightly on the production release don’t give me a hard time.

    1) Right out of the gate, support for VSPhere 5!  I talked with Rob Kambach for a while about this one.  They have completed a battery of tests and found no issues.  At this point they need to go through a documented/formal testing regiment before they officially announce support.  Look for this somewhere around Q1 of next year.  It also sounds like they are going to support a wide range of features such as HA, Fault Tolerance, Snapshots, etc.  They are actually publishing a 700+ page document on Virtualization and High Availability for System Platform.  Most of it is Hyper-V focused but there’s a lot of good information in it.  I’ve read through parts of it from the beta version and I definitely recommend it.  Also, Brent Humphreys and I were having a discussion a while back about how we’d configure an RMC between machines running in two different datacenters.  We speculated setting up a dedicated VLAN for RMC traffic “should” work.  Well, in this document they address the issue and confirm that VLAN’’s are supported for all node to node communications, including RMC traffic.

    2) Lots of support for new Server 2K8 R2 remote features.  Once of the coolest new features in 2K8 R2 is the concept of remote apps.  Think terminal services where the app is running on a remote server, but instead of immersing yourself in a complete remote desktop, you run the app from your local machine.  Just double click and icon and you think the app is running on your local machine.  What’s actually happening is that the app is running back on the server and it’s using something like RDP technology to serve up the graphical portion to your computer and interact with your clicks.  This is really really cool stuff.  Here’s the first link I could find on the Microsoft website about this technology.

    http://technet.microsoft.com/en-us/library/cc730673(WS.10).aspx

    3) Skelta/Workflow is now a first class citizen.  Once you install it, all of your objects will have a workflow tab.  How would I use this?  Say you want a supervisor to be notified every time a HH alarm with a priority < 100 goes off with your analog objects.  You can configure a workflow on your template that sends this notification and waits for the supervisor to acknowledge the alarm before the operator is allowed to acknowledge.  I’m expecting some really really big things from the new workflow engine.

    4) Tons of improvements around E-Signatures.  The biggest one is that you can split out the verifier function.  Before you had no good way to limit who could be a verifier.  That’s why we ended up writing our own prompting object that built in all of these features.  We’ve had secured and verified writes for a while now.

    image

    What we haven’t had is a good way to control who can verify writes.  That has changed with a new operational permission called Verify Writes.

    image

    The idea here is that you would setup one group such as operators for an area and they could do the standard operator things.  Then, you could setup another group for supervisors or foremen and they would have the Can Verify Writes permission.  Now an operator can change a value but they have to get a supervisor to verify it.  An even neater concept is the idea that someone from the quality group can have no privileges at all, except Verify Write.  So now when the operator attempts to say a batch is complete and ready for further processing, the quality person could be there with them and verify the answer, essentially authorizing the action.  The log entries have also been improved.  You know the two people who participated in the transaction

    image

    What I didn’t see in my release was the detail that it was a verified write.  I do remember, however, seeing this demoed at the conference and it looks like they’ve updated the Description column to include the fact that it was a verified write.

    Another cool thing they’ve done is that it will allow you to enter operator credentials for an operator that isn’t even logged on.  What’s neat about this is if the operator just needs to change something real quick they don’t have to actually log on.

    Supporting all this functionality is the ability to use smart cards.  Smart cards are akin to an access badge but the operator will place the card in some kind of reader on the HMI station.  Then all they have to do is enter a pin number in place of a password.  More secure and faster.. I love it.

    Finally, there are a couple of really cool features that are similar so I’ll talk about them together.  They have added script functions in the graphics called SignedWrite() and SignedAlarmAck().  The intent appears to be to allow the designer to give the operator an alternate way to enter/modify data.  Once they have entered/modified data the script calls a signedwrite to attempt to write the new value to the attribute.  What you can do with this, however, is to inject a pre-defined comment or pre-defined list of comments.  Imagine this scenario, an operator finds a cold storage chamber out of spec.  They go to adjust the set point.  When they adjust the set point a signedwrite is fired.  They are presented with a pre-defined list of comments they can select from.  They can’t just enter “Didn’t like current temperature so adjusted”.  They would only have comments like “Added Material to Load”,”Ambient Conditions out of Spec”, “Controller too Variable”, etc.  In regulated industries it is critical that that operators don’t get too crazy with their comments on alarms and data entry.  One wrong phrase in a comment could spin off weeks of work trying to explain it away, even if it is the truth.  I think this could be one of the most underrated new features. Wow!

    Here are a couple dummy calls to give you an idea how these are going to work.  See some neat things on the SignedAlarmAck that you like?

    SignedAlarmAck( Alarm_List, Signature_Reqd_for_Range, Min_Priority, Max_Priority, Default_Ack_Comment, Ack_Comment_Is_Editable, TitleBar_Caption, Message_Caption );

    SignedWrite( Attribute, Value, ReasonDescription, Comment_Is_Editable, Comment_Enforcement, Predefined_Comment_List );

     

    5) Buffered Data.  Where do I begin on this one.  Let me be the first to say I’m still a little confused.  According to the help files here is what they say buffered data is

    The buffered data feature enables efficient accumulation and propagation of VTQ (Value, Time, and Quality) data updates, without foldering and data loss, to data consumers such as objects, alarms, the Historian, and scripts from field devices that support buffering.

    Buffered data is defined as data captured and stored locally on a remote device for later transfer to a supervisory system for processing, analysis, and long-term storage. The Buffer property is input-only.

    Ok, that’s pretty clear.  Seems like this is built for RTU’s and the like where the remote unit might accumulate some data and forward it on with quality and timestamps.  Interesting.  Only problem is the demo I saw is 180 degrees from that.  The demo’s I saw were touting Buffered data as a way to collect data really really fast.  Imagine you have the same value from a PLC and the object is on a 1 second scan. Here is what an overlay of buffered and non-buffered data might look like.

    image

    Here is what I think MAY be going on.  The demo’s they are showing might be using buffering on the end device to put together an array of values and then forward these values on to IAS, making it appear faster.  However, when I chatted with Rob K. about this he indicated that what was going on was that the data collection was running as fast as it possibly could, “out of band” (my words not his).  Either way this looks like a really neat feature that could be very useful. 

    My thoughts on how it could be used?  Two areas.  First, imagine you have a piece of equipment that goes through different modes and in one particular mode it’s critical that you capture detailed information about what the machine looked like during that mode, say a pressure test.  If what I was told was true***, that you could turn buffering on and off at runtime, then you could flip this guy into high speed mode during the pressure test then turn it back off after the pressure test.  Another way I could see using this is for super critical data.  In FDA regulated industries losing data is a huge NO NO.  Only problem is that if we lose network connectivity to our PLC there is nothing we can do to recover from that.  The new Foxboro PAC has some neat new features (that may actually dovetail with this) whereby it will buffer history and alarm data locally until a network connection is re-established.  What about doing that with my Allen Bradley Control Logix?  Maybe it detects a lost heartbeat then goes into buffer mode, maybe capturing a value every minute or some reasonable time frame to save on space.  Once the connection is re-established my object hooks back up, sees there is data in the buffer, processes it, then moves on.  This can even work with alarms too.

    I think I’ve got a lot of reading to do on this one.  I suspect the first group of folks to really figure this out could have a serious leg up from a system resiliency standpoint.

     

    Ok, this installment has gone on long enough, back to struggling with my Silverlight App.

    Next week is Turkey week so I probably won’t put anything out then.  However, week after I promise another post on some new features, especially the new ShowGraphic() function.

    - Andy


    Back from Ops Manage

    November 14th, 2011

    Made it back safely from Nashville, although there were a few minutes at the taping of the CMA Christmas Show I thought we were about the throw down.. but that’s a different story for another day.  I’m planning on writing up a few posts on what I enjoyed about the show, where I think they could improve, and most importantly some interesting tidbits I picked up.  There are some pretty juicy items, if you’re into things like that.

    - Andy


    System Platform 2012 Released!

    November 4th, 2011

    In case you missed it.. I know I did, here’s a press release on the official launch of System Platform 2012

    http://iom.invensys.com/EN/Pages/IOM_NewsDetail.aspx?NewsID=393

    I have had access to the beta for a while and there are some really cool things in this release.  For fear of saying more than I should I’ll keep my trap shut until after the conference before I discuss any of the new features.

    A couple items I will comment on, since they are in the press release, are the undying infatuation with Hyper-V and some under the hood improvements. 

    Once again I think Invensys is showing their colors in terms of being squarely aligned with Microsoft.  I’m not saying this is a good or bad thing, just an observation.  Let’s just say in the doc pack with the Beta there is a massive document on deploying a more resilient system with Hyper-V.  I truly do wish they would take a more even handed approach and pay heed to the acknowledged 900 lb. gorilla in the space, VMWare.  Over the last 2 or 3 years while Hyper-V has caught up in a few areas, VMWare has zoomed ahead on countless fronts that Hyper-V hasn’t even thought about tackling.  I guess I just feel like they are doing a dis-service to the customer base by continuing to pound the drum for one hypervisor over another, especially when that hypervisor isn’t even close to best in class.  Ok, off my soapbox or I’ll tick off the wrong people again.

    As for the good stuff, from what I’ve seen and read there are some serious under the hood improvements that I think everyone is going to really like.  Think reliability.  Once an engine and objects are up and running IAS is pretty durn good.  However, when’s the last time you felt good about deploying thousands of objects or an entire platform?  Based on some serious changes these scenarios should be much much better.  There aren’t a ton of new features on the surface, at least from a pure Archestra perspective, but one in particular should really excite folks in the qualified industries (i.e. Life Sciences).  This is an issue I brought to their attention a couple years ago at Ops Manage.  Good to see that real feedback from real customers eventually makes it into the product.

    Ok, I’m starting to say too much now so I’ll let the proper Invensys people introduce the product at Ops Manage then I’ll do some write ups after that.

    - Andy


    Get Ya Boots on Maw, We going to Nashville!

    November 4th, 2011

    Only a few more days to Ops Manage.  I’ve been trolling through the program to figure out what sessions I’ll be attending and what vendors I want to try to meet with.

    I’m still planning on trying to hold an aaBeers get together.  Just a casual get together of fellow archestranauts to have a few beers and talk about whatever comes up.  What could be cooler than a bunch of geeks with nametags chatting about the merits of the MX protocol or beings IOPS bound on your checkpointing processes!!

    I’m not much of a twitter master but I plan to try tweeting throughout the conference.  You can follow me at @archestranaut

    http://twitter.com/#!/archestranaut

    When I figure out a place for aaBeers I’ll probably tweet it along with a blog post.

    Looking forward to seeing everyone in Nashvegas!

     

    - Andy


    Async scripts without using Async

    October 31st, 2011

     

    First off, yes we’re still around.  Got tied up in a rush job on a DeltaV Batch project that has taken me away for the last 3 weeks or so. 

    As promised in our last article we’re going to discuss how you can write a long running script without using asynchronous scripting.  To be totally fair this only works in the situation where your script is long running because you are iterating some kind of array.  If you need an async script because you have a really long database call or some long running file transaction there’s not much I can do for you.

    Before we get into the technique the first thing you need to understand is what an asynchronous script is.  When IAS executes its objects, on a particular engine all objects and scripts in those objects are executed one after another.  It’s pretty amazing that the software engines can get through so many pieces of code so fast.  Kudos to the software architects.  If you find yourself with scan overruns on an engine, the first thing to do if figure out if you have any objects hanging or hogging resources.  If everything checks out there you should create a new engine and move some objects over.  At this point you have now split your object list and allowed them to run in parallel.  A while back I saw some advice that said you shouldn’t have more engines than cores on your computer.  That seems to make sense if we’re trying to run all the engines in parallel as efficiently as possible.  By the way… getting back to the previous post… does this help you understand why it takes at least one scan to hookup to an attribute that is on another engine?  Back to what we were talking about, there’s not a lot of good information in the books on what async scripts are but here’s one line I found that helps a little bit

    Asynchronous scripting mode is a group of scripts running on the same, lower priority execution thread.

    There are two key points here to think about.  First, the scripts are running on a different thread.  If you know much about software development you know that in a single thread all actions are lined up and happen one after another.  In other words, if there is a slow poke in the group then everybody waits behind them to get done before they can execute, kinda like the old person in the grocery store who still insists on writing a check.  Extending the analogy a little bit imagine you’re at a Wal-Mart supercenter with 32 checkout lines.  Only problem is the manager only has 3 lines open.  Wow, did I just tie-in Wal-Mart to a discussion of asynchronous scripting in Archestra?  The second part of the analogy makes us think about the difference in speed of the checkout folks. We all know that all cashiers are not created equal.  Imagine you have one cashier that is totally focused on the customer in front of them, that customer is a high priority.  Chances are the line is going to move really fast  Let’s call this cashier Fay.  Now imagine another cashier who puts a low priority on the customer in front of them, say his name is Cletus.  They are answering their cell phone, chatting with the guy stocking the drinks, commenting on what the person behind you is wearing, in general they are being interrupted by higher priority tasks.  That process is going to move pretty slowly.  When you run your scripts in a normal fashion you get into Fay’s line.  While the line may be long she’s really efficient and gets everyone through in a quick manner.  When you run your scripts async you are in Cletus’s line.  You’ll get through, eventually, but you may be waiting around for a while.  What I have left out of this discussion though, is that Fay can only work for 15 minutes at a time and if you are standing in line when her 15 minutes is up, then you have to put everything back and start over again… this is like a scan overrun.  Enough of the Wal-Mart analogies, back to the highbrow stuff.

    So we have an array we’re trying to loop through but because of the length of the array and the nature of what we’re doing in the array, there is no way we can make it through the whole array in one scan.   How do you handle this.

    Well, the first thing to do is figure out how long you have to execute your script.  This is available to you via some parameters on your engine.  Here’s the expression we use to calculate what we call a Halt Time.

    Me._HaltTime = MyEngine.Scheduler.ScanTime + Me._ScanFraction * MyEngine.Scheduler.ScanPeriod / 1000;

    Basically what we’re doing is saying take the start time of the current scan cycle then add a portion of the scheduled scan time.  We will typically use something like 75%.  Say we start this scan at 13.2 seconds after the minute.  If we have a 1 second scan time and a 75% scan fraction we end up calculating a halt time of 13.95 seconds after the minute.  We’ll use this later in our loop.

    Next, we need to setup our tracking devices.  On the first time through our script we reset our tracking variable; say we call it j to 1.  You should keep track of what you have or haven’t done either via flags or some internal step counter.  Next, before you start looping, set your loop index, say we call it i , to whatever the value of j is.  What this is doing is allowing you to restart in the middle of your loop.  After each step you should be incrementing i along with setting j equal to i.  This is a bit hard to explain in prose so here it is in pseudocode.  Apologies in advance for my code not being as pretty as David’s.  I don’t have the patience to do all the formatting like he does.

    Dim j as Integer (in declarations section of script so it persists across scans)
    Dim i as Integer (in regular header part of script)

    If (we’re just starting this sequence) Then
           j = 1
         Go to Step 2
    End If

    If (we’re at Step 2)
         ‘ Initialize the loop index counter to whatever the last value of j was.  First time through this is 1, next time through it might be somewhere between 1 and the end of the array
         i = j

         ‘ Start looping and bail out of the loop if we are at the end OR we have reached our pre-calculated halt time
         While (I < ArrayLength AND Now() < Halt Time)
                    { Do yo thang….}
                     {Maybe a message saying you are x% done}
                     ‘Advance the counter and track the counter with j, which persists across scans
                     i = i + 1
                     j = i

          End While

         If (I >= ArrayLength) Then
                  ‘ We’ve made it all the way through so reset j and advance to the next step
                   j = 1
                   Go to Step 3
          End If

    So, if you follow what we’re doing we’re just using a secondary variable to keep up with how far we’ve made it in the array before we had to bail out.  Then, when we jump back in on the next scan we pick up where we left off.  Pretty slick huh?

    One of the interesting things we discovered while doing our testing was the idea that Async scripts are definitely slower.  During a test using this particular method above we saw a distinct 5x slowdown (70 ms to 370 ms) when we swapped to Async from Sync,  changing nothing else.  That should give you some perspective on what it means to swap to a low priority thread.

    I hope this was useful and worth the wait.

    Anyone else have a method they’ve developed that accomplishes something similar?


    Writing Object with Dynamic Hyper-Scale I/O

    October 3rd, 2011

    Ok so that’s a really dramatic title for what we think is a pretty cool technique we’ve been working on for the last few months. 

    A little back-story if you will.  We’re working on an application for a customer that requires us to download recipe parameters to a PLC.  No big deal, right?  Just use the FMO from the EOM.  First problem was that due to some licensing changes, using the FMO wasn’t an option.  Ok, well I guess we can just write our own FMO, that can’t be too hard.  We started down this path but quickly realized that having to create multiple UDA’s for every attribute was a maintenance nightmare and not something we wanted our customers to deal with.  By the way, to get a sense of scale for what this customer needed, we have a machine where we are downloading over 2000 recipe values to a single machine.  Yes, that 2 THOUSAND.  The machine isn’t that big but it does have a massive array of implements that all have about 16 set points so you have a matrix of about 16 * 100 for one section of the machine.. that’s how you get into thousands of formula parameters.

    After batting around a number of ideas we came across what ended up being the ultimate solution, and that’s what we’re going to share with you.  We’re not going to do a massive code dump and give away the keys to the kingdom (there’s lots of hours in this method) but we will show you the main idea so you can possibly implement your own solution.

    It all starts with Indirects.  If you’ve never used Indirects you really should learn about them.  Basically an Indirect is a variable type you can declare in a script.  Once you declare an indirect you call the “BindTo” function on that variable.  Once you have called the BindTo you can now read and write to that indirect and the values will through to the variable you have bound to.  This method can solve a ton of issues especially if you have an unknown quantity in an array or want to write a generic object that can work in a bunch of different settings.  Below is a really really simple example.

    Dim X as Indirect;
    Dim Timer as Integer;

    X.BindTo(“AttributeName.Value”);

    ‘ Wait till the quality becomes good before attempting to read or write
    While NotGood(X)
         Timer = Timer + 1;
    EndWhile;

    ‘ Read from X
    LogMessage(“Value of X is “ + X);

    ’ Write to X
    X = 12.3;

    There is some basic reading in the help files that give you a little more information.  Here’s where the magic starts. Way back in the 2.1 days there used to be a note… at least I think there was… that said you couldn’t bind across engines.  I always wondered why that was?  Turns out through some other investigations that hooking up to I/O that isn’t on the same engine takes at least one scan.  Why does that matter?  Well, if in a script you declare the Indirect at the top of the script that reference is destroyed at the end of the scan cycle.  If its’ destroyed at the end of the scan cycle then how can it persist across scans to survive to hook up to another engine?

    So how do you get around this limitation?  Simple, move your declaration up into the rarely used declarations section of your script. 

    image

    If you didn’t know, variables declared in this section are persisted across scans.  Now we can declare an indirect in this section, do some binding in the script itself, and use the results of that binding in a subsequent scan.  Very cool!  I’ve been doing IAS for a long time and I was frankly amazed that I never put this little trick together (thanks David!).  Actually after using this for this technique I’ve started using these guys a lot more in place of placeholder UDA’s for basic things like flags and multi-scan counters.  Just remember, these are not checkpointed so anything that needs to persist across a failover should be a UDA, not one of these variables.

    So now that we’ve got the dynamic part, what’s this hyper-scale thing?

    So back to the formulas.  Since the FMO was out we basically wrote our own Formula database in SQL Server.  Not terribly difficult to do a basic one.  We ended up blinging this one out with lots of really cool features; that part we’re not going to share.  At the end of the day we make a stored procedure call to the database and it not only retrieves the basic parts of the recipe (names, values, units, limits, etc.) but it also downloads I/O references.  Using these I/O references we iterate over some arrays and dynamically bind our Indirects to the appropriate I/O source.  Did you notice our arrays of Indirects in our declarations?  Pretty sweet huh?

    So what does all this mean?  This means that you can have an arbitrary set of parameters that you need to read and write to and as long as you can get a list of I/O addresses you can scale to your heart’s content.

    One question you might ask is how do you see the values if they aren’t UDA’s?   We solve that issue by copying out the contents to an array of strings that is dynamically scaled based on the number of I/O we’re dealing with.  Super simple example below.

    Dim Count as Integer;
    Dim Index as Integer;

    Count = Me.ValsArray.Dimension1;

    For Index = 1 to Count

                  Me.ValsArray[Index] = IndirectValsArray[Index];

    Next Index;

    Knowing that you can access the size of an array at runtime via the Dimension1 parameter makes things a lot more flexible.  Also, did you know that you can set the Dimension1 at runtime to dynamically change the size of the array?  Even cooler is that if you set Dimension1 to 0 then immediately to some positive value you just cleared out the array.  Is this more efficient than setting the size then iterating and clearing… don’t know, never tested that.

    Back to the problem at hand.

    At the end of the day what matters just as much as flexibility is how well it actually scales.  We’ve tested our solution at 10,000 I/O to a Control Logix L62 PLC (yes, it’s a big array so lots of efficiency there) and we run at about 250 ms to get through the entire array reading every attribute’s value and quality and copying them into our UDA arrays so we can view them with object viewer.  If you’re doing something with 10,000 I/O you might just have to accept a 250ms cycle time for something this far off the reserve.   With something this big I’m thinking you might be on your own engine to avoid async issues with other objects and scripts.  Something we have noticed in our testing from 1 element through 10,000 elements is that the scaling is pretty linear.  That’s good to know if we needed to keep going.  To be fair things may fall apart at 11,000 elements but it looks pretty good up to that 10,000.

    image

     

    If you were really astute, and you’ve worked with Indirects before, one question you might ask is how in the world you design a script that waits for all of the qualities to go good before reading and writing data.  That’s going to be the subject of our next post, designing a system that allows for multi-scan execution of a script without using Asynchronous scripts.  I know you’ll be waiting with baited breath for that one Smile

    - Andy