As we all know, there are 10 kinds of people in the world.

For those of you who haven’t read Zen and the Art of Motorcycle Maintenance by Pirsig, he spends at least one chapter at the beginning talking about how we naturally tend to divide things into smaller pieces in an effort to understand them. The novice looks at a motorcycle and sees the visible things, like a seat, handlebars, and wheels, but the expert sees a fuel system, a cooling system, and the suspension. The same thing or system (motorcycle) can be subdivided different ways depending on what we want to do with it.

My tongue-in-cheek title of this post is an acknowledgement of the many ways we can categorize something like Automation Software, but for my purposes today, I’m making two categories: hammers and levels.

A carpenter carries both a hammer and a level, but the two have fundamentally different failure modes. If a hammer stops working, you’ll know it as soon as you try to use it. As long as it hammers in a nail, it doesn’t matter if the hammer is rusty, dirty, scratched or dented, it’s a working hammer. The level, on the other hand, is a measuring instrument. As novices, we assume that it comes from the factory pre-calibrated, and we happily hang our shelf or picture without testing it, but a professional carpenter knows that they have to check their levels for accuracy, or else the level is useless. You could use a level for years, but if one day it stopped being accurate, you probably wouldn’t know. This is a very different situation than the hammer.

Software in general, and automation software in particular, both have similar examples. You never need to “calibrate” the Axis 1 Advanced proximity switch on a machine because if it doesn’t work, the machine won’t make parts (and you’ll know about it instantly, usually via a 2 am phone call). On the other hand, testing data collection logic is surprisingly difficult because the only way to test it is to compare it with a known-good equivalent. Assuming you created this data collection logic to automate away a manual process, the only measuring stick we can check it against is the manual process we’re replacing. Once the system is bought off and we get rid of the paper system, how do you prove that subsequent changes don’t break the data collection system?

It’s tempting to brush off the problem by saying that anyone who makes a subsequent change has to do a full regression test of the system, including the data collection system, but anyone who has worked in a real factory environment knows that this is unlikely to work in practice. Full regression tests are expensive.

In the greater software world, they use automated unit tests. They take the logic being tested and they run it through a series of automated checks to make sure nothing changes. This works well in an environment like PC programming, but is very difficult in practice for PLC programming because (a) you usually need a physical PLC to execute the logic (unless you have some kind of emulator) and (b) the people maintaining the system are likely not familiar with concepts like unit tests, and are likely to undervalue their importance.

This screams for a system-level solution. Take accounting for instance. Double-entry accounting (the use of debits and credits to force every action to be made twice) is deliberately created to help catch manual entry errors. If your debits and credits don’t balance, you know you’ve made a mistake somewhere, and you go back and check your arithmetic.

In the automation world, the solution is to measure every input to the data collection system two ways, analyze and aggregate both separately, and compare the end results. Create a system warning or fault if the results don’t match. For instance, measure the amount of material going into the machine, and measure the amount of material exiting the machine, both as finished product, and scrap. If the input doesn’t match the sum of the outputs over the same time period, you know you have a problem. The system becomes self-checking (a hammer rather than a level).

If you follow this route, you need to take care to avoid some common traps:

• Don’t re-use logic between the two sides (in fact, try to make them work differently)
• Try to use different sensors or sensing methods (can we measure the input by speed and duration, and the output by parts and scrap weight?)
• Record both, so if there is a discrepancy, you can check them against manual measurements

It sounds like more work, but making the system self-checking actually reduces the amount of testing you have to do, so it’s not as bad as you think. Besides, writing code is a lot more fun than testing it. We automate everyone else’s job, why not the boring parts of ours?

plc · quality · troubleshooting

Since I’m in the tutorial mind-set right now, I thought I’d mention this little gem. Here’s how you can read the ControlLogix (or CompactLogix) PLC system time into a UDT so you can use the current time value in your ladder logic program.

First I created a UDT called “TIME”:

Then you just need to use the GSV (Get System Value) instruction with the WallClockTime class, and LocalSystemTime attribute to read the controller’s time into an instance of your UDT (here I created a new tag called LocalDateTime of type TIME). Note that I used the Year element of the LocalDateTime tag as the parameter, because that’s the first address of the tag. It starts writing there but fills in the entire UDT with the time values:

Now you can program your sprinklers to turn on and off in the middle of the night!

plc · rslogix-5000

I’m still working on my RSLogix 5000 Tutorial, and I decided to include a brief introduction to a Five Rung Logic Block. If you’re like me, the first time you saw or heard about a five rung, you thought it was was a ludicrous amount of logic just to move a cylinder from one position to another. However, I realized most of my non-five-rung motion control logic always ended up being as complicated, or more complicated, than if I’d used a Five Rung from the beginning, so now I’m a convert.

A five rung block is named after the five coils that make up each block (these names differ between programmers and organizations, but they follow this pattern):

• Safety
• Precondition (aka Trigger)
• Command
• Complete (aka “In Position”)
• Fault

Briefly, your safety rung contains all the conditions that must be present during the entire motion. You put conditions like “no critical faults” here. You also put conditions such as interfering axes clear here. For instance, if axis A and axis B should never advance at the same time, then put A Retracted in the safety rung for the five rung that moves axis B to the Advanced position, and vice-versa. This is your “sanity check” logic.

The “trigger”, or “precondition” rung is the signal that will initiate automatic motion. You can put auto conditions directly in this rung, or you can tie in auto mode sequence logic from elsewhere. The difference between the Safety rung and the Trigger rung is that the Safety rung has to stay on during the entire motion, but the Trigger run only has to be true to initiate the motion.

The Command rung is usually a sealed in coil that turns on while the machine should be trying to move this axis to this particular position. Sometimes this is synonymous with an output itself, but in many cases the Command coil will then be used to drive one or more outputs or motion instructions.

The In Position (sometimes called “complete”) rung condition is the logic that interprets the sensors to tell us that we’re actually in the given position. It’s handy to use this coil throughout your program to indicate the state of the machine, rather than looking at the inputs directly. That allows you to invert input logic or add debounce logic later in just one place, rather than everywhere that depends on this machine state.

Finally, the Fault condition is normally a timeout fault. If the command coil is on too long without seeing the In Position signal, then we determine that the motion timed out, and we stop trying. We use this fault in our alarm logic as well.

Once the command condition becomes true, it’s normally sealed in until one of three things happens: the safety condition changes to false, the motion completes and the In Position signal changes to true (normal), or the motion times out and the Fault condition changes to true.

Here’s what a generic five rung logic block looks like. I left an AFI in the trigger rung because you normally come back and fill in that logic later. Note that this is only to move one axis to one position:

The “PB” bit in the command rung is the HMI pushbutton that initiates motion to this position while you’re in manual mode. Note that both manual mode motion and auto mode motion have to follow the same safety conditions, but just have different trigger conditions.

You have to customize it for each situation. The contents of the safety and trigger rungs are obviously unique, but the command rung sometimes changes depending on the scenario. You may have more than just manual and automatic modes. Some automatic actions may actually happen when you’re not in automatic mode. Sometimes you may want an axis to “jog” (for instance a hydraulic axis moves slow, and you may want it only to move in manual mode if you hold your finger on the button) and in that case, you couldn’t seal in the Command bit around the manual mode pushbutton (you’d just seal it in around the trigger contact).

There are other variations. I hope you find this useful. If nothing else, when you come across this logic in someone else’s program, you’ll have a better idea of how it works.

plc

Those of you who know me remember I’ve been itching to get my hands on a Beckhoff TwinCAT PLC industrial control system for a few years now, but I never found a project or customer willing to take on the risk of trying something new. Like a bunch of penguins on a cliff of ice, nobody wanted to be the first in the water.

I’ve recently had the opportunity to apply a TwinCAT PLC and HMI system to a relatively low-risk application, so I finally have something to share. Those of you who know me also know I don’t pull my punches when I talk about products, so here’s an honest review of the TwinCAT system: the good, the bad, and the ugly.

Benefits

Cost

What draws you to TwinCAT is its combination of low cost with a ton of features. I can’t publish prices, but I encourage you to compare prices from Beckhoff, Allen-Bradley, Omron, and others. Beckhoff offers a similar or better feature-set, with as many or more communication options than it’s competitors, for significantly less money (it’s real-time PC control instead of dedicated hardware). On specs it can do everything, and doesn’t bend you over the table when it comes to the price.

Scalability

First, Beckhoff TwinCAT PLC takes advantage of a PC CPU and memory, which means you can scale up the performance of your system for a fraction of the cost of scaling up to large PLC and PAC systems.

Secondly, if you use EtherCAT, you get to scale up your I/O sizes without as much impact on performance or price. The individual I/O slices are significantly cheaper than for other bus technologies, and the servo drives are less expensive because the bus speed is fast enough to close the loop from the PLC rather than in the device itself.

Where TwinCAT really shines is when you need to integrate it to the MES layer in your organization. Most other vendors require you to buy expensive OPC or other connection software, but Beckhoff provides a free DLL for Windows applications to communicate with the PLC over a communication protocol called ADS. (They also provide an OPC server as an add-on if you need it.)

What’s the Catch

Switching software vendors is never easy. If you’re used to your existing PLC software like RSLogix, you’ve got a bit of a learning curve ahead of you. Here are the things that down-right annoyed me about TwinCAT when I was coming from the RSLogix/PanelView realm:

• When you do an online edit, it resets all of your forces. You have to reload your watch list to re-apply them. (Fortunately I always write perfect code the first time, so I never had to do an online change…)
• The ladder logic editor doesn’t seem to auto-wrap (unless I’m missing a setting somewhere)
• The HMI editor crashed a couple of times when I was modifying the HMI, and I had to restart the TwinCAT system (not the PC, just the software)
• The English documention is lacking
• The TwinCAT System Manager DeviceNET master configuration had a bug where it wouldn’t recognize EDS files where the Class value was stored as a 16 bit, rather than an 8 bit value. The EDS file was valid, according to EZ-EDS, but it wouldn’t read it. Changing the value to 8-bit made it recognize the parameter. This made it hard, but not impossible, to integrate an Allen-Bradley Point I/O DeviceNET node.
• Even though the HMI will log an alarm history, there’s no alarm history control for you to view it in the HMI itself (it’s saved to a CSV file)

Here are some things that are just really different, but I’ve had no trouble adapting to:

• You have to define all variables (tags) in a text syntax, instead of in a spreadsheet-like tag editor
• I/O isn’t mapped directly. You define your hardware I/O and your program I/O and then you map them.
• You don’t do online changes while you’re online with the PLC. You “log off”, then make your change, and then “log on” and tell it to upload your change. (You can do this while the machine is running.)

What’s Cool

Using a PC based controller opened up some interesting features, particularly in the HMI:

• You can launch another application from a button click on the HMI (Excel, etc.)
• You can configure some interesting alarm actions, like launching a program, or sending an email
• There’s an (optional) add-in to let the PLC communicate in a limited fashion with a SQL Server
• It runs on commodity hardware. PC hardware is cheap and you can get much larger touch screens for a lot less than the cost of an equivalent PanelView. The “fieldbus” card (EtherCAT) is under $100. Even a Beckhoff EtherCAT to DeviceNET master solution is cheaper than a straight-up DeviceNET card from another manufacturer.
• TwinSAFE – a Safety Controller sitting right on your fieldbus for less than a stand-alone safety controller from another vendor.

Support

I had nothing but superb support from the local Beckhoff technical support guys during this experience. They respond to emails personally usually within a couple of hours, and they have no problem taking the two hour drive to see us if we have any issues that can’t be resolved over the phone or with email. We’re not a “big” customer in any way.

Summary

I’ve just spent about 2 weeks coming up to speed on TwinCAT, having come from an AB/Omron background, with a smattering of other PC based controllers like Phoenix Contact’s Visual Logic Controller (VLC) and MultiProg (formerly PCWorx).

The ease-of-use of TwinCAT doesn’t measure up to RSLogix, but it’s comparable to Omron, and it blows products like PCWorx/MultiProg out of the water (I think I was using version 3 of PCWorx, and I know there’s a version 5 now). Most of my issues with TwinCAT are that its ladder editor just isn’t polished enough compared with RSLogix, but that’s not surprising given its roots in Germany. North America tends to favor ladder logic, where Europe seems to favor instruction list, structured text, and function block diagram. It feels like the TwinCAT ladder logic editor was neglected a bit, but it is certainly workable. Comparing the TwinCAT sequential function chart editor with the RSLogix sequential function chart editor, they were actually reasonably close. I liked that I didn’t have to position individual steps in TwinCAT – it just auto-arranged everything for me.

Price-wise, TwinCAT is significantly less expensive than any other serious player in the industrial automation equipment space, particularly when you start ramping up the amount of I/O and drives you need, and it has similar (and in some cases better) features to offer. It’s even significantly less expensive than the Phoenix Contact solution (from my memory), and given that, there’s no reason I’d ever recommend a PCWorx/MultiProg solution from Phoenix Contact. The Phoenix offering is worse and more expensive.

If you’re in a position where your facility is in the market for a new automation platform, I definitely recommend including the Beckhoff TwinCAT platform in your deliberations. It hits a really unique sweet-spot on price, flexibility, and features that nobody else seems to be able to offer right now. Now that we’re a few weeks in, we have a running machine, it looks like the project will be successful and I have no reservations about saying: if you don’t mind learning something new, give them a try.

beckhoff · hmi · plc · twincat

The “low intellectual property” Fashion Industry as a model for innovation?

I have to agree. Since I’ve started getting involved in open source software, I can tell you that intellectual property laws are the biggest obstacle to innovation that I face. I can say that because I took the time to consult with intellectual property lawyers on everything from trademark to copyright to open source license to software patent. Here’s what I found out: you can’t write a single line of code without violating someone’s patent, somewhere. However, the only time that person will enforce their patent claim is if you’re really successful. The only defense is to pay tens or hundreds of thousands of dollars to build up a patent portfolio of your own so you can partake in a ludicrous arms-race deterrent. On top of that, your patent portfolio offers no defense against so-called “patent trolls”: companies that only own patents and don’t write software of their own. They’re the equivalent of a terrorist cell with nuclear weapons… your nukes are ineffective because you don’t have a city to target.

Copyright is under control, mostly because it’s so much weaker than patents, and it’s free. It stops whole-sale copying of your software, but allows someone to look at your source code, learn from it, and then write it themselves without violating copyright (if they’re careful). Plus, the wide adoption of open source licenses give us lots of material that we can freely copy or integrate.

The irony is that patents were supposed to speed up innovation, but they’ve done exactly the opposite in the software industry.

Do you want to know why none of this matters though? I’m going to make a bold prognostication here:

If the last decade (have we agreed on what to call it yet? The Oh’s?) was about more crap, then the next decade will be about customized crap. Seriously, we gorged ourselves on the mountain of crappage that is sold at Wal-mart and people are starting to wake up with a hangover. We’ve already seen the desires start to change from more to better but I think we’re going to see it change to mine. People are going to want unique stuff. Things that they had a hand in customizing. Something they can use to express themselves.

This has major implications for manufacturing, so it affects the automation industry, and I think it also affects software. Just like the fashion industry, where people use clothes to express themselves, we’re about to enter an age when all the crap you sell to consumers has to have a story, a personality, and be unique. Mass production will remain for your basic staples, and China is going to continue to provide us with that, but North America and Europe are going to have to start producing bespoke crap.

What does this mean for automation? We’re going to see a rise in the build-a-bear style build-your-own-product-on-a-website and have it manufactured and sent to your door the next day. Imagine the logistical changes that need to take place to make that happen. Internet merchandising companies like cafepress already have a leg up in this department. Their merchandise is manufactured after you order it. Now imagine a product with 10,000 or a million different variations, and the automation that has to support it. Imagine the automation required to manufacture products whose specifications change as fast as fashion trends.

On the software side, we’re going to see every major application support custom 3rd party add-ins so everyone can customize their software so it works for them. I think we’re also going to see a rise in “pseudo-programming languages” that let people who aren’t programmers actually customize their applications in ways that we’ve never let them do before. Platforms that give people the power to build solutions to their own needs are going to flourish, and one-size-fits-all solutions like word processors will fade into obscurity.

So I think we’ll have two options: the vast majority of us are going to spend our time building custom things for individuals or businesses. The successful ones will produce products that people can customize themselves.

deep-thoughts · innovation · plc

When you’re immersed in the world of industrial automation for a long time, you may start to think that PLCs, relays and valves are automation. That’s really not the case. Automation applies to doing anything automatically. Those little timers you plug your lights into when you go on vacation are automation. Automatically billing your customer as soon as product is scanned when it goes on the truck is automation.

To me, we’ve solved a lot of the excruciatingly difficult technical challenges, like motion control, because they can be boiled down to academic pursuits and packaged into a black box. The pain is in the integration. The tyranny of dry contact I/O and 4-20 mA analog signals is because they’re universal standards that nobody’s figured out how to patent yet (or the patents have lapsed). These universal standards make integration of components from various manufacturers possible, and it’s the backbone that supports the industrial automation industry.

The leap from the mechanical and electrical to the digital world should have made integration easier and more flexible. To some extent it has, but we now have these proprietary walls built between our solutions. I once wouldn’t hesitate to connect a valve from company A to a relay from company B, and those companies knew it, and the competition kept prices down and drove innovation forward every year. Now I have to make design decisions about which vendor’s hardware plays better with others before I can buy the first item on my bill of materials. What kind of valve bank do I buy? It depends on whether or not I need a ProfiBus or a DeviceNET communication module on it.

There’s no doubt that the digital alternatives are cheaper than hard wired solutions, and we have made progress on standard bus systems thanks to organizations like the ODVA, but what we really lack is a common open file format for storing our automation programs. As long as we’re stuck with the legacy traditional PLCs, then automation itself is stuck in the mud. But there are open automation systems right around the corner. Systems that aren’t going to be focused on industrial automation, and they’re going to tackle the integration problem in a way that proprietary systems just can’t do for us.

I’m building one of these next-generation systems right now. It’s an automation platform. It’s open. It’s free. It’s not a PLC.

plc · snap

Programming both PCs and PLCs sometimes gets me thinking about programming from a higher level. I’ve written a lengthy answer over on StackOverflow about the differences between PC and PLC programming. What I haven’t talked about before is how they are the same.

First, let me define what I mean by PC and PLC programming. By PC programming, I’m generally referring to imperative programming. There are actually two popular PC programming paradigms: imperative and declarative, and the paradigm with new-found popularity, functional, is actually a subset of declarative programming.

How PC and PLC programming are NOT the same

Most PLC programming falls into the declarative category, and most PC programming falls into the imperative category. For examples:

PC:

• Visual Basic, C/C++, C#: imperative
• Lisp, F#: functional
• HTML, XAML: declarative

PLC:

• Ladder logic: declarative
• Function block diagram: functional
• Structured text: imperative

So normally when we talk about the differences between PC and PLC programming, we’re talking about the differences between imperative and declarative programming, but there’s obviously overlap on both sides of the fence.

The major difference, however, is audience. In North America at least, we write PC programs with the expectation that other programmers will have to read, understand and make changes to them, but we write PLC programs with the expectation that people in the maintenance department will be expected to go online with and troubleshoot our programs. Just think about how odd that would be in the PC world: when a word processor crashes, nobody whips out their debugger, figures out what caused the program to crash, makes a fix and continues writing their letter. Primarily this is because the source code doesn’t come with the word processor, but it’s also because the programming language can only be understood by programmers.

How PC and PLC programming ARE the same

When you look at what makes a PC program good or bad, on a high level it’s the same thing that makes a PLC program good or bad: readability. Now as I’ve pointed out, the people who have to read the program is different in each case, but really readability is the fundamental measure by which experienced programmers rate programs.

On the PC side ,the name of the game with readability is modularity. You want to divide up your program into parts, and you want to make those individual parts as self-contained as possible. You want to minimize the interaction to these parts as much as possible. That makes it easier to reason about the program because you’re abstracting away the underlying complexity on each piece and leaving a less complex interface that you can interact with. The entire domain of object oriented programming is an extension of the concept of modularity.

On the PLC side, readability is equivalent to being able to troubleshoot the machine when it’s down. Experienced PLC programmers ask themselves, “if this machine stopped unexpectedly and I had to figure out why it stopped, what would I do? How can I make it easier for someone following that process to figure out what’s wrong with the machine?”

It turns out that most people troubleshooting a machine follow a similar procedure: you start at the outputs and you work your way backwards. You generally have a good idea what the machine is supposed to do next (e.g. move slide A to position B). You can look at the print set, or even the valve itself and figure out what output should be turning on. You look at the indicator on the output card and it’s not on, so the logic isn’t telling it to turn on. You crack open the laptop, and you find that output. You’re looking for one thing: the COIL.

Notice the one big mistake you could make if you’re writing a program: you could use a whole bunch of set and reset (or latch and unlatch) instructions to drive your outputs. Based on my description, you can easily see why that would make the program less readable: which set instruction is the one that’s supposed to be turning on the output right now? If there’s only one, it’s easy, but if there are 10, you’re already lost.

Let’s assume you do find the coil that drives this output. Your next step is to follow the logic back through the rungs, right clicking on the conditions that aren’t satisfied and cross referencing until you understand what the machine is waiting for. What are some obvious mistakes you can make that would hinder this process?

• Using an integer (or sequencer – yuck!) to store your automatic process step number rather than using individual coils for each step
• Using set/reset or latch/unlatch instructions more than once on each bit
• Using really long tag names so readers have to scroll left/right or up/down more than necessary to read one rung
• Calling subroutines more than once per scan so you can’t see the state of the logic in the subroutine (newer controllers have function blocks where you can drill down into individual instances, which is nice)
• Using For Loops – same reason
• Having logic that is conditionally scanned – particularly in controllers where it isn’t obvious if the logic you’re looking at is scanned or not
• Mapping your inputs or outputs by block copying them to or from a user defined type, word or array (Don’t make the reader start counting bits! The line is down!)

Once you start thinking from the point of view of someone troubleshooting the machine, your perspective on good vs. bad programming really changes. You realize that techniques that seem to save you time while you’re programming end up costing the company hours of lost production time while maintenance picks their way through your cryptic logic.

Next time you’re writing your ladder logic, think of the poor maintenance guy who has to figure out what’s wrong, and try to make his life a little less miserable.

ladder-logic · plc

After my recent rant about Rockwell Automation and their frustrating online support, I received an email from Joe Harkulich, Global Quality Leader at Rockwell Automation. The first thing he did was, helpfully, get me straightened out with my TechConnect ID. Apparently the one we were given wasn’t our real TechConnect ID, and it happened to be the ID of another company in Mexico, which caused some confusion on my end. Once that got straightened out, I could access the article I was looking for (a bit late, but better than never).

Joe then invited me to join a conference call with Esther Beris (Rockwell Knowledgebase), Jon Furniss (TechConnect), and Rob Snyder (Senior Manager for Rockwell’s remote support). I’m thankful for them taking the time to do this, as I’m really only a moderate user of Rockwell Automation products. (I really don’t think they’re very concerned about my legions of blog readers and twitter followers). I’ll relate a few of the details of the call here, and I really want to express my gratitude to everyone involved for their time.

The first item on the agenda was Joe addressing some misinformation in my previous blog post. I had made a comment about Beckhoff’s 350MB knowledge-base available for download, and I had suggested that Rockwell should do something similar. It turns out that Rockwell does offer TechConnect subscribers the ability to receive 3 DVD’s full of all their product manuals and their entire knowledge base. Joe offered to send me a copy, and I accepted. I received them yesterday and installed them on the laptop at work. This will be nice pending an on-site trip that’s scheduled next month. I’m a bit concerned that the discs are dated May of 2008. I was expecting them to be about 6 months old. We did discuss the idea of making them automatically download updates of the knowledge base, etc., so your offline copy is always up to date, but they pointed out that this would be a huge amount of information so it probably couldn’t happen, but they would take the idea into consideration. Personally I think a solution like that would have a lot of value.

We then turned our discussion towards some of the issues I had raised with Rockwell’s online support in my previous post. I certainly made it clear that I have always been impressed with Rockwell’s paid telephone support, and I really have nothing to complain about there; they are simply awesome! When it comes to the online support, we talked about these issues:

• Single sign-on
• Expiry of your account if you don’t log on for 6 months
• Access to product manuals, EDS files and support directly from the product page
• Paid vs. unpaid content

Single Sign-On

It makes no sense to me why you have to sign on separately at www.rockwellautomation.com and at their knowledgebase. Apparently it doesn’t make a lot of sense to the folks at Rockwell either, and they’re working on it. However, there’s no time-line for the resolution of it (or at least they couldn’t give me one). I think it’s one of those cases where the politics of a big organization are getting in the way of doing something really simple, and I can appreciate that. I did get the impression that the people involved do care, and it will get done, eventually.

Expiry of your account if you don’t log on for 6 months

In past years I would frequently do a lot of automation work, and then switch to PC programming for a project or two, and then switch back to automation work as I’ve recently done. I always found it a bit frustrating the first time I tried to download some EDS file and Rockwell had canceled my account because it had been longer than 6 months since I’d signed in. Apparently I’m not the only customer to complain about this, and Rockwell is in the process of changing this. Bravo!

Access to product manuals, EDS files and support directly from the product page

Let’s say I go to the ControlLogix Product Page. On a normal website when I look at a product I see exactly what that one component looks like, a link directly to the datasheet or user manual, a tab with all the specs on it, and all the information I need to buy that product. But the ControlLogix page is a brand page, not a product page. It’s almost completely useless to me as an Engineer. There is a Literature link, and that has a selection guide and something else, but then it tells me I have to go to the literature library to find all the good documentation.

When I explained this on the conference call I got some good laughs, because again, I’m not the first person to complain about this. Apparently this is being resolved, and they do have a time-line: September 2010. That’s great news and I’m really happy to hear it.

Paid vs. Unpaid Content

I’m afraid the issue of paid vs. unpaid content is still a sticking point for me. They explained to me that they chose a paid model for their support because they can provide better support. There’s no question in my mind that the quality of Rockwell’s telephone support comes through time and again, but I have to disagree with the pay-wall model they’re using for online support. I really think what’s happened is that the technical support group within Rockwell is literally scared of what could happen if they didn’t maintain a strangle-hold over the information that guys like me need to do their job.

I’ve talked before about vendor lock-in in the industrial automation industry, so I’m not going to go into that again. But this is a case where a company is selling me upwards of $10,000 of automation equipment per project and once they’ve done that, they want to charge me extra for access to the information I need to make it work the way it was supposed to work in the first place. I understand the paid telephone support model because there’s a one-to-one relationship between the amount of time I spend on the phone and the amount of time they have to pay someone to talk on the phone. But while they’re doing that, they’re already creating a knowledge-base of information. As smarter people than I have pointed out repeatedly, the marginal cost of distributing that electronic information to one more person is as close to zero as humanity has ever seen. No, you don’t have to because copyright is always on your side, but if you don’t, you open yourself up to being undercut by the following business model:

• Start an online knowledge base where the customers can build it
• Offer subtle rewards to the people who contribute (peer recognition within the knowledge community)
• Let Google index the site and drive traffic to it

…which is exactly what ControlsOverload is. Remember in my last post where I said that I took 3 minutes of my own time and posted the question and answer related to my problem at ControlsOverload? Check out what happens when you search for compactlogix type 01 code 01 fault or even compactlogix powerup fault. The first item in the search results is my question and answer. You can even go to that page and add more information, or correct inconsistencies and you don’t even need to sign-up!

Rockwell has one group that makes products and another group that charges people for support on those products. In order to protect the revenue model of the second, they put barriers in place to maintain a monopoly on information, even though every single industry that has relied on maintaining a monopoly on access to information is dying a slow miserable death. Meanwhile I wasted another hour of my time trying to get a product I’d already purchased to work, and they had the information to cut that time down to 2 minutes, but they put all these barriers in my way even though I’d already paid for support!

The price to hire a PLC programmer is anywhere from about $50 to $100 an hour in my experience. Let’s say $75. Every time Rockwell’s support barriers waste an hour of my time, the total cost of ownership of their equipment goes up by, on average, $75. On top of that, it frustrates me enough to blog about it, and there’s an opportunity cost as well. I could have been using that hour to improve the efficiency of the machine in some other way rather than fighting with Rockwell’s internet site.

Here’s my suggestion to Rockwell to fix the situation:

• Create and harness a money-free and barrier-free user generated content site, just like ControlsOverload, where customers can help other customers
• Keep the excellent telephone support, but have the support people contribute to and maintain the free site rather than the paid site
• Move all the paid support information to the free site (this doesn’t include premium content, as described below)
• Use the free site to advertise premium online content that users like me are willing to pay for

What do I mean by paid premium content? Tutorials. Training videos. Example applications. Code review services (a support person can review your automation program and offer advice). E-books. Insider news & tips. Access to beta versions of upcoming software releases. In short, something I’m willing to pay for above and beyond what I believe I’ve already paid for.

Good luck Rockwell – the next few years are going to be interesting.

controlsoverload · frustrations · plc · rockwell-automation

I’ve decided to create a tutorial for beginners getting started with RSLogix 5000 from Rockwell Automation. Part 1 is already posted: Creating a New Project. I will be filling in the rest over the next few weeks. I hope new automation graduates and experienced members of the automation industry who are migrating from another platform will both find something useful.

As always, I welcome any and all constructive feedback.

allen-bradley · plc · rockwell-automation · rslogix-5000 · tutorial

I recently had a problem with an Allen-Bradley CompactLogix processor. The power went out, came back on, and the processor had faulted with a major fault, type 01, code 01. The fault message said “Power lost and restored in RUN mode”. There must be a way to disable this fault, and just have it go back into run mode so the operator can recover.

I Googled for the fault message and I got a link to a helpful forum thread. In that forum thread there was a link to a Rockwell Automation Knowledgebase Article that seemed to have the information I was looking for. I clicked the link and it told me I needed to login. That’s annoying enough, but fine. I tried my normal username and password for such sites, and that didn’t work. I went into my encrypted file and pulled out the username and password I’d saved for Rockwell Automation. That didn’t work.

Ok, fine. It had an option to email me my user name. I waited a few minutes for the email – yes the username I was using was OK. I clicked on the option to reset my password. Got the email, reset the password to the one I had saved anyway. Success! I was logged in, but it had forgotten what article I was trying to get to, so I went back to the original forum post and clicked the link again. “This article has been locked, or …” blah blah blah about a TechConnect ID. Ok, so I go to my profile page and click on the tab for TechConnect support IDs. None registered. Hmm, no obvious way to register one. I go to the other profile page… TechConnect ID… excellent! I enter that, click save, and now it needs my company name, address, etc. Ok, I enter that… but now I can’t click save. I have to go backwards, re-enter the TechConnect ID, and my company name and address, and then it saved. Ok, great!

I go back to the forum and click the link again. “This article has been locked, or …” What?!? I check my list of TechConnect IDs again, and there’s nothing there. This is really annoying me at this point. Oddly, under my name it now has the name of some Mexican company, and some place in Mexico. I check my profile page again and it all looks right, and I double-check the TechConnect ID. It’s right (and I know it’s right because I’ve used it to call Rockwell tech support recently).

I go back to Google and search again for the fault code, and I see a link to the Rockwell knowledge-base article. I click on that… same message! What’s going on here? How can Google even index a page that’s behind a sign-up wall and doesn’t even show you the page unless you have a valid TechConnect ID?

Going back to the original forum post, I did find some useful information in there, but obviously the knowledge-base article would have helped me the most. How far behind is Rockwell Automation’s online support? They’re still in the 90′s. I realize there are still a lot of people in this industry who are happier to pickup a phone and call their phone support in a situation like this, but as time goes on and the Millennial Generation continues entering the workforce, self-help focused people like myself are going to be more and more commonplace. We won’t settle for getting our questions answered in hours or even 30 minutes anymore; we want to solve our stupid problems like this in minutes or even seconds, and the technology is there to let us do it. Stop putting unnecessary barriers in our way! Rockwell Automation online support: FAIL!

By the way, I just spent 3 extra minutes and entered my technical question and answer over on ControlsOverload, a website for technical questions and answers about industrial automation. A website where you don’t have to sign in, and nothing is ever blocked behind any kind of wall. If Google can see it, so can you. This is the future of finding information on the internet. Rockwell Automation: Make it easy for me to use your hardware, and maybe I’ll buy more! You know what… here’s Beckhoff’s whole 350MB knowledge-base available for download so you can take it with you onsite when you don’t have an internet connection. Brilliant isn’t it? It’s called openness and it’s the new name of the game. Wake up!

Update (9 MAR 2010): The Global Quality Leader from Rockwell has contacted me and it looks like we’re going to have a constructive discussion about some of these issues (and he also gave me a different TechConnect ID to try). +1 to Rockwell for having their ears on!

Update (20 MAR 2010): I posted a write-up on Rockwell’s response here.

controlsoverload · frustrations · plc · rockwell-automation