Author Archives: Scott Whitlock

About Scott Whitlock

I'm Scott Whitlock, an "automation enthusiast". By day I'm a PLC and .NET programmer at ETBO Tool & Die Inc., a manufacturer. In the evenings, my wife and I run a question and answer site for parents called moms4mom. On weekends, I write and maintain an open source ladder logic editor called SoapBox Snap. You can follow me on twitter: I'm @autom8it

The PLC is Hallucinating

If you’ve been anywhere around YouTube or any TED talks recently, you’ll probably have heard the idea that your brain hallucinates your conscious reality. Simply put, your senses aren’t good enough to give you a perfect picture of reality as it is now, so your brain fills in the gaps by creating a model of the world in your mind and it uses your senses to direct that model, to keep it grounded in reality (whatever that is). This internal model of the world is what you use to make decisions.

Probably the most famous example of the fallibility of our senses is the blind spot we all have in our eyes, where the optic nerve enters the eye. Yet we don’t experience this blind spot. What we experience is an internal model of the world, and the lack of visual information at the eye’s blind spot simply has no effect on our internal model. Certainly, something happening in our environment can be obscured by the blind spot, but even with one eye closed we don’t perceive a big hole in our visual field.

Have you ever seen something out of the corner of your eye, or at a distance, and thought it was something else? I routinely experience this when I’m driving to work in the dark and passing a forested area beside the road. A weirdly shaped bush or a branch can be recognized as a deer about to jump out on the road. I instinctively tap the brakes, only to realize it’s not a deer, but a bush. When this happens I’m inclined to say I “thought it was a deer, but I was mistaken.” In truth, I literally saw a deer, and in my internal model of the world it was a deer, fully formed, ready to leap onto the road, and I reacted accordingly. Occasionally this happens, and as I get closer, I confirm it really is a deer.

A while ago, I wrote an article on this blog about part tracking, and if you don’t remember reading it, I suggest you go back and take a look (it’s a rather short read). No, honestly, go back and read it.

Sound familiar?

For the most part, in the world of industrial automation we’re blessed with very reliable sensors. Thru-beam and proximity sensors are remarkably reliable. But as I discussed in the part tracking post, even our best sensors can sometimes send us erroneous data. There are blind spots in our industrial sensors.

Novice PLC programmers are obsessed with inputs and outputs. One of the earliest exercises I like to give to a new PLC programmer is to program a light to flash continuously. More than half will start their program with an input (usually a pushbutton). Clearly this is incorrect… it’s a flashing light, so the only input is time. The solution requires an internal model of time (in our case the on-delay timer). But to someone fresh out of their first PLC programming course, inputs and outputs are king.

In my part tracking post, I literally ask you to “focus on part tracking”. Your first job, as a PLC programmer, is to create a simplified model of the world in the PLC’s internal state and maintain the accuracy of this model using the inputs. I call this “part tracking”, but it’s literally the PLC’s hallucination of its world. The internal model of the world stored in the PLC is the proper foundation of the decisions the PLC will make. This is the core idea behind of the patterns of ladder logic programming. The mission pattern, step pattern, and five rung patterns are all on the output side of the program. They are the actuators of a decision that was made in the mission controller based on the model (a.k.a. part tracking).

Now here’s a bonus thought… is the PLC’s internal model an accurate representation of reality? Well, yes, your goal is to make it more accurate than our sensors. But is it a complete representation of reality? Clearly not. We pare down the model to the simplest, most fundamental level. We only include the bare minimum of what we need for the machine to do its job.

So… what could that mean about our own internal model of reality? If we evolved this consciousness (or I guess if it was designed like we designed the PLC program), there’s a good point to be made that accuracy has a lot of value. Our internal model of reality is likely more accurate than our senses can perceive. But is it complete? Almost certainly not. Modeling a part of reality that we don’t need in order to survive is just wasteful.

Fundamental vs. Incidental Sameness

When a programmer sees duplicated code, our gut reaction is to recoil in disgust. After all, the Once and Only Once principle is at the very core of the programmer’s thought process, but I would argue that this caveat is even more fundamental to the concept:

Beware of introducing unnecessary coupling when refactoring for Once and Only Once.

What does “unnecessary coupling” mean? That’s what I want to look at here.

When you apply the Once and Only Once principle, you’re defining everything that uses that code to work the same way. For a simple example, consider the code to reverse a string (in place):

int n = str.length();
for (int i = 0; i < n / 2; i++)
swap(str[i], str[n - i - 1]);

If I advocated for writing out that code every time you had to reverse a string, you’d rightly conclude I was being ridiculous.

Fundamental Sameness

The code to reverse a string clearly belongs in a function because reversing a string is something that’s Fundamentally the Same no matter what string we’re trying to reverse anywhere in our software. If we discover a better way to do it, we want to be able to change it in one place.

Strings are also fundamentally the same. We can define them in strict terms. They don’t change, and even if the programming language designers decided to change the implementation of a string, all strings in our program have to change at the same time.

As the programmer we have the power to shape the internals of the program however we want, and this is powerful. We can define how strings are stored, and build upon that by defining how to reverse them.

However, we rarely get to define the real world that we’re interacting with.

Incidental Sameness

Anyone who has ever created business logic in their software will understand the pain of making rigidly defined software that represents the… less rigidly defined.. real life business rules of a company.

Let’s say your company has account managers separated into two groups. One group of account managers handles industrial clients and the other group handles commercial clients. The head of the industrial account manager group comes to you and says they need a new feature in the CRM software: their new group policy says we need to review accounts monthly, and they need a reminder alert generated if any account hasn’t been reviewed in 28 days. You go and check with the head of the commercial accounts manager group and they agree that this is a good idea.

You write a function to tell you when to generate alerts for an account:

bool generateAlert(Account acct) {
return Today.Subtract(acct.LastReviewedDate).TotalDays >= 28;
}

Do you see the problem? The two groups may both have the same policy: “accounts must be reviewed monthly”. However, those policies are only Incidentally the Same. There’s only one function and it defines alerts on all accounts to be generated the same way regardless of which group handles the account. If the head of the commercial group later comes to you and says they want an alert if it’s been 14 days since the account has been reviewed, then you’ll need to rewrite this feature.

Yes, it’s a contrived example and changing it is unlikely to be a big deal, but what you’ve done is created unnecessary coupling between the two groups. You can’t change the code for one group without affecting the other. In fact, tomorrow, the commercial group may not want alerts at all, or they may want alerts generated in different formats or sent to different people. Depending on the structure of your company, the two groups may need to operate completely differently due to differing client needs. It’s unlikely that management would have created two different groups unless they recognized the need to operate independently.

Application to Industrial Automation

As Engineers we love to copy tried and true designs. That gripper design works well, so let’s use the same gripper on robot 1 and robot 3. Those VFD drives seem to be really robust, so I think we should use them on all the pumps and conveyor drives of this new machine.

However, industrial machines are constantly modified for a variety of reasons: changing product and process needs, temporarily bypassing failed components, or replacing components with different parts because a better product is available, or the old model is no longer being manufactured.

When programming an industrial machine, you need to treat identical components as only Incidentally the Same. Your program needs to allow for the complexities of real life in a manufacturing environment. Components that are only incidentally the same will often need to be changed, and they usually won’t all change at the same time. Even if you want to change all the pumps on your machine, you’ll likely only change one this week, make sure it works, and then change the other three during some scheduled downtime next month.

That doesn’t mean you shouldn’t follow the Once and Only Once principle. Make a function block (or an Add-on Instruction in Allen-Bradley) for communicating with your specific model of VFD drive and use it everywhere you use that drive. That’s because VFD drives with the same part # are Fundamentally the Same. Make a function block for logging events to your plant-wide data collection system, because you’ve created an explicit definition of an event, and you can control it. Write common functions for calculating the distance between two points, or calculating voltage given current and resistance, because those are defined and won’t change (or, if they do change, you’ll want to change them everywhere).

On the other hand, the chance that the grippers on robot 1 and robot 3 are going to be identical two years from now is vanishingly small. And those pumps? One of them will have a bypassed flow sensor for a week next July. Plus, someone’s got a crazy idea for controlling the second conveyor section based on the measurement from a laser thickness gauge.

Be thoughtful about code re-use. Use it for things that are Fundamentally the Same, but beware of components that are only Incidentally the Same. Don’t create unnecessary coupling between incidentally similar components.

Thinking Critically?

Every once in a while I’m talking with someone and they open up about their beliefs. I’m pretty sure this is a normal thing humans do with each other, but it always catches me off guard.

Sometimes their beliefs are strongly at odds with my worldview, like they might say, “You know, I really think the moon landings had to be faked.” I always react immediately, and it doesn’t appear to come from the thinking part of my brain. No, the first thing that happens, completely automatically, is an emotional reaction. I know because I can feel my face turn red.

It only lasts a few seconds. I automatically take a deep breath. The thinking part of my brain starts working again. “Remember,” I tell myself, “there’s always a chance they have information you haven’t heard yet.”

I ask, “Well, how do you know? What convinced you?”

“You know, I’ve been watching this guy on YouTube and he just makes a lot of sense,” they say.

The Rational Animal?

These people I talk with aren’t dumb. They’re logical, thoughtful people. In fact they can’t do their jobs without being ruthlessly analytical.

I often have to diagnose a problem with a machine, and it’s clearly a problem in the I/O network. Whether it’s DeviceNET, Ethernet/IP or EtherCAT, we’re often dealing with a daisy-chain configuration. You open up your diagnostic tool and you can see that nodes A, B, and C are online, but nodes D and E aren’t. Clearly the problem is between nodes C and D.

This where I say, “well, it can only be one of three things: the transmitter in node C, the communication cable, or the receiver in node D.” I usually check the communication cable first, not because it’s the most likely, but because it’s the easiest to check by simply grabbing my 100 foot cable and temporarily replacing the existing cable. Same problem? It’s not the cable. Now it’s either the transmitter or the receiver.

Now I can try bypassing the receiver… if I connect node C directly to node E, and node E communicates, then I know the problem is node D. Problem #1 solved. Replace node D, pack up the tools, and problem #2 gets a promotion.

Why does logical thought come so easily in this case, but completely elude us in other cases?

Bias?

There’s one very important distinction between these cases. When I’m diagnosing the I/O network on a machine, I don’t have a vested interest in whether it’s the transmitter, cable, or receiver that’s faulty. I’m an unbiased judge. I don’t identify with any of them.

That isn’t always true. If I’m working on the same machine with a co-worker, and they designed the transmitter, and I designed the receiver, then I really wouldn’t want the receiver to be the problem, would I? What would happen if it was the receiver? Would I feel embarrassed in front of my team? Would I lose standing within the group? Probably not where I work, but it’s conceivable. In some places it might be quite likely.

One thing’s for sure… our impartiality is in question.

Perhaps more importantly, the machine doesn’t have an agenda. It’s not actively trying to influence you, like a person is. To diagnose a machine, all you need is logic. To listen to an argument, you need to think critically.

Fight? Flight?

Our bodies and our minds are finely tuned to threats in our environment. When something threatening happens, our bodies initiate a stress response (also called the fight-or-flight response) by releasing hormones that get us ready to deal with threats. It happens when a deer jumps out in front of your vehicle at night. You react quickly and automatically, but afterward you can feel the physical changes. Your heart is racing, your senses are more sensitive, and your reactions are faster.

How does the body improve your reaction time? Less thinking.

Oh, later it may feel like you saw the deer jump out, you hit the brakes and swerved out of the way, all because you decided to do it. But you didn’t decide anything. You reacted, and the thinking part of your brain couldn’t possibly have worked fast enough to be a part of that process.

How does the automatic part of our brain know what to do? I’m not sure. I’ve often wondered if that’s what our dreams are: our brains simulating stressful situations, letting the thinking part do it’s thing, and recording the result for future automatic playback in a stressful situation. Of course, I have no evidence of that, but it’s fun to speculate.

You see, it’s not just deer jumping out at us that causes stress. Many of the most common bad dreams are about social embarrassments, like showing up naked in an inappropriate place, or showing up late to a meeting.

Not only are we finely tuned to physical threats, but also to threats to our social status. We’re very sensitive to anyone thinking badly about us, or thinking badly about a group we identify with. Nobody wants to be voted off the island. It isn’t just that it makes us feel bad… it’s that it causes our body to prepare for a fight. It inhibits the thinking part of our brain.

It’s not about the Moon Landing

So when someone tells me they think the moon landings were faked, my body somehow sensed a threat and reacted. Why?

It doesn’t make sense to be upset because someone is wrong. We’re wrong all the time. Disagreements always involve at least one person being wrong, but disagreements lead to learning. I think we should have a right to say wrong statements (just don’t be shouting “Fire!” in a crowded theater).

I didn’t react because I thought they were wrong. The reaction was almost instant, so I wasn’t thinking anything. Any disagreement causes the reaction, because any disagreement is a potential threat.

It may not seem like it, but we go out of our way to avoid disagreements, particularly with the people we interact with directly. In fact we tend to align our views with our social circle. People measurably change their political views when they move to a new place. Some people will give an answer they know to be wrong just to conform.

Winning Friends? Influencing People?

I really am curious. I’m fascinated by how stuff works, whether it’s stars and planets, subatomic particles, machines, or even people. I really want to know the truth, and I don’t actually have a reason for wanting to know. I’m just curious. But I’m also comfortable with the answer, “I don’t know.”

I think we all tend to assume everyone else is like us, and I was no different. I assumed everyone else was just curious like me. I now see that’s clearly not true. We’re hard-wired to care about belonging in a group. Belonging is comfortable. It reduces stress. We crave it. There’s safety in numbers.

A few years ago I was at a dinner hosted by a local startup incubator and maker space. Not big internet startups, just local people starting small businesses. In front of us, there was a young woman who was growing a business that sold dietary supplements and provided nutritional advice and that sort of thing. I asked her how it was going, and she said, in a sort of defeated tone, “I’ve learned that telling people what they need to hear doesn’t work. I just tell them what they want to hear.”

It’s funny what we remember. I don’t think I’ll ever forget that short conversation.

This Guy on YouTube

“You know, I’ve been watching this guy on YouTube and he just makes a lot of sense.”

Partly this is a problem with videos. When we read words on a page, we have more time to reflect, to think critically. We can read at our own pace. Videos don’t work like that.

But really, it’s pretty easy to convince you something if believing it makes you feel good. Every rock band walks out on stage and tells the crowd how awesome their city is. The audience always agrees, but let’s face it… some of those places have to suck.

Also, since we’re so finely tuned to threats, it’s actually pretty easy to convince you something if believing it makes you feel threatened. We’re tuned to threats because over-reacting is a better survival strategy than under-reacting. It’s better to err on the side of caution and assume there might be a lion hiding in that tall grass. If you’re wrong you got some more exercise, and if you’re right, it saved your life.

Thinking critically is hard. Like… really hard. We’re not built to do it.

If you want to start thinking critically about a video, or an article, you need to figure out who the author is, and what their motivation is. It’s rarely curiosity. It’s usually money or status. People post videos to YouTube to make money, and they do that by getting more and more people to watch their videos. They don’t need to post factually accurate information. They just need to post something people will share. Popular videos just say what people want to hear. They appeal to emotion, not logic.

But if you really want to be a critical thinker, it’s much harder than that. Before you watch that video or read that article, you need to stop and ask yourself, “What do I want them to say?” Because unless you’re watching a video or reading an article about a topic that means nothing to you (doubtful), then you’re biased, and you need to be honest with yourself about your own biases before you can worry about theirs.

That’s hard.

Focus on Part Tracking

There are many ways to visualize your program at a higher level. Take a look at this model for a second:

Inputs -> Part Tracking -> Outputs

I tend to focus on Part Tracking as a core part of my programming work. You can think of the part tracking as the PLC’s internal model of the outside world. That is, it’s the world that can’t be sensed directly by the inputs. In some cases the part tracking information can be used in spite of the current state of the inputs. Let me explain.

Imagine a work cell where a robot places a part in a fixture, a nut feeder feeds a nut onto the part, and then a weld gun extends from the top, and welds the part.

There are various places where we may place sensors in this situation, and you sometimes don’t have much control of it. Perhaps there’s a sensor in the fixture detecting that the part is there, but sometimes the mechanical engineers can’t find a place to fit one in. Sometimes the robot gripper has a sensor (or sometimes we use vacuum sensors to detect a part we’re picking up with vacuum). Perhaps neither is the case and we just know the robot has a part because we knew there was one in the infeed fixture and we know the robot went there and gripped, so we just assume it’s there.

Part Tracking gives you a way to remove some of this uncertainty. Create a memory bit (M1) to indicate there’s a part in the robot gripper. If the robot gripper moves to the fixture position and opens, then clear memory bit M1 and set memory bit (M2) indicating there’s a part in the fixture.

Keep this part tracking logic separate from your fault logic (so your part tracking logic is predictable and similar from project to project). If you happen to have a sensor in the fixture, create a separate rung that seals in a fault if the M2 bit is on and the fixture part present sensor is off, and use a delay timer of 100 ms or so just to allow for a sensor blip. Make sure the fault stops the cell. In this case I typically require the operator to go into the HMI and manually clear the “ghost” part in the fixture (i.e. clearing the M2 bit) before they can reset the fault. That way I know that they know the part really is missing. Likewise, a second fault rung seals in if M2 is off, but the sensor indicates a part.

Now as I said, sensors can sometimes be unreliable, and nowhere is this more common than in a weld cell. The high magnetic field generated by the weld current, and the weld expulsion splattering around is a surefire way to mess with both inductive proxes and optical sensors. Separating part tracking from sensors gives you a way to deal with this. For instance, when the welder is firing, mute out the faults so they don’t trigger if the sensor suddenly reads incorrectly.

To put it another way, use your part tracking to make decisions, and use the sensors to validate the part tracking, but only at times when you’re sure of their validity.

Furthermore, you can use inputs to initiate actions without requiring them to be on for the entire duration of the action. For instance, you can require that the fixture sensor is on to indicate a part is present before you fire the welder, but you can seal in the Fire_Welder bit around the sensor contact so the sensor can flicker after the output turns on without causing the output itself to flicker on and off.

Idiomatic Ladder Logic

I want to talk about the concept of “idioms” or the idea of “idomatic” when it applies to programming languages.

Python is said to have “strong idioms“:

One reason for the high readability of Python code is its relatively complete set of Code Style guidelines and “Pythonic” idioms.

When a veteran Python developer (a Pythonista) calls portions of code not “Pythonic”, they usually mean that these lines of code do not follow the common guidelines and fail to express its intent in what is considered the best (hear: most readable) way.

Many years ago I did some programming in Perl. A main philosophy of Perl is the acronym TMTOWTDI (There’s more than one way to do it). That’s an example of a language with “weak idioms”. This goes hand-in-hand with Perl’s apparent lack of focus on readability. In fact, some detractors of the language claim it’s a “write-once, read-never” language.

When I started writing the Patterns of Ladder Logic Programming page, I wasn’t thinking of it at the time, but in retrospect I was trying to document “Idiomatic Ladder Logic.” Yes, there are many ways to accomplish the same task in a PLC, but you should stick to idioms when they help communicate the meaning of your code.

When you deviate from these idioms, you’re communicating to the reader that something about this case is different. If I expect a fault coil to be sealed in, and you make it a set-reset, you’re communicating to me that this fault condition needs to survive a power outage. That’s useful information. If your fault coil isn’t sealed in, that must mean you intend it to be self-clearing.

Even your deviations should be idiomatic. Don’t make fault coils self-clearing by putting a Reset instruction for the same coil somewhere else in your code. That won’t be obvious to the reader, and is the reason for the idiom “don’t use a coil more than one place in your logic.”

Most or all new PLCs can be programmed in multiple languages, from the IEC-61131-3 specification. These languages are very different, and the idiomatic way to do something in one language isn’t necessarily the way to do it in another language. For instance, doing any kind of loop (for, while) is non-idiomatic in ladder logic, but is certainly an idiomatic construct in structured text. That means part of our job as programmers is to pick the correct language to express our intent.

Data collection, string parsing, and math is naturally expressed in structured text (ST), but control logic is naturally expressed in ladder diagram (LD). Part tracking logic can go either way. Sequential function chart (SFC) is perfect for expressing a sequence, ladder diagram is a good runner up using the Step Pattern, and structured text requires that you define a state machine, which is the least expressive option.

I once wrote a bubble-sort routine in ladder logic for a SLC500 PLC. I wish I’d had structured text back then. First pick the right language, and then pick the right idiom.

General Principles of PLC Programming

The PLC tutorials on this site focus on specific principles, but I’d like to point out general principles too. In fact, general principles of PLC programming are the same as PC programming, though the way we satisfy those principles can vary widely between those two domains.

Principle 1: Readability

This is number 1 because it trumps everything (short of functional correctness, of course). Many of the principles focus on making the code easy to change, but before you can change it you need to understand how it works at a deep level. The easier it is to understand, the easier it is to change, so readability drives most of my PLC programming tutorials and explanations. Also remember your audience. Readability means someone with only an electrical background and no C/C++/Java/C# experience should still be able to walk up and understand your logic. That is the entire point of PLCs.

Principle 2: Keep Things that Change Together Close Together

If you know that changing one piece of code will require you to change another piece of code, and you can’t somehow combine them into a single piece of code, then at least put them next to each other. The more related they are, the closer they should be in your program. Do whatever you can to help your future self see the trap you’ve laid.

Principle 3: Once and Only Once

This is an ideal form of Principle 2. If you have an array that has 10 elements in it, and you need to reference the number of elements in lots of places, then declare a constant and use that everywhere. Then you only need to change it in one place.

However, don’t get too carried away. Remember not to let this trump readability. I had a thought-provoking discussion with my colleague recently. All of our devices use millimeters, but we have to display our measurements in inches. We have lots of places where we multiply or divide by 25.4, which is the conversion factor between millimeters and inches. If you follow principle 3, then you should define a constant, e.g., MILLIMETERS_PER_INCH = 25.4 and use that everywhere. On the other hand, the conversion factor between millimeters and inches is unlikely to change anytime soon, and even if it were, you could find all the instances of 25.4 in our program and replace them in less than 60 seconds with a search and replace tool. Plus the constant is just longer. Concise is good. Furthermore, the context in which it’s used explains the value (because it’s typically used like this: distance_mm = distance_inches * 25.4). For these reasons I’m OK, in this particular case, using 25.4 instead of a named constant.

My point is that these ideas aren’t always black and white. Don’t apply these blindly and assume you’ve done your job. Make sure you use your brain too.

Principle 4: Isolate Things That Change Separately

This is the corollary of Principle 2. Just because your system has three identical pumps now doesn’t mean it’ll have three identical pumps 5 years from now. You might feel like a genius because you made a function block to control those pumps, but when the feedback on one of them malfunctions and you need to go in and bypass that one feedback without messing up the feedback logic on the other pumps, you’ve just made your life more complicated. Plus, if it’s the electrician that has to put that bypass in, how comfortable will they be modifying your function block vs. modifying a rung that only affects one pump? The best 2 am support call is the one you never get.

Principle 5: Use Patterns for Consistency

You are not the first person to program a PLC. Those who’ve come before you and learned through trial and error have settled on some useful patterns of ladder logic programming that perform specific functions in well understood ways. These are the nouns, verbs, and adjectives of our field. You can expect someone reading your logic to recognize these patterns quickly and understand what you’re doing.

You’ll also start coming up with patterns that are specific to your machine or facility. Patterns have the advantage that once you learn it, you understand it whenever you see it. Consistency is good.

Principle 6: Build a Domain-Specific Language

Whether you’re programming a machine or a family of similar machines, you’re likely to find that the same problems arise again and again. Use function blocks to create a short-hand notation for the nouns, verbs, and adjectives that are specific to the problems you’re solving. Look for repeated logic. I don’t mean repeated because the hardware repeats (because hardware changes) but look for cases where your ideas are repeated.

For a simple example, we have a lot of mechanical presses in our facility and we often want to know if a press is in a certain “window” such as from 90 to 180 degrees, or a more complicated test is from 350 to 10 degrees (because it goes through zero). I created a Window function to handle these simple tests and encapsulate the more complicated logic of when the window includes the 360 to 0 rollover point. The use of the function is readable, it’s used widely, but unlikely to change in a way that might break all the places it’s used, so it’s a good candidate for a function block.

Another good candidate is a function block that logs an event to your plant-wide event logging system.

Conclusion

Remember, readability and correctness trump everything else. Being concise is good but not at the expense of being cryptic. A simple 3-rung pattern repeated 10 times is easier to understand than a single complicated 10-rung block, even if the latter is a third the size. Ask yourself at the outset, “what’s likely to change?” and be honest with yourself. Let the answer guide your decisions. And above all, think, “why am I doing it this way?”

What is “AI” Anymore?

When I was growing up we had many examples of Artificial Intelligence (AI) in the movies. Of course we had R2-D2 and C-3PO in Star Wars, and HAL in 2001: A Space Odyssey. It was clear to anyone that these machines were actually intelligent.

These days the media is calling anything and everything “AI” with little evidence of any intelligence whatsoever. Here are some examples from the news:

An “AI” running on a $10 Raspberry Pi… to make your fridge “smart”? Give me a break! Scientists have been working on modelling what’s going on in the human brain, and according to this article:

It took 40 minutes with the combined muscle of 82,944 processors in K computer to get just 1 second of biological brain processing time. While running, the simulation ate up about 1PB of system memory as each synapse was modeled individually.

To be fair, that’s what it takes to simulate all the neurons in a human brain, and it’s not clear that this is a good analog for an artificial intelligence. Still, in 2015, the IEEE published an article saying that the human brain is 30 times faster than the world’s best supercomputers. Certainly you’re not doing that on a Raspberry PI.

We’re only scratching the surface of AI right now. “Deep Learning” is the big new buzzword. It works like this: you feed it a big dataset, like a bunch of X-ray images, and you have an expert in the field, like a radiologist, pick examples from that dataset and categorize them (“cancer”, “not cancer”). You then let the deep learning program go at the dataset and try to build a model to categorize all the images into those two groups. The expert then looks over the result and corrects any mistakes. Over time and many cycles, the software gets better and better at building a model of detecting cancer in an X-ray image.

Clearly this is pattern matching, and it’s something we humans are particularly good at. However, I’d also note that most animals are good at pattern matching. Your dog can learn to pickup subtle clues about when you’re about to take her for a walk. Even birds can learn patterns and adapt to them.

If your job can be replaced by a pattern matching algorithm, isn’t it possible that your job doesn’t require that much intelligence? It’s more likely you relied on a lot of experience. When I walk out to a machine and the operator tells me that the motor’s making a weird sound when it powers up, chances are I’ve seen that pattern before, and I might be able to fix it in a few minutes. That’s pattern matching.

We hear a lot in the media about AI coming to take our jobs, but it’s more correct to say that Automated/Artificial Experience (AE) is really what’s about to eat our lunch. Lots of highly paid professions such as medical doctors, lawyers, engineers, programmers, and technicians are in danger of deep learning systems removing a lot of the “grunt work” from their profession. That doesn’t mean the entire profession will have nothing left to do. After all, these systems aren’t truly intelligent, but we can’t hide from the fact that in large teams, some of the employees are likely only doing “grunt work.”

So don’t worry about AI just yet. Just make sure you’re using your real intelligence, and you should be safe.

“X-Year Old” is a Categorical Variable

Just a quick note to parents out there: when someone says “3-year old” it refers to any individual who is from 3 years of age to 3 years and 364 days of age. It’s a categorical variable. The average “3-year old” is 3 years and 6 months of age. All of the measured “norms” of what a 3-year old can do is based on children who fall in that age range.

So if your 3-year old is one month away from his 4th birthday, it’s not OK to refer to him as a 4-year old just because he’s “almost 4.” He’s still closer in age to the average 3-year old than the average 4-year old, so you’re inadvertently comparing him to a group of children who are older than him, and you’re naturally going to feel like he’s falling behind.

In unrelated news, my 3-year old son is almost 4. 🙂

PLC Programming goes Imperative

Decades ago, computer science emerged from the dark ages of assembly language programming and created two new languages: Lisp and Fortran. These are two very important computer languages because they exist at opposite ends of an imagined spectrum in the eyes of computer scientists: functional languages vs. imperative languages.

Fortran “won” the first battle, not least because imperative languages are closer to how the CPU actually does things, so back in the day when every little CPU cycle mattered it was easier to understand the performance implications of a Fortran program than a Lisp program. Plus, if you were already programming in assembly, then you were already thinking about how the computer was executing your code. In fact, the next big imperative language, C, is often referred to as “portable assembly language.” Fast forward to now, and modern languages like C#, Java, Python and Ruby have all grafted a lot of functional programming features onto their imperative programming basic syntax. In C#, for instance, Linq is a direct rip-off of Lisp’s S-Expressions and it now has Closures and lambda functions. Functional languages provide ways to think at a higher level than imperative languages. In a functional program you describe what you want and in an imperative program you describe how to do it.

Here’s an example in C#, using imperative programming:

var data = new int[] { 1, 2, 3, 4, 5 };
var sumOfSquares = 0;
for(var i = 0; i < data.Length; i++)
{
    sumOfSquares += data[i] * data[i];
}

…and the same thing done functionally:

var data = new int[] { 1, 2, 3, 4, 5 };
var sumOfSquares = data.Select(x => x * x).Sum();

In the second case, I’m taking the list of numbers, using Select to translate that into a list of their squares (also known as a Map operation) and then using Sum on the resulting list to compute an aggregate sum (also known as a Reduce operation). It has some interesting advantages. For instance, the original code can’t be split across multiple cores, but the latter can. Also, if you know both syntaxes, the latter is easier to read and understand.

Now take ladder logic. I’ve made the claim before that basic ladder logic (with contacts and coils) is actually a functional language. A simple example might be ANDing two inputs to get an output, which in C# would look like this:

var output = inputA && inputB;

That’s actually functional. If I wanted to write it imperatively I’d have to do something like:

var output = false;
if(inputA && inputB)
{
  output = true;
}

In ladder logic, that would be the equivalent of using an unlatch (or reset) instruction to turn off an output and then using a latch (or set) instruction to turn on the output if the A and B contacts were true. Clearly that’s not considered “good” ladder logic.

Similarly, a start/stop circuit goes like this:

var run = (start || run) && !stop;

Now historically, mathematicians and physicists preferred functional languages because they just wanted to describe what they wanted, not how to do it. It’s worth noting that electricians, looking at ladder logic, prefer to see functional logic (with contacts and coils) rather than imperative logic (with sets, resets, and move instructions).

In recent years we’ve seen all major PLC brands start to include the full set of IEC-61131-3 languages, and the most popular alternative to ladder logic is structured text. Now that it’s available, there are a lot of newer automation programmers who only ever knew imperative programming and never took the time to learn ladder logic properly, and they just start writing all of their logic in structured text. That’s why we’re seeing automation programming slowly shift away from the functional language (ladder) towards the imperative language (structured text).

Now I’m not suggesting that structured text is bad. I prefer to have more tools at my disposal, and there are definitely times when structured text is the correct choice for automation programming. However, I’d like to point out that the history of computer science has been a progressive shift away from Fortran-like imperative languages towards Lisp-like functional languages. At the same time, we’re seeing automation programming move in the opposite direction, and I think alarm bells should be going off.

It’s up to each of us to make an intelligent decision about what language to choose. In that respect, I want everyone to think about how your brain is working when you program in an imperative style vs. a functional style.

When you’re doing imperative programming, you’re holding a model of the computer in your mind, with its memory locations and CPU and you’re “playing computer” in your head, simulating the effect of each instruction on the overall state of the CPU and memory. It’s only your intimate knowledge of how computers work that actually allows you to do this, and it’s the average electrician’s inability to do this which makes them dislike structured text, sets, resets, and move instruction. They know how relays work, and they don’t know how CPUs work.

If you know how CPUs work, then I understand why you want to use structured text for everything. However, if you want electricians to read your logic, then you can’t wish-away the fact that they aren’t going to “get” it.

As always, be honest with yourself about who will read your logic, and choose your implementation appropriately.

Why good ladder logic looks like it was written by an 8 year old

When traditional PC programmers see ladder logic, they think ladder logic programmers are terrible programmers. Being both a .NET developer and a ladder logic programmer, this has caused me a lot of frustration and confusion over the years. I have one foot in each world, and yet I choose to write C# programs one way and ladder logic programs another. Why?

Let’s ignore the fact that most traditional programmers just don’t grok ladder logic at all, because their minds think about programs sequentially rather than in parallel. The real reason they hate ladder logic is because ladder logic programmers avoid things like loops, indexed addressing and subroutines. To them, this means you’re programming at the level of an 8 year old.

The thing is, I know how to use loops, arrays, and subroutines, not to mention object oriented programming and functional programming constructs like s-expressions, closures and delegates. Still, I choose to write simple and straightforward ladder logic. Why would I, an experienced programmer, choose to write programs like an 8 year old? Do I know something they don’t?

I spend a lot of time trying to get people to think about why they do things a certain way. Everyone wants that simple rule of thumb, but it’s far more valuable to understand the first principles so you can apply that rule intelligently. Decades of computer science has given us some amazing tools. Unfortunately, a carpenter with twice as many tools in her tool box is simply twice as likely to pick the wrong tool for the job if she doesn’t understand the problem the person who invented that tool was trying to solve.

The first time you show a new programmer a “for” loop, they think, “Amazing! Instead of typing the same line out a hundred times, I can just type 3 lines and the computer does the same thing! I can save so much typing!” They think this because they’re still an idiot. Don’t get me wrong, I was an idiot about this too, and I’m still an idiot about most things. What I do know, however, is that for loops solve a much more important problem than saving you keystrokes. For loops are one of many tools for following the Once and Only Once (OAOO) Principle of software development.

The OAOO principle focuses on removing duplication from software. This is one of the most fundamental principles of software development, to the point where it’s followed religiously. This principle is why PC programmers look at ladder logic and instantly feel disgust. Ladder logic is full of duplication. I mean, insanely full of duplication. So how can you blame them? God said, “let there not be duplication in software,” and ladder logic is full of duplication, thus ladder logic is the spawn of Satan.

That’s because programmers who believe the OAOO principle is about removing duplication are idiots too. Don’t they ever wonder, “why is it so important to remove duplication from our code?” Should we really worry about saving a few bytes or keystrokes? NO! We focus on:

  1. Making it do what it’s supposed to do
  2. Making it obvious to the reader what the program does
  3. Making it easy to make changes when the requirements change

… in that order.

In fact, #3 is the real kicker. First of all, satisfying #3 implies you must have satisfied #2, so ease of understanding is doubly important, and secondly, satisfying #3 implies you can predict what will change.

Imagine if you have to print the numbers from 1 to 5. If I asked a C# programmer to write this, they’d likely write something like this:

for(var i = 1; i <= 5; i++)
{
    Console.WriteLine("{0}", i);
}

… of course I could write this:

Console.WriteLine("1");
Console.WriteLine("2");
Console.WriteLine("3");
Console.WriteLine("4");
Console.WriteLine("5");

Why is the first way better? Is it because it uses fewer keystrokes? No. To answer this question, you need to know how the requirements of this piece of code might change in the future. The for loop is better because many things that might change are only expressed once. For instance:

  • The starting number (1)
  • The ending number (5)
  • What to repeat (write something to the screen)
  • What number to print
  • How to format the number it prints

If the requirements of any of these things change, it’s easy to change the software to meet the new requirements in the first case. If you wanted to change the code so it prints every number with one decimal place, the second way clearly requires 5 changes, where the first way only requires one change.

However, what if the requirements changed like this: print the numbers from 1 to 5, but for the number 2, spell out the number instead of printing the digit.

Okay, so here’s the first way:

for(var i = 1; i <= 5; i++)
{
    if(i == 2)
    {
        Console.WriteLine("two");
    }
    else
    {
        Console.WriteLine("{0}", i);
    }
}

… or if you wanted to be more concise (but not much more readable):

for(var i = 1; i <= 5; i++)
{
    Console.WriteLine(i == 2 ? "two" : i.ToString());
}

Here’s the change using the second way:

Console.WriteLine("1");
Console.WriteLine("two");
Console.WriteLine("3");
Console.WriteLine("4");
Console.WriteLine("5");

Here’s the thing… given the new requirements, the second way is actually more readable and more clearly highlights the “weirdness”. Does the code do what it’s supposed to do? Yes. Can you understand what it does? Yes. Would you be able to easily make changes to it in the future? Well, that depends what the changes are…

Now think about some real-life ladder logic examples. Let’s say you have a machine with some pumps… maybe a coolant pump and an oil pump. Your programmer mind immediately starts listing off the things that these pumps have in common… both have motor starters with an overload, and both likely have a pressure switch, and we might have filters with sensors to detect if the filters need changing, etc. Clearly we should just make a generic “pump” function block that can control both and use it twice, right?

NO!

Look, I admit that there might be some advantage to this approach during the design phase if you had a system with 25 identical coolant pumps and your purchasing guy says, “Hey, they don’t have the MCP-1250 model in stock so it’s going to be 8 weeks lead time, but they have the newer model 2100 in stock and he can give them to us for the same price.” Maybe it turns out the 2100 model has two extra sensors you have to monitor so having a common function block means it takes you… 20 minutes to make this change instead of an hour. We all know how much you hate repetitive typing and clicking.

On the other hand, when this system goes live, making an identical change to every single pump at exactly the same time is very rare. In fact, it’s so rare that it’s effectively never. And even if that were to ever actually happen, the amount of programming time it actually saves you is so tiny compared to the labor cost of actually physically modifying all those pumps that it’s effectively zero.

However, since these are physically different pumps, you’re very likely to have a problem with one pump. When your machine is down and you’re trying to troubleshoot that pump, do you want to be reading through some generic function block that’s got complicated conditional code in it for controlling all 50 different types of pumps you’ve ever used in your facility, or do you want to look at code that’s specific to that pump? And maybe the motor overload on that pump is acting up and you need to put a temporary bypass in to override that fault. Do you really want to modify a common function block that affects all the other pumps, or do you want to modify the logic that only deals with this one pump? What’s more likely to cause unintended consequences?

So this is why ladder logic written by experienced automation programmers looks like it was written by an 8 year old who just started learning Visual Basic .NET last week. Because it’s better and we actually know why.