Over the past year I’ve been working on a TwinCAT 3 Tutorial, and with the posting of the 13th chapter about the TwinCAT 3 Scope View, I’m considering it “complete.” I’ve covered all the major topics that I wanted to cover.
Writing a tutorial like this was actually a learning experience for myself. Before I made any statement about the software I tried to validate my assumptions first, and many times I learned that I had preconceived notions about how it worked which may not have been completely accurate. If you’re a TwinCAT 3 programmer, then you need to know your tool set inside and out. If you’re not sure how something works, it’s best to setup a quick test and try it out. That’s one of the advantages of PC-based software: if you want to run a test, your office PC is a laboratory environment. Write some code and run it.
Thank you to everyone who offered feedback on the existing sections, and to those of you who kept pushing me to continue writing more. I sincerely hope this can be that missing link to take people from the world of traditional PLCs into the more interesting and ultimately more powerful world of PC-based control.
It looks like TwinCAT 3 has a viable future ahead of it. Interest seems to be growing over time (according to Google Trends, anyway):
That makes TwinCAT 3 a valuable skill to learn if you’re an integrator, and an interesting technology to investigate for existing plants.
Staring down the barrel of a big automation programming project is intimidating. It’s hard to even know where to start. Even when you’ve done a few before, you’re only marginally more confident the next time.
I have quite a few big automation programming projects under my belt, so I think I can generalize the process a bit. Here goes:
1. Get the Prints
There’s almost no point in starting to program unless you have an almost final set of electrical drawings. If you don’t have them yet, push for them, and go do something else productive until you get them.
2. Create a Functional Specification
You don’t always have to write out a functional specification, but it at least needs to exist very clearly in your head. If at any point, you don’t know exactly how the machine is supposed to work in excruciating detail, stop writing code and go figure it out. Ask stakeholders, talk to operators, whatever it takes. Functional specifications are best written as a list of “user stories”. If you’re not sure what a functional spec should look like, check out Painless Functional Specifications by Joel Spolsky.
3. Shamelessly Copy
Identify what other projects you can find with logic that you can steal. Any code that works in another machine has the advantage of already being debugged. Don’t re-invent the wheel. (At the same time, never blindly copy logic without understanding it. Copying the code by re-typing it one rung at a time is still faster than writing it from scratch, and it’s a form of software review.)
4. Structure Your Project
Now you break open the ladder logic programming software and start creating your project. Pick your CPU type, setup the I/O cards based on the electrical drawings. Map your inputs. Plan out your program by creating programs or routines for each functional unit of the machine. Setup your fault summary rungs and your alarm logic.
5. Write the Manual Mode Logic
PLC logic is typically written “bottom up.” Manual mode logic is the lowest level of logic because it deals directly with individual functions in the machine. Advance cylinder. Retract cylinder. Home axis. Jog axis. While you’re writing the manual mode, this is when you take extreme care making sure that actions are interlocked so the machine can’t crash. If you’re using the Five Rung Pattern, this means paying attention to what goes in the Safe rung. Does cylinder A always have to advance before cylinder B can advance? The Safe rungs should reflect that. Make sure that even in manual mode, the operator (or you) can’t break the machine. Make sure to hook your faults and alarms into the applicable fault summary rungs and alarm logic.
6. Write Part Tracking Logic
Now that manual mode is complete, write the logic that tracks parts (and their state) through the machine. Remember, you should be able to run the machine in manual mode, and the part tracking should (ideally) work correctly. I know this isn’t always the case but surprisingly part tracking in manual mode can work 95% of the time. That means part tracking works based on the state of the machine. Closing the gripper with the robot in the pick position and the part present in fixture sensor on should latch a bit “remembering” that the gripper has a part in it.
Once you’ve written your part tracking logic, go back and use the part tracking and state bits to condition your Safe rungs. Don’t let the operator (or you) mistakenly open the gripper if the gripper has a part and isn’t in a safe position to let go of the part. Of course, you may need to add a way to manually override this (that’s what output forcing was created for), but in most cases you want to prevent improper operation.
Part of writing the part tracking logic is adding “ghost buster” screens. Operators often need to remove parts from a cell, and if the machine can’t detect their removal, then you have to provide the operator with a way to clear these “ghosts.”
At this point you’re actually ready to dump the program in and start testing out the machine electrically and mechanically. While it’s ideal to have a fairly complete program when you go onsite, we all know that’s not always entirely possible. At the very least you want to get to this point before startup begins.
7. Write the Auto Mode Logic
The complexity of your auto mode logic depends on what type of machine you’re programming. You’ll always need a cycle start and a cycle stop feature. Even if you’re in auto mode, you usually don’t want the machine to start until the operator specifically tells it to start. Once it’s running, we call this “in auto cycle.”
In simple machines, you can write the auto logic by filling in the Trigger rungs in your Five Rung logic. Start by putting the In Cycle contact at the beginning of the rung, and then writing logic after that which expresses when the action should take place. For instance, an advance reject part cylinder’s Trigger rung could be as simple as In Cycle, Part Present, and Part Rejected. As long as the Part Present tracking bit gets cleared once the cylinder is in the advanced position, then this is all the auto mode logic you need for this motion. Have the retract Trigger rung be In Cycle, No Part Present and Not Retracted.
More complicated machines need more complicated auto mode logic. If your machine has to perform a series of steps (even if some of them are in parallel) then consider using the Step Pattern. If your machine needs to choose between several possible courses of action (commonly seen in a storage and retrieval system) then consider using the Mission Pattern.
It’s hard to write correct logic. Review your functional specification, point by point, and make sure your logic meets all of the requirements. Check your logic for errors. A fresh look often uncovers incorrect assumptions, typos, and outright mistakes. The earlier you find and fix problems, the easier they are to fix.
Make a list of everything you have to do during startup. Starting up a machine is time consuming and therefore expensive. Anything you can do to prepare saves you time and money.
Good luck, and keep your fingers out of the pinch points!
The recent drop in the world price of oil has caused a similar drop in the value of the Canadian dollar. Much of that has to do with the huge oil production in Alberta, so that province is being hit particularly hard right now. The “oil sands” in Alberta are particularly expensive to exploit, as oil sources go, so companies there are quick to make cuts when the price drops below their break-even point.
The previous run-up in oil prices saw a lot of migration out of manufacturing-heavy Ontario westward to Alberta, and control system specialists were a big part of that. It’s difficult to see them facing these challenges, and none of us want to be in that position.
Back here in Ontario, the pendulum looks like it’s swinging the other way. When rising oil prices drive up the Canadian dollar, it makes it a lot harder for Canadian manufacturers to compete in the export market, because their products are automatically more expensive. The recent drop in the Canadian dollar, if it persists, is going to mean growth in the manufacturing sector, especially along the 401 corridor in Ontario. Personally I’m looking forward to seeing some growth in what has been a relatively stagnant market over the past 10+ years.
Automation is poised to be a big part of the new growth in Ontario. At first companies are likely to be cautious about adding new capacity, likely adding temporary labour, but if the trend holds, I think we’re going to see lots of interest in automation projects throughout the area. Companies are still a bit leery of the economy, so they’ll be looking for flexible automation that can adapt to changes in demand and re-tool quickly. Technologies that can support flexibility are going to be winners.
While disappointing for Alberta, I’m excited to see what the future holds for Ontario.
If you do happen to be out in Alberta and you have discrete automation programming experience, and you’re interested in moving back to Ontario, make sure you drop me a line. We’ve experienced lots of growth ourselves in recent years, so we’re interested in hiring an experienced automation programmer for our Electrical Engineer position. We use TwinCAT 3 and C# and mentoring is available to get you up to speed with these technologies. Even if you’re local and you’re tired of the endless travel or 2 am service calls, then maybe this family-friendly company is right for you.
One thing I’ve discovered about automation blogging is that it’s a pretty lonely place. Don’t get me wrong, there are a couple gems out there, but I don’t find many people writing about what it’s like in the trenches wading through rungs of ladder logic. In the .NET world there are tons of programming blogs with posts about every issue you could ever come across. Why such a dearth of information in the automation space?
It occurs to me that blogging seems difficult to a lot of people, as if you need to be a web programmer. That’s totally untrue (I’m absolutely not a web programmer). In fact there are a ton of inexpensive and simple options out there.
Maybe people wonder what to write about. That’s easy. At first I worried about writing something people didn’t already know, but it turns out there’s no shortage of shiny new graduates looking for any toe-hold they can get in this industry. Try to think back to when you didn’t know anything. How did you figure out how to get online with a PLC the first time? Did you have to call the office to ask someone, and feel foolish because you didn’t know what a Null Modem cable was? We were used to asking our peers for help, but the new generation grew up looking up how to do things on the web.
Maybe you’re worried about the cost. Shared hosting plans are very inexpensive. I use DreamHost, specifically because they’re inexpensive (starts at $9 per month including domain name, unlimited storage and bandwidth), the hosting is rock solid, they offer one-click installs of blogging software (such as WordPress), and their technical support is excellent.
It would really be great if I didn’t feel like the only voice in this cloud. Grab a blogging account and chime in. Write about a hard problem and how you solved it. Disagree with me! Help me learn something new!
As a PLC programmer, you’ll often be asked to do a change to an existing system. If there’s a significant amount of functionality to be added, you generally get your changes ready “offline” and then do all the changes during a short window of time to minimize disruption to the production schedule.
If you’re using an Allen-Bradley PLC, the procedure is typically this:
- Get a copy of the latest program from the PLC (a.k.a. an “upload”)
- Make your changes to the offline copy, and write down every change you had to make
- Go online with the PLC and apply your changes as online changes
Step 3 is much safer than just taking your modified program and doing a “download”. That’s mainly because when you download, you’re not just downloading the program, but the memory state of the PLC as well. The PLC typically has to track things in memory (like recipe data, part tracking, data collection, sequence numbers, machine counters, etc.). If you do a download, you’re going to overwrite all those values with previous values, and that can cause a lot of problems. The other thing step 3 saves you from is simultaneous changes that were done online while you were busy making offline changes.
The only other option you have is upload-change-download, but you really have to shut the machine down for the duration to make sure that the internal state doesn’t change.
When I did a lot of Allen-Bradley programming, I didn’t question that. It’s just how it was. I remember visiting a plant one time for a service call, and the local maintenance person was a bit suspicious of what I was going to do (after all, I was a young kid who had never seen this machine before). He decided to quiz me a bit, and one of the things he asked was, “when you go online, do you download or upload?” I said “it depends,” but his answer was, “you never download.” I agreed that someone in a maintenance role should never need to do a download unless they’re replacing a CPU, or recovering from a corrupted PLC program.
Now that I mostly do Beckhoff TwinCAT 3 programming, I realized one of the benefits are that offline changes are a breeze. It’s due to the fact that TwinCAT 3 completely separates the program from the memory data. The program is stored in local files on your hard drive and compiled into a TMC file. The persistent data is stored in a different place on your hard drive.
When I want to do offline changes to a TwinCAT 3 project, here’s the procedure:
- Get a copy of the latest program
- Make your changes to the offline copy
- Copy changes back to the machine (keeping a backup, of course), rebuild, and activate configuration
This makes offline changes go a lot more smoothly, of course. I don’t have to copy and paste my changes in while online, so it takes less time and eliminates the possibility of a copy/paste error.
Since we also use Mercurial for version control, getting a copy of the latest program is a matter of pulling the latest from the source control, and copying it to the machine is a matter of pulling the offline changes to the machine. Any changes that were done in parallel can be merged with Mercurial’s built-in diff and merge utilities. (Note: I/O changes can’t be merged nicely, so if someone changed the I/O while you were doing your offline changes, you have to copy those changes in manually, but that’s rare and at least it tells you that it can’t merge them.)
This got me thinking that Allen-Bradley probably has a better way of doing offline changes that most of us just don’t know about. I know that you can do an upload without uploading the memory. However, it seems like it requires you to download both the program and data at the same time. I wonder if anyone out there knows how to do better offline changes to a ControlLogix. If so, I would be interested to know that.
If you don’t have kids, I think I have an insight that you may find interesting.
It’s funny, but before we had kids, it’s amazing how much less housework got done. I remember the dishwasher didn’t get unloaded in a timely manner, or the floors didn’t get vacuumed as often as they should. The laundry didn’t get folded. The weird thing is, now that we have three kids, we’re way more on top of that stuff. Yet we have a lot less time. How is that possible?
There are a lot of things in our lives that distract us. The big time suckers have screens: smart phones are the new offenders, but before that it was the internet, video games, and of course television. Personally I’ve gone to excessive lengths to curtail these distractions, particularly at work. I leave my phone in airplane mode about 95% of the time. Email notifications are always turned off. I use a browser add-in that blocks my favorite websites except at specific times of the day, or blocks them after a specific number of minutes used each day. Overall, these small barriers to distraction really do make a difference, but there’s actually another much more potent barrier to screen distractions: young children!
Whether it’s one child or three, they demand attention almost constantly. You can’t get lost for 10 minutes on your phone if your two-year-old interrupts you every 90 seconds. Parents never “surf” the web, they go to a website, lookup whatever it is they need, and slam the lid on the laptop before the sticky fingers come around. God forbid you get the idea to play a video game… you’ll never get past the loading screen before you have to attend to some crisis.
When kids are around, you can’t do anything that requires concentration and focus (a.k.a. flow) because the constant interruptions prevent you from making progress. On the other hand, if you focus 100% on your kids, you’ll drive yourself insane. They’re always doing something that makes you slightly nervous, like playing too close to a lamp, crawling over the back of the sofa, or jumping on their bed (no matter how many times you’ve warned them). In fact, so called “helicopter parents” are just the parents that spend all their time focused on what their kids are doing. The rest of us realize that we need to ignore most of it, and just keep one ear open for the real dangerous stuff.
So as parents we’re left with having to do something that doesn’t require our full attention. Do you know what kinds of things are ideal? Housework. Laundry. Tidying. Cleaning. Taking out the garbage. These are the kinds of tasks that can be done with half your brain. It doesn’t matter if you get interrupted while you’re folding laundry, because you can pick it back up almost instantly. It takes just enough of our attention to keep us from actively focusing on the kids, but your subconscious still alerts you when they get (a) too noisy, or (b) too quiet. Best of all, it gives you a much needed feeling of productivity. Yes, housework is actually a coping mechanism for parents to keep their sanity.
So if you don’t have kids, and you wonder why you’re not the productivity powerhouse that you remember your Mom being when you grew up, don’t worry… she probably wasn’t that on top of things until she had you!
Some of you may have noticed the new section on this site: TwinCAT 3 Tutorial.
I’ll be building this over the next several weeks or months. I’m working on making it more detailed than the RSLogix 5000 Tutorial.
Rather than being an introduction to PLCs, I assume most readers are coming to the new tutorial with some automation experience and they really want to know what this new technology can do for them. I won’t be touching on every single feature of TwinCAT 3, but I certainly want to touch on most of the common ones, and particularly some advanced features that really set TwinCAT 3 apart from traditional PLCs.
As always I greatly appreciate any comments you have. Please send them to my email address (which you can find on the About page).
This information is for a Fanuc R30-iA, RJ3-iA or RJ3-iB controller but might work with other ones.
If you’re looking for a way to send the robot world TCP position (X, Y, Z, W, P, R) over to the PLC, it’s not actually that difficult. The robot can make the current joint and world position available in variables, and you can copy them to a group output in a background logic task. There is one caveat though: the values only update when you’re running a program. They don’t update while jogging. However, there is a work-around for this too.
First you should make sure that the feature to copy the position to variables is enabled. To get to the variables screen, use MENU, 0 (Next), 6 (System), F1 (Type), Variables.
Find this variable and set it to 1 (or True):
The name of that variable is Current position from machine pulse
Now create a new robot program, and in it write the following:
Note that I’ve multiplied the X, Y, and Z positions by 10, so you will have to divide by 10 in your PLC. Likewise I multiplied the W, P, and R angles by 100, so divide by 100 in the PLC.
To run this program in the background, use MENU, 6 (Setup), F1 (Type), 0 (Next), BG Logic. Configure it to run your new program as a background task.
Obviously you need to send these group outputs to the PLC. Ethernet/IP is great for this, but you can use hardwired interlocks too. You need to make sure that you have enough bits to handle the full range of motion. A 16-bit integer should work pretty well for all of these. Note that the robot will happily send negative numbers to a group output as two’s complement, so make sure you map the input to the PLC as a signed 16-bit integer (a.k.a. INT in most PLCs). For the X, Y, and Z positions, a 16-bit integer will give you from +3276.7 mm to -3276.8 mm of total range. For the W, P, and R angles you’ll get +327.67 deg to -327.68 deg. For most applications this is good (remember this is TCP, not joint angles). Please check that these are suitable though.
As I said, these numbers don’t update while you’re jogging, and won’t update until the robot starts a move in a program. One little trick is to do a move to the current position at the start of your program:
J PR[100:SCRATCH] 10% FINE
This starts sending the position without moving the robot. In my programs I typically enter a loop waiting for an input from the PLC, and inside this loop I turn a DO bit on and off. The PLC detects this as a “ready for command” heartbeat, and as long as the PLC sees this pulsing, then it knows the program is running and the position data is valid.
Another trick you can use is to detect when the robot has been jogged:
The name of this variable is Robot jogged. The description from the manual is: “When set to TRUE,the robot has been jogged since the last program motion. Execution of any user program will reset the flag.”
That’s how you get the world position of the TCP into the PLC. If you just want joint angles, you can use
$SCR_GRP.$MCH_ANG[n] as the variable, where “n” is the joint number.
Important note: The I/O will probably change asynchronously to the program scan, so what you want to do is make a copy of the X, Y, Z, W, P, R values coming into the PLC and compare the current values to the values from the last scan. If they haven’t changed, then update your actual values, otherwise throw them away because they might not be valid. If you have a fast scanning PLC and I/O then you should still be able to keep up with the robot even during a fast move. If you have a slow scan time on your PLC, then you might only get valid stable values when the robot is stopped.
Now what if you want to know what the TCP position is relative to one of your user frames? The robot controller doesn’t seem to give you access to this, but the PLC can at least calculate the X, Y, and Z positions of the TCP in your user frame itself, given the world position and the user frame parameters.
First you need to find the accurate user frame parameters. Under the normal frames screen you can only get one decimal point of accuracy, but you need the full 3 decimal points to have your numbers in the PLC match the user frame position given in the robot. You can find these accurate positions in a variable: use MENU – 0,6 (SYSTEM) – F1 (TYPE) – Variables – $MNUFRAME[1,9] – F2 (DETAIL). The second index in square bracket is the frame number, so
$MNUFRAME[1,1] is frame one and
$MNUFRAME[1,2] is frame 2. Copy these numbers down exactly.
Here’s the math for calculating the TCP relative to your user frame. All variables are LREAL (which is a 64-bit floating point variable). I don’t know if you can use a regular 32-bit float or not.
Result is your TCP in user frame.
Point is your point in world frame (from the robot) and
Frame is the accurate user frame data you copied from the
Result.X_mm := Point.X_mm - Frame.X_mm;
Result.Y_mm := Point.Y_mm - Frame.Y_mm;
Result.Z_mm := Point.Z_mm - Frame.Z_mm;
RadiansW := DegreesToRadians(-Frame.W_deg);
CosOfAngleW := COS(RadiansW);
SinOfAngleW := SIN(RadiansW);
RadiansP := DegreesToRadians(-Frame.P_deg);
CosOfAngleP := COS(RadiansP);
SinOfAngleP := SIN(RadiansP);
RadiansR := DegreesToRadians(-Frame.R_deg);
CosOfAngleR := COS(RadiansR);
SinOfAngleR := SIN(RadiansR);
// Fanuc applies rotations WPR as W (around Z), P (around Y), R (around X)
// AROUND Z
temp := Result.X_mm;
Result.X_mm := Result.X_mm * CosOfAngleR - Result.Y_mm * SinOfAngleR;
Result.Y_mm := Result.Y_mm * CosOfAngleR + temp * SinOfAngleR;
// AROUND Y
temp := Result.Z_mm;
Result.Z_mm := Result.Z_mm * CosOfAngleP - Result.X_mm * SinOfAngleP;
Result.X_mm := Result.X_mm * CosOfAngleP + temp * SinOfAngleP;
// AROUND X
temp := Result.Y_mm;
Result.Y_mm := Result.Y_mm * CosOfAngleW - Result.Z_mm * SinOfAngleW;
Result.Z_mm := Result.Z_mm * CosOfAngleW + temp * SinOfAngleW;
DegreesToRadians() is just
Run that on your PLC and check that the values in your Result variable match the user frame TCP position reported on the teach pendant.
I haven’t gotten around to calculating the W, P, and R angles of the TCP in user frame yet. Currently I just look at W, P, and R in world frame if I need to know if I’m “pointed at” something. If you get the math to work for W, P, and R, I’d really appreciate if you could share it.
In recent years I’ve been interested in the effects of automation on our economy and our society. Throughout history every advance in technology has brought more wealth, health, and opportunity to pretty much everyone. With every revolution people changed jobs but their lives got significantly better. When farms mechanized, workers moved into the city and got factory jobs, and Henry Ford’s assembly lines made use of this labor to great effect.
Early factories needed labor in great quantities and as industrial processes became more efficient at utilizing labor, the value of human labor rose and the demand kept increasing. So did pay. Factory workers up through the 70’s could afford a nice house to raise a family, a big car, and even a boat or nice vacations. Since the 70’s however, the purchasing power of a factory worker or even a bank teller has been pretty flat. These are two professions that have seen the most advances in automation in the last 30 years, due to industrial robots and automated tellers. If automation makes workers more productive, why aren’t we seeing that translate into purchasing power?
There are two types of technological improvements at work here. A farmer with a tractor is very productive compared to one with a horse and plow. The displaced farm workers who went to the city were given the tools of the industrial revolution: steam engines, motors, pumps, hydraulics, and so forth. These technologies amplified the value of human labor. That’s the first kind of technological improvement. The second kind is the automated teller or the welding robot. The older technology adds value even to the lowest skilled employees, but the new technology is reducing their value and the new jobs require significantly higher skill levels. There’s something about this new revolution that’s just… different. The demand for low skill labor is drying up.
The increasing divide between the “haves” and the “have-nots” has been documented extensively. Some divide is good and promotes the economy and productivity. Too much separation is a recipe for significant problems.
I’m not the only one worrying about this issue, and as I’ve followed it over the last few years I’ve been surprised by the amount of interest in a Guaranteed Minimum Income or some such plan. Basically it involves getting rid of every low-income assistance plan such as social security, welfare, minimum wage laws, etc., and creating a single universal monthly benefit that everyone is entitled to. Some people are talking about a number as high as $24,000 per year per adult. Considering that the 2015 federal poverty level in the US is just below $12,000 for a single adult, you can see that $24,000 per adult isn’t just a trifling amount.
For comparison, a little Googling tells me that the US GDP per capita is around $55,000. Think about that for a second. You’re talking about guaranteeing almost 45% of the productivity output of the country to be distributed evenly across all adults. One presumes you would also provide some extra money per child in a household, but to be fair the “per capita” figure includes kids too. It’s possible. Sure seems a bit crazy though.
Is it practical? Won’t some people choose not to work? Will productivity go down? It turns out that we’ve done some experimenting with this type of program in Canada called MINCOME. The results were generally positive. There was a small drop in hours worked by certain people, mostly new mothers and teenagers. These costs were offset in other areas: “in the period that Mincome was administered, hospital visits dropped 8.5 percent, with fewer incidents of work-related injuries, and fewer emergency room visits from car accidents and domestic abuse.” More teenagers graduated. There was less mental illness.
I’m fiscally conservative, but I’m mostly pragmatic. It’s only my years of exposure to automation, technology and working in factories that makes me ask these questions. Not only do I believe that people should contribute, I believe that people need to contribute for their own happiness and well-being. That’s why I don’t think paying people to sit at home is the ultimate solution.
The elephant in the room is this: as technology improves, a greater proportion of the population will simply be unemployable. There, I said it. I know it’s a disturbing thought. Our society is structured around the opposite of that idea. Men are particularly under pressure to work. The majority of the status afforded to men in our society comes from their earning potential. The social pressure would still be there to work, even as a supplement to a guaranteed minimum income, so we still need to find something for those people to do. Perhaps if we expand the accepted role of men in society then we can fill that need with volunteer work. Maybe.
What’s the right answer? I don’t know. For lack of a better term, the “American Dream” was accessible to anyone if you were willing to work hard and reinvest that effort into yourself. Not everyone did that, but many people created significant fortunes for themselves after starting in the stockroom and working their way up. That security gave people a willingness to take risks and be entrepreneurial. Proponents of the idea say that a minimum income would bring back that innovative edge. Entrepreneurs could try new ideas repeatedly until they found one that worked, and not worry about their family starving. With your basic necessities met, you could start to realize your potential..
I do know that as we continue down this road of increasing automation, we can’t be leaving a greater and greater proportion of the populace without the basic resources they need to survive. Do we expect them to grow their own food? On what land? Do we expect them to do a job that I could program a robot to do, if the robot’s average cost is only $10,000/year? Do you have some valuable job we can retrain them to do? One that pays enough to support a family?
Look, I don’t like the alternatives either, but it’s better than an armed revolt.
This Christmas I asked Santa for a Brultech ECM-1240 whole home energy monitoring system, specifically the DUO-100 package. After reviewing various products, I really liked that this one was priced quite reasonably for the hardware, and that they published the communication protocol.
You (or a licensed electrian) install the ECM-1240 at your main electrical panel. Each ECM-1240 has 7 channels. In my case, the first channel on the first unit measures the incoming main line current. For the other 13 channels you are free to choose various circuits from your panel that you want to monitor. You can gang various circuits together into a single channel if you like (all lighting loads, for example). The device uses current transformers on each circuit for the monitoring. These are installed inside the panel. The hot wire coming out of each breaker has to go through a current transformer, so this isn’t a simple plug-in installation; there is wiring to be done.
I’m not a fan of sending my home energy data to someone else’s server. Apart from being a huge privacy concern (it’s pretty easy to see when you’re home, and when you went to bed), I don’t want to pay a monthly fee, and I don’t want to worry about how to get my data from their server if they go out of business or decide to discontinue their product. For those reasons, I installed their software. It’s basically a flash website that I hosted on our Windows Home Server, which is conveniently the computer hooked up to the ECM-1240’s as well.
At first their software worked quite well, but over time it started to show problems. After just less than 2 months of logging, the flash program was so sluggish to load a page that it took over 2 minutes to refresh a screen. Admittedly I was running it on an older PC. Additionally, in that amount of time it had already logged about 680 MB of data. That seemed excessive. It also logged to a SQL Lite database (a single-user file-based database), and unfortunately kept the database file locked all the time. The website would end up locking the database and packets started getting buffered in the logging software until you closed down the website and released the lock.
I decided I’d just write my own software:
I’m a C# developer, so that was my language of choice. If you’re a .NET developer too, I’ll include a link to my source code at the end of this post. Note that this software isn’t commercial quality. It’s semi-configurable (channel names, and so on) but it assumes you have a 14 channel system where the first channel is the main panel input. If your system is different, you’ll have to make some modifications.
These devices I purchased use a serial port for communication, but they give you an RS232 splitter cable so you only have to use up one serial port on your PC. Their software relies on putting the device into an automatic send mode… the device itself chooses when to send packets. If the load changes significantly on one of the circuits, it triggers an immediate packet send. Unfortunately with 2 devices on the same RS232 port, you can sometimes get a collision. Their software deals with this by detecting it and ignoring the data, but I had a sneaking suspicion that once in a rare while, it ended up getting a corrupt packet, and there was at least one point where their software logged an extremely high energy reading, and it didn’t make any sense. Therefore I wrote my software to use a polling mode. It requests data from the first device, waits about 5 seconds, requests it from the other device, waits 5 seconds, and repeats. On average we get one reading about every 10 seconds from each device. Testing seemed to indicate this was about as fast as I could go because there was a reset period after you talked to one device where you had to wait for it to time out before you could address the other one. In retrospect, if you just hooked them up to 2 separate serial ports you could probably poll the data faster.
Next I had to choose how to log the data. I really wanted the data format to be compact, but I still wanted decently small time periods for each “time slice”. I also didn’t want to be locking the file constantly, so I just wanted to be able to write the data for each time slice, and be done with it. I settled on a binary format with fixed length records. Here’s how it works: each day’s data is stored in a separate file, so the data for April 1st is stored in a file called 2015-04-01.dat. The device generates values in watt-seconds. I calculated that at a maximum current of 200A (100A panel x 2 legs) x 120V, then in a 10 second time slice I should log a maximum of 240,000 watt-seconds. A 20-bit number (2.5 bytes) can store a maximum value of 1,048,575. I didn’t want to go smaller than half-byte increments. So, 14 channels at 2.5 bytes per channel gave me 35 bytes for each 10 second time slice, and since the devices generate voltage I figured I’d log the voltage as a 36’s byte just to make it an even number. Using those numbers, running 24 hours a day for 365 days a year, this will use up under 110 MB/year. Not bad. A flat binary file like this is also fast for data access. The data is small, and seeking a time slice can be done in O(1) time. After you find the first time slice you just keep reading bytes sequentially until you get to the end of the file or your last time slice. Hopefully no more 2 minute load times.
I broke up my application into multiple programs. The first is a utility program called Uploader.exe. This is a command line program that reads the data from a device and just spits it out to the screen. Useful for testing if it works.
The second is called Logger.exe and it does just that. It uses Uploader.exe to read values from the two ECM-1240 units at 5 second offsets and writes the data to XML files into a PacketLog folder. Each file has the timestamp of when it was read, and the device number to indicate which device it came from.
The third program is called PacketProcessor.exe. This program monitors the PacketLog folder and waits until it has at least one packet from each device after the end of the current time slice it’s trying to build. If it does, it calculates the values for the current time slice and appends it to the end of the current day’s data file. It then deletes any unnecessary packets from the PacketLog folder (it keeps at most one packet from each device before the end of the most recently written time slice).
To host Logger.exe and PacketProcessor.exe, I used nssm a.k.a. “the Non-Sucking Service Manager”. It’s a small command line program you can use to run another command line program as a service. Very handy.
The fourth program is called Viewer.exe. It’s the graphical interface for viewing the power data. Here’s a screenshot of data from one day (just the main panel power, click on it to make it bigger):
That’s a rather typical day. The big spike around 5 pm is the oven, and you can see the typical TV-watching time is after 8 pm once the kids are in bed. On the right you can see it breaks out the power usage in kWh for each channel, and in our area we have time-of-use billing, and this actually breaks the power usage out into off-peak, mid-peak, and on-peak amounts.
The viewer program uses a line graph for all time periods under 24 hours, but switches to an hourly bar graph for periods longer than that. Here is a 48-hour time period, and I’ve changed from just the Main Panel to all the branch circuit channels instead:
If you ask for a time period of more than a week it switches into daily bars:
Pulling up two weeks of data like that was actually very fast, just a few seconds.
One final feature I added was email alerts. One of the channels monitors just my backup sump pump circuit. If it ever turns on, that’s because my main sump pump has failed, and I want to know. Through a configuration file, I configured an alarm to email me if that circuit ever records more than 5W of power in a 10 second time slice. I did a little test and it works (it does require that you have a Gmail account though).
Here’s the source code, if you’re interested:
As always, feel free to email me if you have questions or comments.