I was unsure on whether to write this post or not. Why? Well, because it is so specific that it is quite possible that no one would get any benefit from it. But then I thought: "Knowing this would have greatly helped me and my customer 2 years ago, when we decided to start this project". So maybe there's someone out there that could be in a similar position and get some help from this effort.
In order to avoid people to read and finally wonder why they wasted their time, I'd better start with some relevant keywords; if you've never heard of them, then it's very likely you don't want to spend your time reading more; here we go: GEMSTART & GEMDRIVE.
Know them? Or just curious? Well, GEMSTART is the name of a motor protection relay, but I am not talking about the Gemstart 5, but the Gemstart 3, an ancient version of the device which is out of production and one struggles to find information about. Same story for the GEMDRIVE GD2000, a Variable Frequency Drive.
Both this products have a glorious past, but if you run out of spares you need some good luck and maybe an eBay account.
Still reading? Great, then you probably know a production plant in a situation similar to my customer's. Here a very brief history:
Big production plant, with hundreds of motors to move
Control system implemented 20+ years ago, chosen motor drivers Gemstart 3 for fixed speed, GD2000 for variable speed motors, with a Gembus capable controller (Gem 80 maybe?)
The controller gets obsolete, so it needs to be replaced
There are plenty of controllers available, with plenty of fieldbuses, but guess what? Gembus is not among them, it didn't get the standardization attention that other fieldbuses got (like PROFIBUS, MODBUS, CAN and so on). In other words, upgrading the main controllers means replacing all your motor drivers, which is a complete different (and much more difficult) endeavor
If all of this hit your attention, then keep reading to see how I solved the problem.
Early days: BM85
If I had come into this at this moment in the history I would probably made different choices, but as a matter of fact I didn't participate to the first episode in this transition. In fact, the customer took the decision to migrate towards Schneider controllers, which means embracing the modbus family of protocols as a first class citizen. And indeed this seems a good choice, considering that at least one of our GEM-devices (GD2000) supports modbus RTU, in addition to Gembus.
But, for reasons I can only imagine (when I appeared on the scene I couldn't get an explanation), it was decided not to change the protocol and leave all the devices untouched.
Instead, it was decided to develop a bridge/gateway component. The Gembus protocol had to be developed in software, though, so there was a need for a programmable component. Schneider had a piece of hardware in their catalog, called BM85, made explicitly to work as a concentrator for modbus networks. By default it offered a modbus plus port to the main controller, plus 4 output serial ports to dispatch data to remote trunks of modbus devices. It also had the capability to host a C program, allowing a software engineer to override the defaults and transform it in a custom gateway. And this is what they did: they transformed the BM85 into a modbus+/gembus gateway. with a network structure like this:
This was working fine, so good that they almost forgot about it.
End of 2013: oops! Houston we have a problem
Everything worked fine, I said. Well, almost all the time. Because sometimes there were comms issues. And it's exactly when one of these issues came around that I was asked to look into it. And when I did, it was like opening pandora's box. After a short investigation, I discovered that
The BM85 wasn't used with factory settings, but needed a custom program to work
Nobody seemed to have the C source code for the program. The only thing available was a compiled binary to transfer through a quite weird procedure that nobody had never executed in the customer's staff
The customer had only 3 BM85 spares still available
The BM85 product was obsolete, out of production and out of support. The tools to rebuild the program were MS-DOS, while MS-DOS was by long time a museum memorabilia
Running out of spares would mean stopping production, with literally millions in production loss
Hence, in a few hours, what was an almost forgotten grey box hidden inside field cabinets, became a critical risk!
The birth of the BM85U project
The plant had not only 1 bridge, but around a dozen or more. They needed to be replaced with something in its active market life-cycle. But the biggest issue was that, at the time the risk was brought to light, all the documentation about the protocols was unreachable, so I was prospecting a long and uncertain protocol reverse engineering process based on traffic logging and analysis.
Fortunately, I was able to track back to one of the engineers who worked on that C program years back in time, and more fortunately he still had the source code for that program.
Using that source directly was not possible though, because it only worked on the BM85 and because the compiler to be used wasn't available any longer. Moreover, the customer staff wasn't comfortable with C, because they were most PLC programmers, so used to IEC61131 languages like FBD and ST.
Another problem was that the plant was in production 24/7 with no plan to stop it, so the replacement needed to occur in the fastest and least invasive way.
My choice went to a Beckhoff CX. Here the reasons:
Open serial protocol library
Possibilty to install 4 embedded and many more RS-485 bus interfaces
Programming compliant with the IEC61131 standard
Availability of pretty much all known industrial protocols, including modbus
Low cost
I had previously worked with Beckhoff CX products, and I especially like the programming experience based on Codesys, which in my opinion is one of the best IEC61131 engineering platforms.
A new project was born and I needed to baptize it. Being a BM85 Upgrade project, I decided, admittedly with very poor creativity, to name it BM85U. This is what the new structure looks like:
Project execution
1.Serial protocol Reverse Engineering
First of all, I thoroughly studied the BM85 custom C program, and reconstructed the serial protocols for Gemstart 3 and GD2000, byte by byte. The source was well commented, but understandably it didn't contain a full copy of the protocol documents, so it took a while to reconstruct all the byte streams, including special cases, retry policies and timings.
I couldn't run the original project, but I did create several test programs pasting parts of the original project in a Visual Studio C++ project and using a PC connected to a lab device taken from the spares.
Also my previous experience with telecommunication projects helped quite a lot when something seemed out of line and I had to guess what was going on.
2.Reimplementing the bridge program
With the protocol deciphered I then proceeded writing the logic for the CX. In this specific case I was programming a comms state machine rather than a machinery automation, so the most suitable language is ST; the CX program is fully written in ST.
The application can be used with a variable number of serial ports, and it's not difficult to extend beyond the 4 ports if needed, as the CX is not limited to the embedded ports, but can contain further serial cards in the IO bus.
3.Migrating from Modbus Plus to Modbus TCP
The BM85 offered a modbus Plus port to the main controller. Modbus Plus (aka modbus+) is a modbus extension to allow Schneider/Modicon controllers to overcome the plain modbus RTU limits and allows a single controller port to link to several modbus RTU trunks, using a concentrator/dispatcher like the BM85.
The other variant of the Modbus protocol is the TCP version, which instead of using a serial cable, leverages on a Ethernet TCP/IP infrastructure.
Due probably to the ubiquitous popularity reached by ethernet, Modbus TCP has known a much happier development than Modbus Plus, which I think has not been used in new projects for a while nowadays. Even Beckhoff, with one of the most complete protocol suites, provides Modbus TCP support but not Modbus Plus.
Therefore, the CX logic has been built to expose the data as Modbus TCP server. This was not enough though, since the logic in the main Schneider controller dedicated to these comms couldn't be used being for Modbus Plus. So, as last component of this project. I had to write a Unity DFB (again using ST as language) to manage the link with the CX. Particular attention in the project had been put in this step, as it was critical to not touch anything else in the main controller logic, to cut by design any potential unwanted side effects on the process control.
Project outcome
The outcome has indeed been very good:
By mid 2014, all the BM85s were replaced
Production loss = zero, as all the work could be done during planned shutdowns in a few hours, almost plug-and-play
After more than 3 years of service, no comms issue have been found, so the solution proved to be stable
So, I replaced a grey box with a different grey box, and the new has now been almost forgotten as the first one, but with important differences:
The CX product is new and well supported
The program is written in IEC61131 and can be maintained by a much larger base of professionals (a C programmer doesn't usually have problems in working with ST code, but the other way around is different)
The source code is saved along the compiled binary inside the bridge memory, and so it can be uploaded, eliminating the risk of losing it.
Conclusion
I am particularly proud of this project, because I think its benefit/cost ratio is very high (and well above 1000), even if, sadly, few have really realized that.
If anyone is interested in more details, leave a comment with your question and I'll try to answer.