Online Monitoring of Industrial Equipment using Custom Software & off-the-shelf HardwareDavid LaVine2022-11-09T10:07:02-05:00
Online Monitoring of Industrial Equipment
Using Custom Software & Off-The-Shelf Hardware
We’ve been using the NI cRIO for online monitoring applications since 2010
Over 500 cRIO projects completed (including test & embedded applications)
Platinum-level National Instruments Alliance Partner
Do you need to continuously monitor your industrial equipment / machine / system? Maybe you care about uptime. Maybe you’re more focused on maintenance costs. Maybe something else entirely. Whatever the reason, if you haven’t found an off-the-shelf solution that meets your needs, you’ll want to consider this path.
We develop custom software for off-the-shelf hardware (namely the NI CompactRIO).
How we can help: our capabilities & expertise
Custom software development including data acquisition, signal processing, communication interfaces, and GUI development
Hardware selection (cRIO controller, I/O modules)
Synchronized data acquisition – even for distributed systems (e.g., IEEE-1588, TSN, GPS)
FPGA-based algorithm development
Want some proof points? Check out these case studies:
Condition Monitoring – Improving the Uptime of Industrial Equipment
Monitoring the Health of Industrial Equipment
Client: A large industrial company that uses industrial-grade compressors.
Increase awareness of potentially harmful operating conditions.
Record detailed data upon event detection.
Reduce unnecessary equipment shutdowns due to spurious vibration transients.
We utilized an off-the-shelf controller (NI cRIO) combined with custom software in order to augment and create the first system with ~2 man-months of effort. This solution has been installed in several facilities and is projected to be installed in hundreds of facilities around the world.
Send alerts via email when potentially harmful operating conditions occur.
Record detailed data upon event detection for failure analysis and predictive maintenance.
Suppress spurious vibration transient signals to reduce unnecessary equipment shutdowns.
Condition Monitoring for Electric Power Generation
Monitoring generator and turbine components of power generation equipment
The CompactRIO-based system has allowed for continuous monitoring, rather than just a periodic review of turbine and generator performance. In addition, by combining the FPGA and the RT processor in a physically small device, the solution has been able to ensure very fast data acquisition, data reduction, and sophisticated analysis.
Client: A multi-national power generation equipment manufacturer
Continuous monitoring of power generation equipment can have a great impact on maintaining a reliable flow of power to consumers as well as alerting the power generation equipment operator to potential equipment damage if timely repairs are not made.
This case study will focus on two measurement systems utilized by a multi-national power generation equipment manufacturer to monitor the generator and turbine components of their power generation equipment.
The manufacturer’s systems needed relatively high-speed waveform sampling, well-suited to the National Instruments CompactRIO platform. Viewpoint Systems provided technical assistance in the development of these systems.
The difference in the types of analyses and data rates of the measurement systems required a flexible yet capable hardware platform. Each system needed to work on a generator outputting 50 Hz AC or 60 Hz AC.
The CompactRIO platform and LabVIEW proved to be an excellent solution for the electric power generation condition monitoring system’s data acquisition and analysis needs. The small size and robustness of CompactRIO allowed the system to be placed at a preferred location. In both the flux probe and the blade tip timing, the CompactRIO FPGA could acquire and pre-process the data. The CompactRIO successfully managed – and continues to manage – all analysis, data archiving, and communication with a host PC.
In the case of the tip timing, the data rates were high enough that the detection of the tip location for each signal needed to be performed in the FPGA so that the real-time (RT) layer received a much-reduced data rate of tip locations. The RT processor was able to perform higher level analyses on these timings. Occasionally, a snapshot of a raw tip timing waveform could be passed to the RT processor for archiving and presentation to an engineer. However, due to the data bandwidth and processor loading of the CompactRIO, such snapshots must be infrequent.
For both systems, a master PC managed the operator user interface, long-term data collating, reporting, and archiving of files and statistics. Each CompactRIO connected to this master PC via a TCP/IP connection.
The CompactRIO-based system has allowed for continuous monitoring, rather than just a periodic review of turbine and generator performance. In addition, by combining the FPGA and the RT processor in a physically small device, the solution has been able to ensure very fast data acquisition, data reduction, and sophisticated analysis. By deploying CompactRIO devices, the multi-national power generation equipment manufacturer achieved a cost-effective method of monitoring the power generation facility equipment, ensuring detection of operational issues quickly and easily.
Both measurement systems described required sampling rates greater than 10 kHz, restricting the use of traditional PLC-based data acquisition devices and requiring a programmable automation controller (PAC). Each system measured the performance by connecting to special sensors and associated signal conditioning, provided by our customer, such that the data acquisition equipment only needed to support ±10 V signals. Furthermore, each of these systems needed to push data to a master PC for data trending, result archiving, and operator display.
Despite the significant differences in the measurement types, Viewpoint Systems was able to utilize a common set of data acquisition, processing, and connectivity tools, based on the NI CompactRIO platform and LabVIEW, to monitor the system.
More information about each measurement system follows.
The flux probe system looks for shorts in the windings of the generator. Each time a winding passes under the flux probe, the probe output increases. When a winding is shorted, the field created by the winding is reduced and detected as a lower amplitude output by the flux probe. The position of a shorted winding inside the generator can be located by measuring a key-phasor signal that pulses once per revolution and converting the timing offset of this weakened signal into an angular position. Both flux and key-phasor signals are measured at about 50 kS/s.
Figure 1 shows an example signal output by a flux probe. The local peaks are indicative of winding current. Automated analysis of the amplitudes of the flux signals can be challenging due to changing waveform shape as a function of generator load and severity of shorts.
Figure 1 – Example flux signal over a single rotation
The turbine tip timing system looks for displacement of each turbine blade tip from nominal position. At slow rotational speeds, the spacing between each tip closely follows the uniform blade spacing. At higher speeds, vibrations and resonances can make the blade tips wobble slightly, causing small deviations in the timing of the tip passing by a sensor.
A special proximity sensor detects the tip of the turbine blade, and can be based on optical, eddy-current, microwave, and other techniques. Any positional deviations of a tip from nominal give indications about the mechanical forces on the blade as well as compliance of the blade to those forces as the blade ages. Specifically, each blade has natural resonances and compliance, both of which can change if the blade cracks.
A turbine typically contains several stages and each stage contains many blades. See Figure 2 below for an example. The number of tip sensors per stage is variable; if blade twist is measured, at least two sensors are oriented perpendicular to the rotation direction. Also, the acquisition rate from each sensor is fast. For example, consider a stage with 60 blades, the width of each blade occupying about 1/10 the space between adjacent blades, and a generator running at 3600 RPM (60 Hz). The tip sensor would detect a pulse every 1/3600 s, lasting for less than about 1/36000 s, as the blades passed by. Accurate location of the pulse peak or zero-crossing then requires sample rates over 100 kS/s. Because multiple sensors are typically used, tip timing measurement systems can easily generate 10s of MBs of data per second.
Industrial Embedded Monitoring – Remote Structural Health Monitoring
Using a cRIO to remotely assess structural health
By connecting these systems with a host PC, we can monitor continuous vibration activity and alarm conditions on a variety of structures despite inclement weather.
Continuously monitoring the structural health of the Long Island Railroad (LIRR) Viaduct despite the relative inaccessibility of the structure.
Using CompactRIO, LabVIEW FPGA, and the LabVIEW Digital Filter Design Toolkit to measure the modal analysis of vibration data generated from ambient excitation, capture this data remotely, and analyze significant events.
Engineers use structural vibrations to assess the conditions of many constructions and machines, including buildings, bridges, dams, towers, cranes, and mountings. Although we have had tools to monitor structural vibration for decades, these tools restrict data collection to short durations of high-fidelity waveforms or longer durations of summarized power in frequency band results. Many structures vibrate in meaningful ways only in the presence of ambient forces such as wind, vehicle activity, nearby construction, or random events such as earthquakes and tornados. Therefore, data collection needs to be active during these events.
Due to recent improvements in memory storage, processor speed, and wideband wireless communications technology, we can collect high-fidelity waveforms over long periods. We can also communicate to host PCs that aggregate structural vibration data across multiple collection locations, providing permanent data collection and superior analysis and reporting capabilities.
STRAAM Corporation, a leader in structural integrity assessment, and Viewpoint Systems, a Select National Instruments Alliance Partner, collaborated to develop a system that functions outdoors and in other less-accessible sites and maintains the capabilities of the available PC-based solution. Ultimately, we produced an enhanced version of STRAAM’s SKG CMS™ system to install on a Long Island railroad bridge.
The system needed to perform the following operations:
Collect data from accelerometers and other environmental sensors
Store weeks of data locally at full acquisition rates
Analyze custom data in real time
Publish summary statistics periodically to the host
Contain flexible architecture to handle future capabilities
Ensure secure user access control
We chose a system based on the NI CompactRIO platform and dynamic signal acquisition (DSA) C Series modules. The CompactRIO and associated C Series signal conditioning modules have an operating temperature range of -40 to 70 °C, well within typical environmental extremes for most installation locations. Additionally, the CompactRIO controller has no moving parts, increasing the mean time between failure and ensuring it can withstand physical mishandling during shipment and installation. For software, we decided to use the NI LabVIEW Real-Time Module and the LabVIEW FPGA Module. We used LabVIEW FPGA for basic signal acquisition as well as some custom antialiasing filtering to allow for sampling rates below the capabilities of the DSA module.
Figure 1 – Equipment mounted to LIRR Support Beam
Data Acquisition and Filtering
The DSA module acquired acceleration signals via special sensors, supplied by STRAAM, that output information about tilt and acceleration. Because large structures resonate at low frequencies, it is important that these sensors have extremely low noise, high dynamic range, and low frequency response to gather information about structures at less than 1 Hz. The low frequency range and long-term data storage need combine to create a maximum data collection rate frequency of 200 samples per second (S/s). The NI 9239 does not sample that slowly due to its delta-sigma converter technology, so we sampled at 2,000 S/s and used lowpass digital filtering on the field-programmable gate array (FPGA) to produce an antialiased signal at 200 S/s. Simple subsampling through decimation would violate the Nyquist criterion. Using the LabVIEW Digital Filter Design Toolkit, we produced a 28-tap infinite impulse response (IIR) filter with a 3 dB roll-off at 0.8 times the sample rate with a stopband attenuation greater than 90 dB. The Digital Filter Design Toolkit includes tools to automatically generate code to deploy the filter to the FPGA. We carefully selected fixed-point arithmetic to ensure proper operation without using excessive FPGA resources. The final filter was a 24-bit fixed-point solution with a 4-bit mantissa.
Figure 2: Remote Front Panel Displaying Acceleration Waveform Capture
Configuration, Signal Processing, and Alerts
STRAAM uses proprietary analysis routines, based on the structure’s resonant frequencies, to extract relevant information from the continuous stream of acceleration data. Because ambient energy excites the structures, we analyzed some initial data to locate these resonances. After this initial period, we configured the CompactRIO to perform the proprietary analyses based on the location of these resonances. We handled all activity in this initial setup remotely via wireless communications. We connect to CompactRIO over a wireless connection, then to a LabVIEW remote panel where we initially acquire and assign resonance bands.
The signal processing requires the spectral power and time-domain structure of the waveforms inside those resonant bands. The CompactRIO processor and FPGA module can calculate fast Fourier transform (FFT)-based power spectrums and perform time-domain filtering calculation so we can base calculations on the complicated algorithms provided by STRAAM. Furthermore, the large CompactRIO RAM can archive raw acceleration waveforms for later retrieval. The LabVIEW development environment greatly simplifies adjusting these calculations. We apply additional calculations to identify noteworthy events to alert the engineers when important conditions occur. These conditions may signify the presence of a meaningful ambient excitation or that considerable changes to the structure have occurred.
In order to successfully operate, this system needs to communicate effectively to the host PC. Because the system is deployed in almost-inaccessible and outdoor locations, all interactions with the system should occur remotely. Using cellular modems, the system connects via TCP/IP to upload important information, issue event alerts, and allow remote configuration. We designed the LabVIEW application to send periodic summary information via custom binary messages to the host with information about the condition of the structure and the CompactRIO system. The host then tallies this information along with all other SKG CMS™ systems deployed in the field. In addition to this summary information, the host can pull raw waveform data from the CompactRIO RAM. To avoid tampering and unauthorized access, we password protected all connections.
Figure 3 – Data File Configuration Screen
We have successfully installed several functional SKG CMS™ systems based on the CompactRIO platform. By connecting these systems with a host PC, we can monitor continuous vibration activity and alarm conditions on a variety of structures despite inclement weather. Our customers enjoy the benefits of modern Ethernet-driven, Web-based connectivity to verify the status of their structures and we enjoy the benefits of the rugged, reliable, low-cost, and reprogrammable CompactRIO system for data collection.
A manufacturer of large industrial mission-critical equipment in the electrical energy / power industry.
Our client had three main goals in mind. They wanted to:
Decrease unanticipated downtime and maintenance expenses
Provide a more complete picture of machine operation and state
Improve equipment usage tracking.
The solution is a multi-node (i.e. multi-site) remote monitoring system that utilizes an NI cRIO-based controller with customized NI InsightCM monitoring software.
Monitors vibration signals to predict expensive equipment failures
Monitors current machine state via Modbus from other equipment in the system, including the primary system controller
Provides alerts via email when any designated parameter is out of range
The remote monitoring system monitors equipment condition by taking several vibration signal measurements along with reading over 500 Modbus registers. Local InsightCM vibration analysis on the cRIO extracts key features from the accelerometer data. Limit detection is run on these features and other equipment state and alarms are triggered when data is out of bounds. Information collected at multiple sites is sent to a central location either at periodic intervals or based on an alarm condition.
NI InsightCM software
Modbus register configuration & reading
Dead banding-style register data collection to decrease amount of data captured and transferred
Dynamic signal data capture
Data transfer scheduling
Semi-real-time alarm channel display
NI IEPE Analog Input Module
Microsoft Windows Server to host the NI InsightCM server software
Online Monitoring of Industrial Equipment using NI CompactRIO
Improving Maintenance of expensive industrial equipment
Client – Large Industrial Equipment Manufacturer
The maintenance of the equipment was not always done at the prescribed intervals because the cost of shutting down the plant is significant. This sometimes resulted in an equipment failure. This particular application is for equipment/machinery in the energy/power industry (a generator).
The online monitoring system monitors a particular parameter of interest to send warnings and alarms to the control room so that the operators know when maintenance needs to be performed on the particular part of interest. This system has been installed in multiple plants.
Enables condition-influenced maintenance intervals vs periodic intervals
Reduces probability of catastrophic failure by providing warning indicator
The system monitors the generator collector health. NI-based data acquisition hardware acquires the signal of interest, logs the raw data, processes the parameter of interest, and triggers/sends warnings and alarms to the control room. LabVIEW FPGA was used for analog and digital IO and a sensor check. LabVIEW Real Time was used for the calculation, data logging, serving data to the HMI and alarm/warning checking.
Touchscreen GUI for data/alarm display and system configuration
Remotely Monitoring Electrical Power Signals with a Single-Board RIO
Electronics Design for sbRIO Mezzanine Card Combines Custom Needs with Flexibility
Client: A designer and manufacturer of leading-edge electrical power monitoring equipment.
Smart Grid investment is growing. Two important premises for Smart Grid design are access to local power sources and an understanding of loads and disturbances on the grid at various locations. These local power sources are typically alternative, such as solar and wind, which have intermittent power levels. Since the levels fluctuate, an important feature of proper Smart Grid operation is handling these erratic supplies. Optimal understanding of these disturbances and load changes increasingly requires measurements on individual AC power cycles.
Local power analysis systems typically have constraints in equipment cost, size, and power usage balanced against the need for simultaneous sampling front-end circuitry and custom data processing algorithms on the back-end. Furthermore, many of these systems are presently deployed as prototypes or short-run productions, requiring a combination of off-the-shelf and custom-designed components.
A custom RIO Mezzanine card was designed and built for the National Instruments Single-Board RIO platform to provide access to simultaneously-sampled signals from the 3-phase and neutral lines of an AC power source. Timing synchronization between physically-separated installations was provided by monitoring GPS timing signals. Custom VIs were developed to retrieve the sampled data points and GPS timing for subsequent processing and analysis.
Figure 1 – Power Line Data Acquisition sbRIO RMC Module with GPS Timing
We needed 8 channels of simultaneously-sampled analog inputs (AI), each capable of sampling at least 50 kHz. These AI channels sample the voltage and current of the neutral and three phase power lines. Furthermore, to coordinate power and load fluctuations across many measurement locations, a world-wide synchronization signal is needed.
The Single-Board RIO (sbRIO) platform from National Instruments offers an excellent balance between off-the-shelf capability and custom design needs in a reasonably small package. The sbRIO provides the processor, memory, and connectivity while the RIO Mezzanine Card (RMC) provides the I/O and signal conditioning needs. See our white paper, Developing Embedded Systems: Comparing Off-the-Shelf to Custom Designs, for a discussion of the benefits of using this approach.
We designed the RMC for the simultaneously-sampled analog inputs and a GPS receiver. The RMC was mounted to a sbRIO-9606. Some design specifications were:
8 analog input channels: simultaneous sampling at 50 kHz, ±10 V range, 16-bit resolution
GPS receiver with Pulse Per Second (PPS) timing signal with 60 ns accuracy
SMA Connector for external GPS active antenna
20 position terminal block for analog inputs and shields, removable for wiring
Operates inside an enclosure with internal conditions -40 to 55 °C temperature
An image of the designed RMC and the sbRIO-9606 is shown below. Since the A/Ds reside on the RMC, the data bytes are accessed by sbRIO FPGA VIs code communicating through an SPI data bus designed into the RMC. The internal real time clock coupled with the GPS PPS signal allowed for timing accuracy within a GPS region well under +/- 1 uS of accuracy for all data sampled no matter the location, internally or from unit to unit within feet or 1000s of miles away.
The combination of the sbRIO off-the-shelf platform and the custom RIO mezzanine card (RMC) for I/O makes a powerful, cost-effective, and yet configurable solution for measurements of AC power signals. With the GPS component on the RMC, measurement units can be placed at dispersed locations while still providing adequate synchronization of acquired waveforms for localizing and understanding disturbances in power transmission and distribution, irrespective of any specific application. If you have an embedded monitoring application that you’d like help with, you can reach out to chat here. If you’d like to learn more about our circuit board design capabilities, go here.
Viewpoint Systems helped us develop our in-house capability for both traditional and R&D condition monitoring. With us from the beginning, we had them architect the data collection application on a custom cRIO configuration. Then, as we took over development, Viewpoint was always ready to help us out when we discovered issues. While most of the project was handled by one Viewpoint Systems engineer, our team appreciated that several engineers were available as needed to move the project forward. I’m pleased with the overall results and look forward to working with them in the future.
Greg HenzlerAssociate Director Turbomachinery R&D, Praxair
We’ve helped teams at some of the world’s most innovative companies