Industrial Control System architecture

Industrial control system (ICS) is a general term that encompasses several types of control systems, including supervisory control and data acquisition (SCADA) systems, distributed control systems (DCS), and other smaller control system configurations such as rail-mounted Programmable Logic Controllers (PLC) and fieldbuses often found in the industrial sectors and critical infrastructures. ICSs are typically used in industries such as electrical, water, oil and gas. Based on information received from remote stations, automated or operator-driven supervisory commands can be pushed to remote station control devices, which are referred to as field devices. Field devices control local operations such as opening and closing valves and breakers, collecting data from sensor systems, and monitoring the local environment for alarm conditions.

Figure 1.2. SCADA system

SCADA system

DCSs are used to control industrial processes such as electric power generation, oil and gas refineries, wastewater treatment, chemical, food and automotive production. DCSs are integrated as a control architecture containing a supervisory level of control overseeing multiple, integrated sub-systems that are responsible for controlling the details of a localized process. Product and process control are usually achieved by deploying feed back or feed forward control loops whereby key product and/or process conditions are automatically maintained around a desired set point. To accomplish the desired product and/or process tolerance around a specified set point, specific programmable controllers are used.

PLCs provide boolean logic operations, timers, and (in some models) continuous control. The proportional, integral, and/or differential gains of the PLC continuous control feature may be tuned to provide the desired tolerance as well as the rate of self-correction during process upsets. DCSs are used extensively in process-based industries. PLCs are computer-based solid-state devices that control industrial equipment and processes. While PLCs are control system components used throughout SCADA and DCS systems, they are often the primary components in smaller control system configurations used to provide regulatory control of discrete processes such as automobile assembly lines and power plant soot blower controls. PLCs are used extensively in almost all industrial processes.

SCADA is the acronym for Supervisory Control And Data Acquisition. The term refers to a large-scale, distributed measurement (and control) system. SCADA systems are used to monitor or to control chemical or transport processes, in municipal water supply systems, to control electric power generation, transmission and distribution, gas and oil pipelines, and other distributed processes.

A SCADA system includes input/output signal hardware, controllers, Human Machine Interface (HMI), networks, communication, database and software. The SCADA is usually a central system that monitors and controls a complete site or a system spread out over a long distance (kilometres/miles). The bulk of the site control is actually performed automatically by a Remote Terminal Unit (RTU), a fieldbus system such as Profibus with intelligent transducers or by a Programmable Logic Controller (PLC). Host control functions are almost always restricted to basic site over-ride or supervisory level capability. For example, a PLC may control the flow of cooling water through part of an industrial process, but the SCADA system may allow an operator to change the control set point for the flow, and will allow any alarm conditions such as loss of flow or high temperature to be recorded and displayed. The feedback control loop is closed through the RTU or PLC; the SCADA system monitors the overall performance of that loop.

Data acquisition begins at the RTU, PLC or fieldbus level and includes meter readings and equipment statuses that are communicated to SCADA as required. Data is then compiled and formatted in such a way that a control room operator using the HMI can make appropriate supervisory decisions that may be required to adjust or over-ride normal RTU (PLC) controls. Data may also be collected in to a Historian, often built on a commodity Database Management System, to allow trending and other analytical work.

SCADA

SCADA systems typically implement a distributed database, commonly referred to as a tag database, which contains data elements called tags or points. A point represents a single input or output value monitored or controlled by the system. Points can be either "hard" or "soft". A hard point is representative of an actual input or output connected to the system, while a soft point represents the result of logic and math operations applied to other hard and soft points. Point values are normally stored as value-timestamp combinations; the value and the timestamp when the value was recorded or calculated. A series of value-timestamp combinations is the history of that point. It's also common to store additional metadata with tags such as: path to field device and PLC register, design time comments, and even alarming information.

It is possible to purchase a SCADA system, or Distributed Control System (DCS) from a single supplier. It is more common to assemble a SCADA system from hardware and software components like Telvent, Allen-Bradley, ABB, Siemens, DirectLogic or GE PLCs, HMI packages from Adroit, Wonderware, Iconics, Rockwell Automation, Inductive Automation, Citect, or GE. Communication typically happens over Industrial Ethernet.

SCADA systems traditionally run on DOS, VMS and UNIX, in recent years many SCADA vendors have moved to Windows NT and to Linux.

In general SCADA consist of two basic layers: the "client layer" for the man machine interaction and the "data server layer" which handles most of the process data control activities. The data servers communicate with devices in the field through process controllers. Process controllers, e.g. PLCs, are connected to the data servers either directly or via networks or fieldbuses that are proprietary (e.g. Siemens H1), or non-proprietary (e.g. Modbus). Data servers are connected to each other and to client stations via Industrial Ethernet.

Figure 1.3. SCADA client server architecture

SCADA client server architecture

The software componets interface with based a database(s) located in one or more servers. Servers are responsible for data acquisition and handling e.g. polling controllers, alarm checking, calculations, logging and archiving.

Figure 1.4. SCADA software architecture

SCADA software architecture

Often there are dedicated servers for the historian, datalogger and alarm handler. Server-client and server-server communication is in general on a publish-subscribe and event-driven basis and uses a TCP/IP protocol, i.e., a client application subscribes to a parameter which is owned by a particular server application and only changes to that parameter are then communicated to the client application.

The data servers poll the controllers at a user defined polling rate. The controllers pass the requested parameters to the data servers. Time stamping of the process parameters is typically performed in the controllers and this time-stamp is taken over by the data server. SCADAs provide communication drivers for most of the common PLCs and widely used field-buses, e.g., Modbus, Profibus and Worldfip. A single data server can support multiple communications protocols: it can generally support as many such protocols as it has slots for interface cards.

The provision of the OPC and OPC UA standards for SCADA to access devices and exchange data in an open and standard manner is also supported. Besides OPC the following application layers are often used for data exchange:

  • Open Data Base Connectivity (ODBC) interface to the data in the archive/logs.

  • ASCII import/export facility for configuration data with XML encoding progressively replacing the use of ASCII or Comma Separated Values (CSV).

  • Libraries of APIs supporting C, C++, and Visual Basic (VB) to access data in the RTDB, logs and archive with XML based SOAP services progressively incorporated for exchanging data between information systems.

The PC based SCADA products traditionally provide support for the Microsoft standards such as Dynamic Data Exchange (DDE) which allows e.g. to visualise data dynamically in an EXCEL spreadsheet, Dynamic Link Library (DLL), COM and DCOM servers - Microsoft successors to Object Linking and Embedding (OLE).

The configuration data are traditionally stored in a database that is logically centralised but physically distributed and that is generally of a proprietary format. For performance reasons, the real-time database (RTDB) resides in the memory of the servers and is also of proprietary format. The archive and logging data is often stored in a Relational Data Base Management System (RDBMS) which can be accessed using Structured Query Language (SQL) via an ODBC interface.

Scalability is understood as the possibility to extend the SCADA based control system by adding more process variables, more specialised servers (e.g. for alarm handling) or more clients. SCADAs achieve scalability by having multiple data servers connected to multiple controllers. Each data server has its own configuration database and RTDB and is responsible for the handling of a sub-set of the process variables (acquisition, alarm handling, archiving).

The products often have built in software redundancy at a server level, which is normally transparent to the user.

Access control and authentication is implemented at the application layer using Users whom are allocated to Groups, which have defined read/write access privileges to the process parameters in the system and often also to specific SCADA functionality.

SCADA HMI interfaces support multiple screens on the client workstations, which can contain combinations of synoptic process diagrams and text on a GUI driven interface. They also support the concept of a "generic" graphical object with links to process variables. These objects can be "dragged and dropped" from a library and included into a process diagram.

The process is modelled in terms of "atomic" parameters or process variables (e.g. pressure PV value, its maximum value, process limits, etc.) to which a tag name is associated. The tag names is used to link graphical objects to the process variable. SCADAs include a library of standard graphical symbols commonly used in industrial process and manufacturing industries. Standard windows editing facilities are provided: zooming, re-sizing, scrolling. On-line configuration and customisation of the MMI is possible for users with the appropriate privileges. Links can be created between display pages to navigate from one view to another.

SCADAs provide trending facilities which can be summarized as follows:

  • Parameters to be trended in a specific chart can be predefined or defined on-line.

  • Real-time and historical trending are possible, although generally not in the same chart.

  • Historical trending is possible for any archived parameter.

  • Zooming and scrolling functions are provided.

  • Parameter values at the cursor position can be displayed.

The trending feature is either provided as a separate module or as a graphical object (ActiveX), which can then be embedded into a graphic display.

Alarm handling is based on limit and status checking and performed in the data servers. More complicated expressions (using arithmetic or logical expressions) can be developed by creating derived parameters on which status or limit checking is then performed. The alarms are logically handled centrally, i.e., the information only exists in one place and all users see the same status (e.g., the acknowledgement), and multiple alarm priority levels (in general many more than 3 such levels) are supported. It is generally possible to group alarms and to handle these as an entity (typically filtering on group or acknowledgement of all alarms in a group). It is possible to suppress alarms either individually or as a complete group. The filtering of alarms seen on the alarm page or when viewing the alarm log is also possible at least on priority, time and group. E-mails can be generated or predefined actions automatically executed in response to alarm conditions.

Logging can be thought of as medium-term storage of data on disk, whereas archiving is long-term storage of data either on disk or on another permanent storage medium. Logging is typically performed on a cyclic basis, i.e., once a certain file size, time period or number of points is reached the data is overwritten. Logging of data can be performed at a set frequency, or only initiated if the value changes or when a specific predefined event occurs. Logged data can be transferred to an archive once the log is full. The logged data is time-stamped and can be filtered when viewed by a user. The logging of user actions is in general performed together with either a user ID or station ID. There is often also a play-back facility to view archived data.

Production reports can be generated using SQL type queries to the archive, RTDB or logs. Facilities exist to be able to automatically generate, print and archive reports.

SCADAs allow actions to be automatically triggered by events. A scripting language provided by the SCADA allows these actions to be defined. In general, one can load a particular display, send an Email, run a user defined application or script and write to the RTDB. The concept of recipes is supported, whereby a particular system configuration can be saved to a file and then re-loaded at a later date.

Sequencing is also supported whereby it is possible to execute a more complex sequence of actions on one or more devices. Sequences may also react to external events.

Some SCADAs support expert systems and the concept of a Finite State Machine (FSM) to support what is referred to as an "intelligent SCADA". Intelligent SCADAs have the preprogrammed ability to perform advanced control functions and to provide guidance to operators in order to make informed decisions.

The development of SCADA applications is typically done in two stages. First the process parameters, tag names and associated information are defined through a parameter definition template and then the graphics, including trending and alarm displays are developed, and linked where appropriate to the process parameters. SCADAs provide an ASCII Export/Import facility for the configuration data (parameter definitions), which enables large numbers of parameters to be configured in a more efficient manner using an external editor such as Excel and then importing the data into the configuration database.

Many of the PC based tools have a Windows Explorer type configuration and development studio. The developer then works with a number of folders, which each contains a different aspect of the configuration, including the graphics. On-line modifications to the configuration database and the graphics is generally possible with the appropriate level of privileges. The following development tools are provided as standard:

  • A graphics editor, with standard drawing facilities including freehand, lines, squares circles. It is possible to import pictures in many formats as well as using predefined symbols including e.g. trending charts, etc. A library of generic symbols is provided that can be linked dynamically to variables and animated as they change. It is also possible to create links between views so as to ease navigation at run-time.

  • A data base configuration tool (usually through parameter templates). It is in general possible to export data in ASCII files so as to be edited through an ASCII editor or Excel.

  • A scripting language to automate tasks and complex user-defined functions.

  • An application Program Interface (API) supporting C, C++ or VB with support being added for Web technologies such as Microsoft.NET and Java web service applications

  • A driver Development Toolkit to develop drivers for hardware that is not supported by the SCADA product.

Modern SCADAs use the concepts of object orientated software design with graphical object classes, which support inheritance. In addition, some extend the concepts of an object orientation within the configuration and production databases.

MES and ERP

Manufacturing Execution System (MES) is a system that companies can use to measure and control critical production activities. Some of the benefits with regards to MES solutions are increased and more cost effective production.

It is important to note that the term MES is held vary loosely across different manufacturing industries. MES has many parts and can be deployed on various scales. From simple Work In Progress tracking to a complex solution integrated throughout an enterprise monitoring and controlling all resources used in the manufacturing process from cradle to grave and touching other enterprise systems like Enterprise Resource and Planning Systems (ERPs), Product Life cycle Management (PLMs), Supervisory, Control and Data Acquisition (SCADA) solutions, Scheduling and Planning Systems both long-term and short-term tactical.

There is an growing demand for information exchange between industrial control systems, MES and enterprise information systems such as an ERP system. The SCADA, DCS and MES interface with the rest of the corporate IT network has become possible due to the introduction of the IT network technology consisting of the Ethernet/TCP/IP or "open system" communication stack with application layers such as HTTP, SOAP and SQL playing an increasingly important role.

Enterprise Resource Planning systems (ERPs) attempt to integrate all data and processes of an organization into a unified system. A typical ERP system will use multiple components of computer software and hardware to achieve the integration. A key ingredient of most ERP systems is the use of a unified database to store data for the various system modules.

Ideally, ERP delivers a single database that contains all data for the software modules, which would include:

  • Manufacturing

    Engineering, bills of material, scheduling, capacity, workflow management, quality control, cost management, manufacturing process, manufacturing projects, manufacturing flow.

  • Supply Chain Management

    Inventory, order entry, purchasing, product configurator, supply chain planning, supplier scheduling, inspection of goods, claim processing, commission calculation.

  • Financials

    General ledger, cash management, accounts payable, accounts receivable, fixed assets.

  • Projects

    Costing, billing, time and expense, activity management.

  • Human Resources

    Human resources, payroll, training, time and attendance, benefits.

  • Customer Relationship Management

    Sales and marketing, commissions, service, customer contact and call center support.

  • Data Warehouse

    The main repository of the organization's historical data.

ERP systems typically handle the manufacturing, logistics, distribution, inventory, shipping, invoicing, and accounting for a company. Enterprise Resource Planning or ERP software can aid in the control of many business activities, like sales, marketing, delivery, billing, production, inventory management, quality management, and human resource management.ERPs are cross-functional and enterprise wide. All functional departments that are involved in operations or production are integrated in one system. In addition to manufacturing, warehousing, logistics, and Information Technology, this would include accounting, human resources, marketing, and strategic management.

In the absence of an ERP system, a large manufacturer may find itself with many software applications that do not talk to each other and do not effectively interface. Tasks that need to interface with one another may involve:

  • design engineering (how to best make the product)

  • order tracking from acceptance through fulfillment

  • the revenue cycle from invoice through cash receipt

  • managing interdependencies of complex bill of materials

  • tracking the 3-way match between purchase orders (what was ordered), inventory receipts (what arrived), and costing (what the vendor invoiced)

  • the accounting for all of these tasks, tracking the revenue, cost and profit on a granular level

Some organizations - typically those with sufficient in-house IT skills to integrate multiple software products - choose to implement only portions of an ERP system and develop an external interface to other ERP or stand-alone systems for their other application needs. For instance, the PeopleSoft HRMS and Financials systems may be perceived to be better than SAP's HRMS solution. And likewise, some may perceive SAP's manufacturing and CRM systems as better than PeopleSoft's equivalents. In this case these organizations may justify the purchase of an ERP system, but choose to purchase the PeopleSoft HRMS and Financials modules from Oracle, and their remaining applications from SAP. The largest ERP vendors worldwide in 2005 according to Gartner Dataquest:

Figure 1.5. ERP vendors

ERP vendors

SAP is the largest business application and ERP solution and software provider in terms of revenue. The company's main product is mySAP ERP. The name of its predecessor, SAP R/3 gives a clue to its functionality: the "R" stands for realtime data processing and the number 3 relates to a 3-tier architecture: database, application server and client (SAPgui). R/2, which ran on a Mainframe architecture, was the first SAP version.

The SAP NetWeaver platform is a Service Orientated Architecture (SOA) supporting a series of web services combined with business logic that can be accessed and used repeatedly to support a particular business process. The SAP Web Application Server provides Web services through platform-independent business web applications and technologies. These encompass Java 2 Enterprise Edition and ABAP. ABAP (Advanced Business Application Programming) is a high level programming language created by SAP. It is currently positioned, alongside the more recently introduced Java, as the language for programming SAP's Web Application Server. The SAP Web Application Server supports open technical standards such as UDDI, WSDL, and SOAP.