This article describes one of the possible approaches to implementation of gateway converting regular Ethernet based GOOSE messages to routable (R) version of this protocol. GOOSE which stands for Generic Object Oriented Substation Event, is one of the protocols introduced by the IEC61850 standard released in 2004. Its primary purpose is to notify peers about different protection events generate at intelligent electronic device (IED) e.g. protection relay. In the first edition of the standard it was specified as a protocol implemented directly on top of the link layer – Ethernet. There are two types of actors in GOOSE communications:
- publisher sending multicast frames on the local network
- subscriber which ‘listens’ to selected types of frames generated by other nodes (publishers)
As a solution for horizontal, peer-to-peer communication, GOOSE over Ethernet, seemed to be a good choice, especially considering the requirement of low latency for this type of traffic. Sending out multi-cast Ethernet frames on event driven or periodic basis, involves very low overhead and satisfies timing requirement of low delay between the time at which an event occurred and generation of the message with the information about it.
One obvious limitation of this approach is that such communication is limited to the local network (LAN) and with the introduction of DER and evolution of the grid towards distributed architecture, there was a need for routable version of GOOSE protocol. This implicated the question of the (cyber) security of the very sensitive information which is sent by IED belonging to the critical infrastructure.
In the following sections I describe the solution which does not require major updates to the existing equipment installed in the field, offers great deal of flexibility and is relatively simple to deploy.
Acronyms
AES – Advanced Encryption Standard
AES-CBC – AES cipher block chaining
AES-GMC – AES Galois counter mode
APDU – application protocol data unit
APPID – application identifier
CRL – certificate revocation list
DER – distributed energy resource
DUT – device under test
GDOI – group domain of interpretation
GOOSE – generic object oriented substation event,
HMAC – hash based message authentication code
IED – intelligent electronic device
IKE – internet key exchange
ISAKMP – internet security association and key management protocol
IP – internet protocol
KDC – key distribution center
OCSP – online certificate status protocol
RBAC – role based access control
R-GOOSE – routable GOOSE
SMV – sampled values
UDP – user datagram protocol
Solution
There are several options for transition from layer 2 (L2) GOOSE to R-GOOSE, however one of the simplest for implementation is to use gateway which on one hand is a subscriber of the ‘traditional’ L2 GOOSE messages published by the IEDs connected to the local network, but on the other hand publishes received messages as an UDP/IP multi-cast traffic. The whole idea is depicted in the picture below.
The main tasks of the gateway are:
- Subscribe to the selected L2 GOOSE messages using one of the parameters: GOOSE message ID or name of publisher control block
- Generate UDP multicast packet containing GOOSE APDU from the received L2 GOOSE messages
- Optionally encrypt payload of the outgoing R-GOOSE multi-cast packet
For easier understanding of the whole process the picture below shows how the stack of L2 GOOSE maps to the protocols of the routable side.
The frame encoding details for L2 and UDP GOOSE messages are specified in IEC61850-9.1 and IEC62351-9 respectively.
Gateway itself can act as an IEC61850 server that subscribes to a multiple types of L2 GOOSE published by different IEDs within the local network. For each subscribed GOOSE, as a bare minimum, there should be defined destination UDP multicast address to which GOOSE will be sent.
Depending on the need of specific applications it is also possible to updated some parameters of the published R-GOOSE like application ID number, goose ID string or customize the delay between re-send messages.
As for the configuration of the gateway, it could be done statically via xml file, in which case once the xml config file is uploaded to the gateway, subscribers and publishers are initialized and could be either enabled or disabled by the IEC61850 client application, but the parameters of the subscriber (L2) side or publisher (routable) side, do not change until the new configuration is uploaded and instance of the gateway is restarted.
If required, it is also possible to modify during run time some parameters of the gateway:
- subscription attributes
- destination multicast addresses
- KDC server address
It depends on the requirements of the specific application and the customer using the gateway.
Challenges
While the overall idea might seem very simple (and actually it is), as the saying goes, the devil is int he details. Putting together all the puzzles, requires deployment of the right technology, tools and skills, especially if the goal is to have a flexible, scalable and salable product. In the following paragraphs I’ll scratch the surface a bit and briefly touch on the issues related to cybersecurity and performance, however this is by no means an exhaustive coverage of the matter.
Encryption
Since GOOSE messages published as UDP/IP multicast, potentially will be sent over wide area networks it is extremely important to protect the information against unauthorized access and modification. Considering the fact that GOOSE message could contain some sensitive information about the state of the critical infrastructure, this is good enough reason not to cut corners when it comes to cybersecurity. An obvious approach is to implement well know encryption standards like AES-CBC or AES-GMC. The details of how GOOSE data units (APDU) are encrypted are outside the scope of this article and could be found in IEC62351-9. AES is a symmetric encryption standard which provides sufficient level of security, it is not difficult to implement in software and thus is very popular and widely available. Some hardware platforms offer build-in (hardware) acceleration of the AES implementation which significantly increases the speed of encryption/decryption operation. Since this is symmetrical algorithm, both sender and receiver of the message must share the key, and secure distribution of the encryption keys is a non trivial task. Also the overhead related to crystallographic operations must be considered, when designing and implementing the gateway which shall serve several GOOSE ‘mapping paths’. Both distribution of encryption keys and issues related to performance are described below.
Key Distribution
The simplest way for distributing encryption/decryption keys among IEDs is to configure them statically on the device. It does not take a rocket scientist to figure out that there are many more cons of this approach than pros. However, one obvious advantage of this method is that it is simple to implement and thus could be used for prototyping and demonstration purposes.
For real life application however, a secure and automated way of key distribution must be used. The task is not trivial because of the nature of the GOOSE communication, involving multicast groups with potentially many nodes publishing or subscribing, which means that keys must be updated for all group participants in a coordinated way. The other challenge is that the crystallographic material must be delivered in a secure way, meaning that only authorized members of the group might receive it, and even if others intercept the communication, it should be impossible for them to determine the keys or temper with them. All these issues are solved by group domain of interpretation (GDOI) protocol which is based on IKE/ISAKMP. In short, each group member establishes secure communication channel with key distribution center (KDC), over which encryption/decryption keys for given group are provided. Normally KCD would periodically update the keys by sending to all active group members encrypted multicast PUSH messages. The details of the whole process are described in IEC62351-9 specification and number of RFCs: 6407, 8263.
Garibaldi tool is an implementation of KDC which handles routable GOOSE and routable sampled value. The product features a web page interface which makes it easy to configure and operate. It is scalable in terms of number of GOOSE / sampled value streams it can handle and implements role based access control (RBAC).
Certificate handling
Another area which must be addressed when implementing any solution involving Cybersecurity is handling of certificates. Currently certificates serve different purposes like
- authentication
- access control (RBAC)
- identity management
- asymmetrical encryption for session (symmetrical) key negotiation
In the context of GDOI, certificates are used when exchanging session encryption keys between the KDC and group members to authenticate each side and generating message signatures.
The most popular form of digital certificate is X.509, which contains, among others, details of the entity for which certificate was issued, public key and information about certificate issuer. Certificates are issued for specific time frame, and must be replaced once that time expires or if they are revoked by the issuer, e.g. if the private key is compromised. The challenge for the distributed system is to configure certificates of the nodes and manage their updates when necessary (expiry or revocation). Handling of revoked certificates could be implemented with certificate revocation lists (CRL) or Online Certificate Status Protocol (OCSP). There is no silver bullet here, each approach has its advantages and selection of the right one depends on the requirements of the specific application.
Performance
Since GOOSE (and sampled value) is used for delivery of time sensitive data, implementation of L2-GOOSE / R-GOOSE must take into account additional latency introduced by:
- routing and switching done by the IP nodes,
- encryption / decryption operations,
- IP routing.
Usually the customer asking about GOOSE performance, expects that the delay of GOOSE generation is below 4 milisec. Unfortunately not many of them realize what it does exactly mean and how to measure such delay. For the sake of demonstration of performance of some IEC61850 solution (module or communication) configuration consisting of
- laptop running Wireshark application and GOOSE publisher
- device with IEC61850 server and GOOSE subscriber/publisher
could be used. The idea is to send GOOSE messages from the laptop to IEC 61850 device under test (DUT) and setup sample ‘loopback’ application which would receive GOOSE sent by the laptop and immediately trigger generation of its ‘response’ GOOSE, e.g. by updating value of some attribute. After GOOSE sent by DUT is received by the laptop, Wireshark can provide delay between GOOSE sent by the laptop and the response which is coarse and very conservative measure of generation of GOOSE by DUT in response to some event.
For L2 GOOSE / R-GOOSE gateway situation is a bit more complicated because the assumption is that it has to encrypt the message, so there must be another GOOSE subscriber which decrypts the message. For the worst case scenario let’s assume that decryption of the GOOSE message is performed by another gateway device (DUT2) and decrypted message is distributed locally as a L2 GOOSE. Sequence diagram below shows two variants of end to end L2 / R GOOSE conversion, with and without encryption.
You have to be aware that there is also a nondeterministic factor related to the transition of R-GOOSE over IP network, related to the very nature of this communication. For the sake of simplicity, let’s assume that all the devices from the above picture are physically connected to singe switch. The assumption is justified since the goal is to assess the impact of introduction of two gateways (DUT1 / DUT2) which convert between L2 / R GOOSE and implement encryption/decryption.
The tests configuration used for the evaluation was like this:
- Windows 10 Laptop running L2 GOOSE publisher and Wireshark application.
- Raspberry Pi 3+ (DUT1)
- Raspberry Pi 3+ (DUT2)
- Fast Ethernet (100 MB) switch
The table below shows the latency statistics for different scenarios:
- L2 GOOSE sent to DUT1
un-encrypted R-GOOSE sent back to Laptop - L2 GOOSE sent to DUT1
encrypted R-GOOSE sent back to Laptop - L2 GOOSE sent to DUT1,
encrypted R-GOOSE sent to DUT2
un-encrypted L2 GOOSE sent to Laptop
The above measurement should give a rough idea about the overhead introduced by encryption and decryption operations. We have to remember that the test was executed on raspberry-Pi 3+ device which does not offer any hardware acceleration for cryptography. However for different architectures there are some options (e.g. AES-NI extension for x86, or ARMv8) offering significant performance improvement which should result in lower processing time.
Conclusions
I wanted to present my take on the issue of deployment of GOOSE protocol with distributed systems. Definitely it is not the only solution possible, and probably depending on the requirements of the specific use case or business cases it might not be the best one. Also the article only covers the issue of protocol conversion and cybersecurity of communication. There are number of other features which could be implemented on gateway used by the power system:
- mapping between different protocols
- redundancy of communication channels
- buffering and storage of periodic data.
However I hope this is a good starting point, at least for the conversation about the matter, so any feedback is much appreciated. If you are interested in trying evaluation kit feel free to contact me via LinkedIn or email (jmazan@jpembedded.com)