Since the start of the COVID-19 pandemic, we’ve all become aware of the supply chain issues facing the food and beverage and every other industry. Among the many pandemic-era product shortages was the case of cream cheese, which went missing from retail shelves last year just in time to put a damper on the making of cheesecake and many other holiday treats.
As it turns out, this cream cheese shortage was not the product of the typical supply chain issues seen by the industry. Instead, a ransomware attack knocked out core systems and operations for several days in October 2021 at Schreiber Foods. Not only was Schreiber on the hook for a reported $2.5 million in ransom, the attack wreaked havoc on peak production season for the large Wisconsin-based cheese producer. It was unable to conduct business as usual for days—a disruption that trickled down to smaller farmers, cooperatives, and companies that buy ingredients from Schreiber.
That’s just one example. Cyber attacks have hit the food and beverage industry hard this past year. In its State of Malware report, Malwarebytes clocked an eye-popping 607% surge in malware detections in the food and agriculture sector in 2020, with things dramatically settling down in first quarter 2021, yet still hovering at a notable 36% increase. One of the most high-profile incidents in 2021 was the strike at JBS Foods, a global meat processor that doled out $11 million in Bitcoin to the REvil ransomware gang and halted operations in more than a dozen U.S. processing plants, prompting meat shortages around the country. Smaller companies were also in the crosshairs. New Cooperative, an Iowa-based farm service provider, and farm co-op Crystal Valley were attacked by the BlackMatter ransomware group, while a U.S. bakery was one of more than 1,000 companies affected by the Kaseya supply chain attack as it lost access to key systems and had to pause operations.
|See the initial announcement of the cyber attack at JBS.|
The flurry of attacks on the sector prompted an official September 2021 warning from the FBI’s Cyber Division alerting food, beverage, and agriculture companies to the growing threat, especially as the adoption of smart technologies and Industrial Internet of Things (IIoT) applications increases the potential attack surface. Larger businesses are targeted because of their ability to pay higher ransom demands, the alert cautioned. But smaller entities are not immune because they’re generally viewed as soft targets, especially those in the early stage of digital transformation that aren’t as tech-savvy and versed in cybersecurity best practices.
“Cyber criminal threat actors exploit network vulnerabilities to exfiltrate data and encrypt systems in a sector that is increasingly reliant on smart technologies, industrial control systems, and internet-based automation systems,” the report reads. “Ransomware attacks targeting the food and agriculture sector disrupt operations, cause financial loss, and negatively impact the food supply chain. Companies may also experience the loss of proprietary information and personally identifiable information (PII) and may suffer reputational damage resulting from a ransomware attack.”
A changing landscape
The increasing number of cyber attacks on the food and beverage industry comes on the heels of threat actors targeting other critical infrastructure sectors, many of which are migrating from closed environments (often referred to as a walled garden) to networks of connected devices, equipment, and systems as part of efforts to digitally transform operations. Leveraging technologies like cloud, IIoT, advanced analytics, and artificial intelligence (AI) and machine learning (ML), manufacturers in this space aim to parlay terabytes of data long collected and stored in industrial equipment and systems into insights that will garner efficiencies, spark innovation, and optimize new business processes. Most share a common goal: to boost quality, improve plant performance and uptime, and enable predictive maintenance.
The global pandemic also sparked major changes to manufacturing operations that increased cybersecurity risks as remote access capabilities were used to accommodate personnel unable to physically be on the plant floor. “What accelerated with COVID-19 was remote access as not everyone could be on premise,” says Marilidia Clotteau, food and beverage marketing manager for the consumer packaged goods segment at automation supplier Schneider Electric. “Before, everything was in the plant. But when you start having a mix of on-premise, cloud, and connected devices, there are more potential vulnerabilities. There needs to be constant review and implementation of barriers to ensure the house is well kept, managed, and secure.”
While many in the food and beverage sector are hungry for Industry 4.0 applications to stake out a competitive edge, often, their existing production environments aren’t ready to digest new technologies securely. Most industrial control systems—from simple programmable logic controllers (PLCs) to more complex supervisory control and data acquisition (SCADA) and distributed control systems (DCSs)—as well as industrial networks were designed decades before cybersecurity was a major concern. As a result, many lack the basic encryption, authentication, and authorization controls along with automated asset management capabilities that are a staple of enterprise IT platforms. Moreover, the alphabet soup of proprietary protocols employed by industrial equipment, the landscape of siloed systems, and the lack of enterprise-grade monitoring tools makes it much more challenging to safeguard OT networks and assets compared with enterprise IT counterparts.
|Learn about the key challenges to using automation in packaging and processing operations.|
“Most of the industrial and control verticals weren’t developed with cybersecurity as a first principle, and plants were dependent on the enterprise to protect operations,” notes Mike Lester, director of cybersecurity strategy, governance, and architecture for Emerson Automation Solutions. “There’s now a spectrum of security capabilities and postures you have to deal with, but it hasn’t been the primary focus in this industry. That has been safety and control, and now there’s the cybersecurity wrinkle.”
Though there are federal and state regulations governing plant and equipment safety, they don’t yet extend to cybersecurity protocols, according to Colonel John T. Hoffman, senior research fellow at the Food Protection and Defense Institute (FPDI) based at the University of Minnesota and established by the U.S. Department of Homeland Security to pursue research, innovation, and education programs to reduce food system disruption. In fact, some U.S. Food and Drug Administration (FDA) rules, specifically those approval processes related to when devices or sensors are changed, are just cumbersome enough to act as a deterrent to upgrading to newer, more secure technology, he contends.
“The mentality in the OT world is: If it’s not broke, don’t fix it. And the result is legacy OT devices scattered through food production connected in many cases in totally illogical ways,” Hoffman says. Consolidation in the industry has made it even more difficult for OT and IT management to have visibility into exactly what equipment is in the plant, let alone have a complete understanding of potential vulnerabilities.
Not only is the number of devices installed in food and beverage plants an order of magnitude higher than other industries, the equipment is much more bespoke and varied. “There are a lot of very unique devices in the food world—for example, a device that cuts corn flakes into a finished product,” Hoffman explains. “That cutter may be unique to the company, been in use for 25 years, and no one wants to fix something that’s not broken.”
A blueprint for cybersecurity
Outdated equipment might still be functionally operational, but it packs a huge deficit when it comes to modern security controls, which means action is required. While escalating attacks have put cybersecurity concerns on the radar of top executives in food and beverage companies, as well as in other sectors, it’s still not a top concern for too many. In its 2021 industry outlook, tax, audit, and advisory firm Mazars USA found cybersecurity ranked surprisingly low as a top concern for business, with only 10% of respondents reporting they were “very concerned” about threats, up slightly between 2019 and 2021.
Nevertheless, there is a growing mandate to move forward, and the first step is to embark on a security assessment—either on your own or, more likely, with a qualified partner—to evaluate the current state of the organization’s OT infrastructure and to define clear cybersecurity goals. Implementing a controls firewall and network segmentation to ensure safe zones should be a central part of the evolving cybersecurity roadmap, along with standard processes for regular backup of data, including air gapping and storing password-protected backup copies offline.
Central to the FBI’s cybersecurity guidance to manufacturers in this sector is to create a recovery plan to maintain and retain multiple copies of sensitive or proprietary data and servers in a physically separate, segmented, secure location—a strategy that is also critical for business resiliency and continuity. Regular installation of software updates, including patches to operating systems and firmware, needs to be codified into ongoing cybersecurity best practices and system maintenance, the FBI alert says.
In terms of the technology stack, cybersecurity safeguards remain relatively consistent between IT and OT and, in fact, should be coordinated as part of a holistic security plan. Most experts suggest following the National Institute of Standards and Technology (NIST) Cybersecurity Framework, a set of industry standards and best practices to help organizations manage and mitigate cybersecurity risks. Technologies such as asset inventory and identification, network segmentation, endpoint protection, incident response planning, secure remote access capabilities, and real-time threat detection are key pillars of a modern security roadmap.
Longer term, experts expect advanced capabilities such as multifactor authentication and use of AI and machine learning to also become central to plant floor defenses. “You should expect to see more use of AI around user action so if an operator suddenly makes a change that’s outside of range, the system would automatically flag something like that as a potential unauthorized operation,” notes Brian Fenn, COO of Avanceon, an IT service management company and systems integrator.
Beyond the synergies, there are also key differences in IT/OT security requirements. Enterprise IT security has historically focused on confidentiality, integrity, and availability, but the order is reversed when viewed through the lens of OT priorities. “The plant doesn’t necessarily care about confidentiality as long as operations are safe and they’re still producing,” Emerson’s Lester says.
Specifically, there’s the problem of incident response time—an area where there are stark differences between what constitutes acceptable downtime for IT systems compared with OT systems. “If I’m a consultant responsible for the service level agreement (SLA) of a large food and beverage IT infrastructure, email could be down for eight to 10 hours on a weekend and it’s not overly critical in terms of dollar importance,” explains Brian Deken, connected services commercial manager for Rockwell Automation. “Yet if I’m down one hour on the plant floor, it could be hundreds of thousands of dollars. You need some sort of automated response system as part of real-time threat detection to have more rapid incident response and recovery.”
Bridging the IT/OT divide
Traditionally, security efforts related to OT and plant floor technology have been outside the purview of IT and the enterprise chief information security officer (CISO), if one exists. Even if there’s some coordination, there hasn’t been much in the way of a formal, shared roadmap. That OT/IT divide needs to close in order to adequately safeguard plant floor assets as they are synced up with enterprise applications and potentially cloud platforms as part of ongoing digital transformation.
While responsibility for OT security will vary depending on the food and beverage company, it’s important that IT and OT work together on cybersecurity initiatives. “The better they manage convergence, the better they do here,” Fenn says. “You’re trying to take principles and concepts from the IT space and make sure they’re applied in a way from an OT standpoint that will keep things running and stable and not cause other issues down the line.”
For example, OT systems, which are typically more isolated and have decades-long lifecycles, demand stability from a production standpoint so they can’t be managed and updated in the same manner as IT systems, where you can automatically push out an antivirus or applications update, Fenn explains. As a workaround, he suggests setting up a develop and test environment where OT and IT can work through application patches and antivirus updates to keep systems secure without having to take mission-critical production systems offline as part of the process.
“If you need to take down a legacy system to patch it, every moment it’s down is loss of money,” says Guilad Regev, senior vice president, global customer success, for Claroty, which markets an industrial cybersecurity platform that includes continuous threat detection and secure remote access solutions, among other capabilities. “If you create segmentation and redesign networks, it factors in all the pros and cons.”
Cybersecurity best practices
With the foundational technologies in place, food and beverage companies can begin to execute a cybersecurity roadmap that will ensure the right protections. Following these best practices will ensure the best results:
Conduct a complete risk assessment. It’s important to understand what’s out on the networks and how it’s all interconnected, but it’s also critical to perform a similar deep dive on the changing landscape from a controls perspective and to fully understand all the possible threat vectors. “Identify all scenarios by severity,” says Sree Hameed, industry marketing manager, consumer products for Aveva. Target controls to the machines with the greatest severity and the highest likelihood of breach—for example, the systems and machines that govern recipes, which would cause the highest levels of damage if breached, he explains.
Perform regular OT asset preventive maintenance checks. Check in regularly with vendors to determine when upgrades are coming out and regularly monitor log files to check for abnormalities. In that way, you can spot something in short order when it starts to go awry.
Invest in cybersecurity awareness and training. It’s not enough for enterprise professionals and key plant floor personnel to understand what’s at stake in the event of a cybersecurity breach—the greater organization needs to share the responsibility and be versed in an action plan for risk mitigation. Conducting cybersecurity awareness training a couple of times annually and requiring certification can go a long way in building up cybersecurity competencies across the organization. “Embed training into employee performance to foster a culture of cybersecurity in the organization,” Clotteau says.
Address the information gap with new talent. Industrial engineers in the food and beverage sector understand the unique protocols, proprietary systems, and uptime requirements of OT, but are unfamiliar with common cybersecurity technologies. At the same time, IT gets cybersecurity, but has no real knowledge of plant operations. To complicate matters, there’s lingering distrust between the groups. “Companies need to groom some unicorns,” says the FPDI’s Hoffman. “To be an IT security specialist in an OT world—that’s a challenge. Universities can help, but it doesn’t exist right now.”