Written by George Elgin, Nexum Principal Engineer

Our partners at Palo Alto Networks recently acknowledged PAN-OS 10.0.5 as a preferred release in the 10.0 code train. This announcement sparked a conversation with my peers about the general issue of code upgrades, which I thought the greater community might find interesting.

For this article, I’m focusing on Palo Alto Networks code, but many of the ideas carry over to other vendors. We will discuss why you want to stay current and then touch on different strategies for managing the code lifecycle for the Palo Alto Firewalls.

Why Stay Current?

This is a question I hear pretty regularly. Most engineers take an “if it ain’t broke, don’t fix it” approach to code upgrades. While there is some logic to this, there are solid operational reasons for maintaining concurrency. Some of these reasons include:

• Leveraging new capabilities
• Incorporating ongoing code improvements
• Regular exposure to HA testing lite (yes, not an actual technical term, more on that later)
• Maintain support access for your platform
• Reduce the pain when it is time to upgrade

So, let’s take a look at these.

Leveraging New Capabilities

Let’s consider the new features introduced in PAN-OS 10.0. There are 60+ new features across all areas of the PAN-OS domain; for the complete list, check out the PAN-OS new feature guide here.

I’m pretty sure there is something in that list that will make you go, “Huh… I could really use that.”

In addition to new security capabilities, there are several operational improvements, logging enhancements, and UI enhancements, all of which help incrementally improve your experience with the platform while adding tools to aid in defending your network.

Incorporating Ongoing Code Improvements

Some things are advertised in the release notes; sometimes, things are quietly addressed that aren’t considered defects per se and yet improve the system’s operational efficiency. By maintaining your code on a preferred release, you are taking advantage of those incremental improvements that are not always advertised.

HA Lite Testing

I say HA “lite” as the code upgrade process is going to require a reboot. That means that a passive HA box that may have lost its magic smoke now needs to prove it’s ready for the task.

This approach also allows you to engage in a more aggressive HA test as part of the upgrade process. Validating that your path and link monitoring is working as expected will reduce the likelihood that you get a surprise call when it fails to capture a failure condition. That’s never a good call.

You will be in a window for the upgrade, might as well build confidence in the operational integrity of your firewall deployment!

Maintain Support Access

Palo Alto Networks, like most vendors, have a support lifecycle for software. You can review that process at the End-of-life Summary page.

Most of us have been in that situation; you are working on a weird problem, no distinct pattern of behavior, no consistent periodicity to the problem, so you open a TAC case to explore the issue. The TAC engineer gets on the session, sees you are running on an older minor version, not the current “preferred” version (which Palo Alto can be is on the PAN-OS Release Guidance page), and they request you update to the “preferred” version.

A more dire form of this occurs when you are running code that is past its End of life (EOL). In the previous example, you could arguably push back; however, they won’t assist if it’s past its EOL until you are on supported code. Now you need to walk through the process of getting to a supported version.

That’s not going to make for a short night.

Reduce the Pain When It Is Time to Upgrade

Going back to that last situation, now you need to upgrade to get to that supported version. How many steps are involved and how long will it take? That’s going to depend a bit but it will be a lot longer than you want it to take, particularly if the sky is currently falling!

To get a sense of this, consider the recommended upgrade path described on Palo Alto’s support page. If you are covering more than one major version, you are doubling your time to get to the endpoint, two major versions… triple the time… Even if you are all lined up with panorama ready to push new code versions, that’s going to take a bite out of your evening.

What Does That Mean For Me?

Palo Alto Networks, like most vendors, typically has multiple supported versions across several major versions of code. As an example, the currently supported code versions of PAN-OS include:

• 8.1, EOL March 1, 2022
• 9.0, EOL February 6, 2022
• 9.1, EOL December 13, 2023
• 10.0, EOL July 16, 2022

Now take a hard look at those dates, notice that the X.1 code of the previous major release outlives the X.0 code of the next major release. That’s an important thing to note. The X.1 versions are major releases but are all about rolling up improvements and fixes from the X.0 releases. They also tend to have fewer net new features. This implies and is our experience that the X.1 releases are more stable out of the box (though you should still wait for a preferred version!) and achieve that preferred status sooner in their lifecycle than the X.0 releases.

As a result, there are two general strategies for PAN-OS lifecycle management: Stability Focused and Early Adopter.

Stability-Focused

In this approach, we move from X.1 code to X.1 code while following the minor releases within the current train. In English, that means, as an example, we would move from 8.1 to 9.1 in production. The upgrade, when it happens, will take a touch longer, but you will be moving between more stable versions. While using a particular code, minor upgrades (9.1.1 – >9.1.5) are executed to keep pace with the TAC preferred code for a given train.

Early Adopter

Early Adopter may be too strong a phrase, but the idea is we track the code for each feature release as they achieve a preferred status. Given that 10.0.5 is now a preferred version in the 10.0 train, you would upgrade to this code, gaining an earlier ability to benefit and leverage the new features in 10.0 vs. 9.1. When 10.1 comes out and achieves that preferred status, you would move to 10.1.x to leverage its new capabilities.

Which One Is the Right Approach for Me?

That is going to be a conversation as each environment will have different levels of risk tolerance. Even within an organization, there might be some systems that we can accept the risk of running newer code vs. other environments in which we take a stability-focused approach.

How Nexum Can Help

Nexum offers Professional Service options to assist in understanding how you can approach this, and given your unique environment, how to establish an ongoing management approach for your PAN-OS code. If you don’t have the time or capacity to manage your own upgrades, Nexum will work with you to design recurring service engagements to let us bring our expertise to your environment on a regular schedule. We can make sure that your Palo Alto Networks products are not just up-to-date but also work with you to identify improvements in your deployment to make sure you’re maximizing the value of your investment. In addition to professional services, we also offer first*defense, our MSSP, which can offload even more of the day-to-day and let you focus on business-impacting improvements.

Contact us for more details.