It’s been quite a while since my last post (almost four years!), so a lot has changed.
I spent the last 4.5 years working as an architect for Veristor – a consultative integrator focused on the south east US. I have nothing but good things to say about that team. It was a great work environment and I got to work in a dizzying array of environments with an immense technology tool set at my disposal. My responsibilities ranged ranged across troubleshooting deauthentication floods, architecting ZTNA solutions, building AR-ready WLAN designs meant for industrial environments, helping with global SD-WAN rollouts, rebuilding WAN edge topologies, CDN and ADC architecture and implementations, datacenter microsegmentation, the list goes on and on.
But two weeks ago I made the jump to Juniper as a senior systems engineer focusing on supporting the Mist platform. The Mist technology set is extremely compelling with a lot of innovation – I really believe that they offer an excellent solution – and I’m really excited to see what the future holds.
So, quick disclaimer before I get into the more technical bits… This post is not meant to be an all-inclusive overview of Juniper Mist. There are many folks out there who can do that much effectively than I can on this simple blog. This is a blogpost written by a wireless enthusiast written for other wireless enthusiasts in the interest of sharing some of the lesser known yet really cool features of the Mist platform that I’ve explored in my first two weeks here.
The Architecture is Really Flexible and Responsive:
I’ve built a lot of WLAN systems in my time. Some were straightforward, some were… not so much. Depending on a customer’s requirements there were times where a ton of complementary systems had to be added in to make the WLAN system fit into the existing network. As I work through the Mist dashboard on several opportunities I keep finding these features that would have really made my life easier had I had them in the past on other engagements. This system really can scale. There’s no hidden “gotchas” lurking that wrench you from one config style / architecture set to another.
One thing that I’ve really liked? The boot time on the APs is ridiculously fast. I’m talking usually less than 30 seconds in total. Also, each AP can operate independently and the firmware versions do not have to match at a single site for things to function properly. I prefer to standardize on a single firmware revision at a site – so I’m not touting this version flexibility itself as the must-have feature – but this version flexibility results in a comparatively painless upgrade process. Mist uses a “Rolling Thunder” upgrade sequence that upgrades several APs at a time at a site in a sequenced way that minimizes client disruption. There isn’t any site-wide outage during an upgrade, there isn’t any complex multi-controller clustering requirements to enable ISSU, so on and so forth.
On a similar point, the Mist cloud is micro-services based and very resilient / responsive. Upgrades are rolled out every two weeks like clockwork.
Ease of Installation / Operations:
The BLE array embedded into each AP is used for more than location services.
Consider this alert from the Mist cloud:
I had this pop up while doing an installation the other week. Every Mist AP has an extremely comprehensive list of status codes it can blink on the single LED to help you troubleshoot – here’s a partial list to give you an idea:
In my case, I saw three yellow blinks on the AP, which means that the AP couldn’t get an IP address. It turned out to be an issue with the cable and it was quickly resolved.
But what if I wasn’t onsite to see the LED codes? How would the cloud know the cause of the outage? For those of you familiar with other cloud managed network platforms you recognize the all-too-common “chicken or the egg” scenario I’m describing. Cloud management is great when ZTP works, but when it doesn’t work you can get yourself into a bad spot because you need the cloud to configure the box, but you can’t get the box to the cloud to configure it so that it can reach the cloud in the first place.
Turns out that Mist APs use the built-in BLE array to relay information between each other. In this case, the AP that was orphaned began broadcasting out via BLE that it was in a bad spot. The other Mist APs in the environment heard about its woes and then informed the cloud platform about what had happened.
It’s a small but thoughtful thing. There are many other similar thoughtful things in the system too – like using the installation app to take pictures of each AP and switch during installation and displaying those pictures in the dashboard to make finding the gear in the future much easier – or the ability to import Ekahau files directly into the cloud platform – and all those small things add up to improve the quality of life of the networking team.
Marvis Query Language:
Just about every WLAN vendor out there these days is touting their AI capabilities. It can be difficult to cut through the noise and actually quantify just how useful the AI engine actually is without hands on experience with each platform.
In my opinion the AI engine is only as good as the data that it has been fed and the Mist platform ingests a lot of data. The Mist system tracks a huge variety of client states for every connected client:
Every state is tracked for every client and reported to Mist every minute. These are tracked with a high level of detail as well – for example, here’s an Authorization Failure:
The Mist APs utilize a ring buffer paired with their dedicated third radio to provide dynamic packet analysis as well – here’s the frames captured for the Authorization Failure above:
So, that’s a lot of “setting the table” so to speak. You can see that there’s a lot of data being tracked by the platform. What does this mean for Marvis, Mists’s AI engine?
In my experience, it means that you can very effectively troubleshoot your network. Rather than simply being told “there’s DHCP problems at site XYZ” or “might want to check your DNS server at site ABC” you can dive deep into the data and find root cause analysis. For example, what component of DHCP is failing? Is it the DISCOVER or the REQUEST? Is it tied to a specific client type or a specific server? Is the DHCP issue correlated with broadcast or unicast requests? Are all DNS queries failing, or just to a specific domain? All this is accessible by the administrator. You can directly interact with the dataset that feeds the AI engine.
Quick disclaimer, most of this data can be digested in a very easy way using other elements of Marvis – if Marvis detects an anomaly it will highlight the issue and provide multiple levels of correlation, both in Insights and in Marvis Actions. I’m not trying to say that you need to get into Marvis Query language every single time an issue pops up.
But if you really want to roll up your sleeves and get into the nitty gritty details the Marvis Query language is very powerful.
For example, let’s say that I want to trim out some datarates to optimize my network – but before doing so I need to identify clients that are constantly utilizing 802.11b rates. I can run the following query:
RANK Clients BY ClientEventCount WITH Protocol b DURING “Last 7 Days”
Or let’s say that I want to troubleshoot DHCP Timeout issues on a particular SSID:
LIST ClientEvents WITH EventType DHCP-Timed-Out AND WLAN sample-network DURING “Yesterday”
Or let’s say that I want to troubleshoot roaming for a particular client who complained about dropped calls yesterday:
ROAMINGOF “sean.freeman” DURING “Yesterday”
“ROAMINGOF” is particularly cool – it brings up several ways to digest the roaming performance of a particular client. I can see how this client roamed through the day, whether it was a healthy or an unhealthy roam, etc etc:
All this information is gathered as part of the standard Mist platform operations… you don’t need to turn on any manual debugging or anything like that. It’s baseline.
The Config Engine is Flexible:
Let’s consider a common scenario when designing a distributed WLAN network. Say that you have several hundred branch offices, all of which should use a similar set of WLAN configs… same SSIDs, same client VLANs, so on and so forth. But there’s one key exception, as there are two service hubs that provide authentication services for corporate clients, one for the East US and one for the West US. So the primary RADIUS server needs to change between the two config groups.
In some WLAN systems this means creating two separate configuration groups. Two configuration groups is somewhat annoying but it’s not a huge deal, just something that has to be kept in mind when you roll out changes in the future. But what if there’s an acquisition several months later and you need to bring in a new SSID just for those offices that are rolling into your infrastructure? Now you have three or potentially four configuration groups. As more complexity is added and as the network grows a standalone “configuration group” approach becomes more and more cumbersome.
Mist solves this through the use of config templates and through configuration variables. Configuration templates are used to apply uniform policy across multiple sites. These config templates can use variables for key parts of the network policy – for example, here is a snippet from a configuration template that I am creating for an SSID that is intended to be broadcasted at multiple locations:
Notice that I have defined the RADIUS server within the config template as a variable labelled {{AUTHSERVER}}.
Then within the Site-specific config, I create a variable entry that will look for {{AUTHSERVER}} from the config template and give it the value of my local 802.1X service, 10.0.1.217.
The end result is that this particular site uses 10.0.1.217 as the authentication server – but other sites, using the same config template, can have different authentication servers specified through the use of unique variables. This way I can avoid configuration sprawl.
I’m at risk of making an overly long blogpost, so I’ll put a pause on this for now. Future blog entries will be more technical in nature. To sum it up, there’s some pretty exciting and innovative things within the Mist architecture – I haven’t even touched the scripting components – and I look forward to getting further and further into the platform.
Packet Wrangling 