Some days, it’s felt a little like being Captain Ahab, hunting the white whale. But, hopefully, this week it’s finally started to pay off. I’m talking about getting an Audio-Over-IP (AoIP) audio path from our studios in Providence to our “new” tower site in Tiverton for 89.3FM WNPN.

When we launched WNPN on September 1, 2018, technically we were using the “backup” Studio/Transmitter Link, or “STL”. It’s an APT/Worldcast “IP Codec”. Outstanding little box that takes AES digital audio in and creates multiple data streams over multiple internet paths. We use them for our STL’s on WELH 88.1FM and WNPE 102.7FM as well. They have fantastic redundancy; any one of the multiple streams can keep the station on the air at any given second, so it handles the inherent unreliability of public internet connections quite well.

But the chief downside to these units is that they pretty have to encode the audio using a lossy codec. We can’t send uncompressed, or “linear”, audio down to the transmitter, because it eats up at least 1.5Mbps of data, which is too much to reliably send over the public internet. We minimize the problem as much as we can by using the Apt-X codec at 192kbps, which sounds pretty good. But I want the best for our listeners.

There’s another reason, though, too. Our NPR satellite dish downlink is located at 1290AM in North Providence. We don’t broadcast on 1290AM anymore, and we can’t count on being to locate that dish there forever. Since there is no room anywhere at 1 Union Station to put a 12ft diameter satellite dish, the only other realistic location is at the Tiverton tower site.

If that comes to pass, we need a means of getting that satellite audio back to our studios with very little delay (three-tenths of a second, aka 300ms, or preferably less) and with top-notch audio quality.

That…has proven a challenge.

The obvious solution is to use the LiveWire AoIP streams embedded in our SFX4104 satellite decoders. They’re linear audio, thus very high quality. And the nature of LiveWire means it’s very low delay; in fact, it won’t work with anything above a few hundred milliseconds of delay, and less than 25 ms of jitter. And each LiveWire stream takes up about 2.5 to 5 Mbps (depends on the settings) per stream. Our 11GHz path is licensed for 50Mbps bi-directional so for four or five satellite feeds that’s no problem at all.

The rub has been that we couldn’t know in advance if the SAF Tehnika CFIP PhoenixC 11GHz radios would support these specs. In theory, and as we found in practice, they’ll do regular internet use just fine. But we’re trying to do something more ambitious.

First, we ran into all kinds of problems with our Cisco Catalyst switches. For enterprise-grade hardware (with enterprise-grade prices, I might add) these things were junk for this purpose. Exceedingly difficult to configure, missing a ton of features, and we later determined that for this specific model and firmware, there’s no way to reset them to factory defaults. After WEEKS of trying to figure out what settings we had changed that were causing so many problems, I declared them dead and moved on.

After much discussion with Axia, the creator of LiveWire, we purchased some Cisco SG350 switches. While these at least were a lot easier to work with, we quickly found they didn’t work well. Besides the VLAN setup being ridiculously complicated, every attempt to use VLAN’s either simply would not pass the LiveWire data, or it somehow passed it right back onto our main office LAN…which instantly swamped the LAN switches and took the office network down. Eek.

Eventually we gave up trying to use VLAN’s and settled for physically different wireless paths between the roof of 1 Union Station and the 24th floor balcony of the Omni Hotel, where our 11GHz link is located. One path, the “office LAN”, went over a pair of existing UBNT Nanobeams that have been up there for a few years now (this path also feeds one half of the STL for WELH) and it provides regular connectivity to the Tiverton site. Remote VNC login, web browsing, part of the backup STL, etc etc etc.

The other half we originally used UBNT AirFiber AF5’s. I picked these mostly because at the time they were the only UBNT offering that was MIMO (Multiple-In, Multiple Out) and thus supported full-duplex networking; sending and receiving data packets at the same time. That’s important with LiveWire, it doesn’t like the latency introduced by half-duplex networking (one end talks, the other end listens, and they switch off many, many times per second). The AirFibers never worked great, though. I think a lot of it is that an AirFiber is meant to transmit data over a 50 to 75 mile path, and we’re trying to use them for a path maybe 1000 ft (0.2 miles) long. It’s too focused a beam, too hard to aim, too difficult to keep power levels low enough that they don’t overload the receiver. All kinds of issues. Recently UBNT introduced the Gen2 version of the LiteBeam, which is only $50 each (vs. $1000/ea for an AirFiber…grrrr) and now it also supports MIMO. A LiteBeam can and often is used for links that’re pretty long…10 to 15 miles is not uncommon, and once I managed to connect a LiteBeam from the Omni Hotel down to the rooftop of Newport Hospital. It didn’t work worth a damn, but it did manage to connect and that’s well over 20 miles! But the smaller dish and weaker radio means a LiteBeam can and will work over much shorter paths.

Another trick we tried, after extensive conversations with SAF Tehnika about the 11GHz CFIP PhoenixC wireless link we purchased, was to make use of the SFP ports on the SG350’s and xSwitches (see below) with fiberoptic jumper cables into one of the SFP ports on the PhoenixC radios. The additional bandwidth isn’t really a factor (the link will only support 50Mbps max, as per our FCC license for the 11GHz) and the slightly lower latency is a nice bonus, but the real reason is that the SFP ports bypass the internal ethernet switch of the PhoenixC, and none of us were sure that internal switch would support the QoS that LiveWire needs.

We also found that substituting the Axia-made “xSwitch“…an ethernet switch designed specifically for use with LiveWire data…made a world of difference in allowing things to just work without fiddling with a zillion settings. Unfortunately they’re like $1200 each so we did have to make some of our Cisco SG350’s behave and ultimately I did. Mostly by stripping it down to one VLAN, and turning IGMP Snooping Status and IGMP Querier Status on in general, but disabling IGMP Querier Status on the VLAN in question. That let the xSwitches deal with the IGMP Queriering and they do that automatically; there’s no way to manually disable it on an xSwitch.

So far things do seem to be working. I’ve rigged up an HD2 broadcast on 89.3FM so the LiveWire path feeds it. That’s so I, and anyone with an HD Radio I suppose, can listen to it while at work or driving around and make sure the LiveWire audio doesn’t drop out. Beginning tonight (Sept 1, 2019) from 10pm to 3am the remote control will automatically switch to the LiveWire audio as a further test.

In a few more days, I’ll start it automatically switching to LiveWire during the work day to see how it goes. By then I should have another LiveWire stream going from the tower back to the studio for monitoring purposes, too, so we can test the bidirectionality of things.

And with luck we’ll get a thunderstorm or two, so I can see how well it holds up during heavy rain. We already know that during really heavy rain the 11GHz link signal drops like a brick due to “rain fade.” In the last year, I think three or four times it’s lost the connection entirely for a few minutes as a really heavy storm rolls through; even with a +40dB fade margin, when you’ve got a 20 mile long path, that’s a lot of water drops that can get in the way!

Hopefully within a week or two, we’ll be running the LiveWire path full-time as our primary STL. With luck, you’ll hear a little bit of an improvement in our audio quality. But it’ll also mean we’ll be ready for the day, should it ever come, that we have to move our satellite dish down to Tiverton.