pfSense

This is mentioned at [[http://support.ntp.org/bin/view/Support/ConfiguringJupiterRefclocks]]

See NTPD refclock 31 page: [[https://www.eecis.udel.edu/~mills/ntp/html/drivers/driver31.html]]

Swapped out PSU rail on my GPSDO for a discrete external PSU. I think it's still losing characters, however pfSense's init was able to push it into Zodiac binary mode with my patch.

Most likely I need to swap the serial cabling. C2G p/n 52147 would be ideal (low profile connectors for something I'm feeding back into the case), but C2G p/n 81378 is what's available in the UK and on order.

I notice that there is no way to set the termios bits directly w/o hacking code. It's a possibility I may have to do that here, e.g. the base64 blob in my patch assumes ONCLR is not set, and hardcodes CR+LF endings.

Yup. I have the u-Blox 5 also in an ALIX 6D2 (older rev) and have the necessary leads soldered for 1PPS and UART. But that's another target. (Also thinking of IRIG, but -- on a budget -- that needs a Winmodem for the MiniPCI slot, and access to the raw PCM codec.)

This area could use some more work anyhow. I have to fight to get my USB GPS to be recognized at all. It appears we need two bitrates, one for initialization and another for actual use. Also the backend code that sets up the gps0 entry in /etc/remote is weak and needs to be improved. It only writes the line if it's not in the file, it doesn't update it once it's there even if the speed is changed.

Bug in present patch: '$PRWIINIT,A,,,,,,,,,,,,,\r\n' is an absolute reset, losing the date. This may have caused intermittent problems with $GPGGA sentences. I'm still evaluating.

When I had this unit working under LinuxPPS yesterday in NMEA mode (which unfortunately on Jupiter, has nondetermistic latency between NMEA sentence and PPS signal), setserial & printf commands were needed. The major difference in LinuxPPS: for Refclock_NMEA, /dev/gpsppsN must point to the /dev/ppsN device, NOT the serial device.

The Jupiter can only be fully configured through ZODIAC binary. gpsctl/gpsd support may have bitrotted. I seem to remember having to configure the unit through a bitwise 'printf'. New cabling has arrived; yet to try.

Swapping out the cables DID help. Always ensure you're using a high-quality, shielded serial cable for talking to a GPS used for timing.

OK. What appears to be happening is that -- by default -- the Jupiter is speaking NMEA at 4800 baud on boot.

If one attempts to use the NMEA Refclock driver (127.127.20.x) with PPS enabled, what will happen is oscillation between +/- 2.000uuu seconds out of sync.

ntpd will eventually declare it a false ticker, provided you have other references configured. I typically point at 2-3 well known and reliable UK sources.

This is expected behaviour. The Jupiter is documented not to deliver the second pulse accurately in time with NMEA sentences.

It does however guarantee it will be synchronised when in ZODIAC binary mode, but -- here is the catch -- only when running at 9600 baud.

The kicker is that even if you throw the device an RBIN command to enter binary mode, it will still be in 4800 baud. It can only be told to use a different baud rate using ZODIAC commands.

(I double confirmed that the device was speaking ZODIAC correctly at 4800 baud using gpsdecode from the gpsd package. ... It's all coming back to me now -- it's been around 5+ years since I set up this particular unit for high precision.)

So in my case, despite the fact I have now got a pair of Rockwell-proprietary NMEA sentences which initialise the unit and put it into ZODIAC mode, I am only halfway there.

Whilst the Jupiter refclock (31) driver is hardcoded to expect data at 9600 baud (and requests this rate with POSIX termios), I can use a rather gnarly hack^W^W^W truly cantankerous workaround specific to FreeBSD: use the /dev/cuXXX.init and /dev/cuXXX.lock device nodes to enforce 4800 baud operation initially, e.g. before starting ntpd:

stty < /dev/cuau0 speed 4800
  stty < /dev/cuau0.init speed 4800
  stty < /dev/cuau0.lock speed 4800

With this, I get the magic line which indicates the refclock driver has (initially) understood the clock:

Jul 9 01:00:10 atom ntpd⁸⁸²²⁵: jupiter_receive: 12 chan ver 01.80, 11/26/97 (0003)

But it takes 7 minutes for ntpd to notice the PPS is out of sync:

Jul 9 01:07:14 atom ntpd⁸⁸²²⁵: GPS_JUPITER(0) 802b 8b clock_event clk_bad_time
Jul 9 01:08:10 atom ntpd⁸⁸²²⁵: GPS_JUPITER(0) 8034 84 reachable
Jul 9 01:08:10 atom ntpd⁸⁸²²⁵: GPS_JUPITER(0) 904b 8b clock_event clk_unspec
Jul 9 01:09:02 atom ntpd⁸⁸²²⁵: GPS_JUPITER(0) 901b 8b clock_event clk_bad_time

It's quite possible that this is because of the baud rate mismatch. I seem to remember that the Jupiter is very picky about how it gets read. I wasn't able to get ntpd to see it after that.

The mode lines (corresponding to the checkboxes on the Serial GPS pane of the Services -> NTP config) have different meanings for Refclock 31 and not 20. E.g. "Enable kernel PPS clock discipline (default: checked)." actually means PPS on clear. (For the Jupiter, PPS capture should be on assert.)

What does this mean for time source support in pfSense more generally? The GPS Initialization actually needs to have at least two phases.

This is not limited to the Jupiter by any means. Many GPSes present NMEA for compatibility on boot, and then when tuning the device for high precision timekeeping, they require the vendor's binary protocol to be spoken directly.

In my case, the interim workaround would be a small shell script in the Shellcmd package which delivers the NMEA init sentence, and the correct binary init payload.

Something in the mix keeps setting the baud rate to 4800, though -- overriding /dev/cuau0.lock settings. My guess is this is something the PHP code triggers, even though that setting doesn't match what was set in the "baud rate" dropdown list box (and those settings aren't valid for Refclock 31 anyway -- 9600 or bust.)

Proceeding under the assumption that refclock_jupiter.c may have bitrotted, I discovered that there is not a snowball's chance in hell that gpsd's PPS sampling could work under FreeBSD, with the present port.

1) gpsd port does not build with PPS support by default;
2) there was a typo in the PPS sampling thread logic which may have caused perfectly good PPS API state capture on FreeBSD to be rejected.
3) as a result, the second NTP SHM segment exported by GPSD was never being updated on FreeBSD. The primary segment only has microsecond accuracy and is not corrected for 1PPS.

This is what I observe, but cannot confirm the fix yet. Our 95%-ile fix was anything +/-20us UTC, from boxes in NY and Chicago slaved to CDMA GPS time, when the box was deployed. The GPSD NTPSHM segment has noticeable jitter when run without PPS. Whilst I haven't run a full analysis, I have eyeballed jitter in the +/-400us range peak...

I'm still having some receiver issues, however...

I can get the higher quality SHM refclock derived from PPS in GPSD -- 127.127.28.1 -- working on FreeBSD. However, GPSD 3.16+ is required; the port is currently at 3.14.

There were indeed bugs [[http://git.savannah.gnu.org/cgit/gpsd.git/commit/ppsthread.c?id=f6fbc5985a349874d92234c7c89bd634f9020b2f]] which, under LinuxPPS, would have fallen back to using TIOCMIWAIT to poll the PPS line (as opposed to RFC 2783 PPS API).

I'd like to get more in-depth testing. I am using a simple active sat-nav 'mouse' antenna for cars, which has worked well with the u-Blox 5. However, it is possible the Jupiter is still configured for a passive antenna. I am testing the appropriate ZODIAC config message.

Interoperability between NTPD (refclock_jupiter) and GPSD is almost impossible, because GPSD requires a location fix before it will output ticks based on PPS (through NTP SHM or otherwise). To do this it requires a full $GPGGA fix (Message ID 1000 in ZODIAC binary). By contrast, whilst refclock_jupiter requires this, it turns these messages off (requiring a reset of the receiver to restore).

(you may wonder why I bother -- it's because receivers phase-locked to a high quality OCXO source like this require sourcing these days... and are often a capability of older GPS receiver modules, not the mass-produced consumer items...)

P.S. Those of us who are only using NTPD for reference clock support (and time distribution), and/or plan to run IEEE 1588 anyway, may be interested that chronyd is (effectively) claiming sub-microsecond sync in software (based on public (presumably non-rigorous) measurements).

It is known that this is implemented using UNIX domain sockets on Linux. My best guess is that results on FreeBSD might benefit from netisr -- provided it can be guaranteed that suitable thread(s) are affine with the queue(s).

If the technique is portable, it might be a good starting point for integrating reference clocks into PTPd (or perhaps even RADclock).

I seem to have a stable fix with 5-6 PRNs now. This is comparable to the uBlox5 unit (pfSense 2.3.1, i386, ALIX 6D2) which normally runs from the same antenna. Spare antenna on order to run them both at once.

The magic for active antenna mode (GPSDO always run with Symmetricon full-on L1 antenna in the past) is:

# ZODIAC Binary: Message 1218, Set Active Antenna.
_zod_1218_set_active_antenna='\xC2\x04\x00\x00\x01\x00\x00\x00\xff\xff'
gpsctl -f -t Zodiac -x $_zod_1218_set_active_antenna ${zoddev}

I can confirm that I have kernel PPS-synchronised timestamps from GPSD 3.16 in the SHM page, however I have not turned NTPD back on. As time permits I should get more data.

Now that the fix is stable, refclock_jupiter would probably also work, but I'd rather not risk it. I've left the Jupiter itself in NMEA mode so gpsmon can monitor it under screen(1).

It would be really useful to have ZODIAC monitoring in GPSD, so as not to break the deterministic time-code / time-pulse time domain behaviour.

Ironically, IEEE 1588 provides for this loss of fix by allowing a clock to advertise that it's lost its primary reference (though it may still have a high-quality oscillator for holdover, as in the above picture) -- by changing its clock class.

It may be that the best way forward is to go with GPSD instead of NTPD refclock_nmea. The NTPsec people broadly overlap. Needs evaluation, though, and we'd ideally like to support PTPd too -- we have switches, hungry for high quality time.

OK. I'm having trouble with the uBlox5. Specifically, it is difficult to force the unit into a binary-only mode; it defaults to a combination of NMEA (not wanted) and UBX (wanted).

If I use the CFG-MSG command (Pg.65 of u-Blox 5 comms spec GPS.G5-X-07036) to disable NMEA messages on USB, (similar to the existing initialization commands pfSense uses in NMEA mode), this appears to have no effect.

If I use the CFG-PRT command (Pg.52 of ibid.) to force the USB port into UBX only mode, I lose communication with the device. (It could be that I have the bitfield's byte order wrong -- I am assuming big-endian.)

Example: gpsctl -t u-blox -x '\x06\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x01\x00\x00\x00\x00' /dev/cuaU0

On the ALIX 6D2, this is difficult to recover from without power cycling the whole unit. The usbconfig(8) command does not generally allow you to power-cycle the root hub. This could be a limitation of the OHCI+EHCI interfaces for the AMD Geode.

A more likely explanation is that the power lines to the MiniPCIe card cannot be directly controlled, as they are probably independent of the USB hub in the Geode's I/O support hub.

Indeed, UBX messages are little-endian by definition. I'll have to revisit this -- being dragged into other things at the moment.

According to Pg. 10 of [[http://www.pcengines.ch/schema/alix6b.pdf]], there is no way to power-cycle the MiniPCIe USB slot on ALIX.6 -- it is connected directly to the main 3V power stage.

I hate little endian. Endian little hate I. This is the endian-fixed CFG-PRT packet. I get only UBX now, but I don't get all the messages needed for gpsmon(1) to tell me about a fix.

gpsctl -f -t u-blox -x '\x06\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x01\x00\x00\x00\x00\x00' /dev/cuaU0

Success pushing uBlox5 into binary mode; but don't let gpsd write to
the GPS (-b switch) just in case. I got NMEA again a few times before
it settled down. The last CFG-MSG needed to be run twice to take effect.
Reason unclear. Will try again on a cold boot.

(Currently not getting a fix -- too few PRNs in view, may try tweaking the SBAS settings as Astra-5 is in view.)

GPSD itself logs everything in JSON, so integration into pfSense
webConfigurator should be relatively easy compared to NTPD.

We don't have 1PPS here to go on, and that code has not yet hit mainline
FreeBSD. Whilst I've made the necessary hardware modifications, I've had
boot problems if I leave the 1PPS connected to the I2C header on ALIX.6.
(My guess is this interferes with the memory controller on the Geode LX.)

So, 1PPS would have to be fed in on another GPIO; e.g. the LED lines.
This may also require removing the "KITT style" LED flashing on boot
which pfSense does (and is useful to know the device is booting).

At the moment, I don't have a fix. This is no worse than the situation
with the Rockwell Jupiter. However, I do have PRN 123 in view.
According to [[https://en.wikipedia.org/wiki/European_Geostationary_Navigation_Overlay_Service]] this is Astra-5 -- a well-known European comms bird,
and its SBAS feed is in testing.

So, here's a list of the configuration I've applied.

When ready, run GPSD with:

 gpsd -n -b /dev/cuaU0

And monitor the UBX interpreted output:

 gpsmon

CFG-PRT: For USB port, accept UBX and NMEA, emit only UBX.

 gpsctl -f -t u-blox -x '\x06\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x01\x00\x00\x00\x00\x00' /dev/cuaU0

CFG-MSG: Enable all UBX messages.

 gpsctl -f -t u-blox -x '\x06\x02\x00\x00\x00\x00\x00\x00\x00\x1F' /dev/cuaU0

CFG-MSG: Disable all NMEA messages.

 gpsctl -f -t u-blox -x '\x06\x02\x01\x00\x00\x00\x00\x00\x00\x00' /dev/cuaU0

CFG-RXM: Max performance mode.

 gpsctl -f -t u-blox -x '\x06\x11\x00\x00' /dev/cuaU0

CFG-TP: Configure for 1PPS.
On +-ve assert, UTC seconds, no antenna/group/user delay factor,
undisciplined seconds permitted (!), 100us pulse length.
(1PPS is NOT supported on USB DCD; and this is not a bridged legacy device.)

 gpsctl -f -t u-blox -x '\x06\x07\xE8\x03\x00\x00\x54\x00\x00\x00\x01\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00' /dev/cuaU0

CFG-TP: Configure for 1PPS.
Same but only asserted if UTC discipline is guaranteed:

 gpsctl -f -t u-blox -x '\x06\x07\xE8\x03\x00\x00\x54\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' /dev/cuaU0

I have had poor results with this so far:

CFG-NAVX5: Allow 2D initial fix, 3 min PRN, 15 max PRN.
(This may be required for 2D-only fixes to work.)

 gpsctl -f -t u-blox -x '\x06\x23\x44\x00\x00\x00\x00\x00\x00\x00\x03\x0C\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' /dev/cuaU0

CFG-NAV5: Configure for stationary 2D-only fix mode.
(I don't have a timing model receiver, so I need 3 PRNs minimum to fix.)

 gpsctl -f -t u-blox -x '\x06\x24\x05\x00\x02\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' /dev/cuaU0

To back out the 2D-only fix change:

CFG-NAV5: Configure for auto 2D/3D fix mode.

 gpsctl -f -t u-blox -x '\x06\x24\x05\x00\x02\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' /dev/cuaU0

Also useful are:

CFG-RST: Controlled GPS software restart with a hot GPS start.

 gpsctl -f -t u-blox -x '\x06\x04\x00\x00\x02\x00' /dev/cuaU0

CFG-RST: Controlled GPS software restart with cold start:
(NOTE: This wipes the cached ephemeris data from EEPROM. Subsequent
fixes may take a very long time.)

 gpsctl -f -t u-blox -x '\x06\x04\xff\xff\x02\x00' /dev/cuaU0

CFG-DAT: Set datum to WGS84. (This is the factory default.)

 gpsctl -f -t u-blox -x '\x06\x06\x00\x00' /dev/cuaU0

I now have a fix. I traced this back to an error in the NAVX5 message.
There is a 2-byte version (0000) in front which I fat-fingered.

CFG-NAVX5: Allow 3 min PRN, 20 max PRN.

 gpsctl -f -t u-blox -x '\x06\x23\x00\x00\x44\x00\x00\x00\x00\x00\x00\x00\x03\x14\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' /dev/cuaU0

Lacking a means of power cycling the MiniPCIe-USB unit, I forced a
Hardware Watchdog Reset. /dev/cuaU0 punted to /dev/cuaU1, and FreeBSD
did not recycle the old device node:

 gpsctl -f -t u-blox -x '\x06\x04\x00\x00\x00\x00' /dev/cuaU0

Putting it all together, this got a fix immediately after hardware reset:

gpsctl -t u-blox -x '\x06\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x01\x00\x00\x00\x00\x00' /dev/cuaU1
gpsctl -t u-blox -x '\x06\x02\x00\x00\x00\x00\x00\x00\x00\x1F' /dev/cuaU1
gpsctl -t u-blox -x '\x06\x02\x01\x00\x00\x00\x00\x00\x00\x00' /dev/cuaU1
gpsctl -t u-blox -x '\x06\x11\x00\x00' /dev/cuaU1
gpsctl -t u-blox -x '\x06\x07\xE8\x03\x00\x00\x54\x00\x00\x00\x01\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00' /dev/cuaU1
gpsctl -t u-blox -x '\x06\x07\xE8\x03\x00\x00\x54\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' /dev/cuaU1
gpsctl -t u-blox -x '\x06\x23\x00\x00\x44\x00\x00\x00\x00\x00\x00\x00\x03\x14\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' /dev/cuaU1
gpsctl -t u-blox -x '\x06\x16\x03\x07\x01\x00\x08\x00\x00\x00' /dev/cuaU1
gpsd -n -b /dev/cuaU1

he very last gpsctl line localises the Satellite Based
Augmentation Service (SBAS) for my location (London, UK).

North American users may wish to adapt the following UBX command
to configure WWAS satellite reception.

CFG-SBAS: Enable 1 SBAS channel for EGNOS in Western Europe: Astra-5.
Allow ranging/integrity/DGPS. Allow birds in testing.
Include only satellites known to be in my view.

 gpsctl -f -t u-blox -x '\x06\x16\x03\x07\x01\x00\x08\x00\x00\x00' /dev/cuaU0

For scanmode1 (Pg. 93 of Ibid.):
Bit 0: PRN 120: Inmarsat 3-F2: Not in view, clear.
Bit 3: PRN 123: Astra 5B, set.
Bit 16: PRN 136: SES-5. Not in view, clear.
(PRN 120/136 can be re-enabled here when we have an L1 reference antenna
with a full sky view.)

Notice I also said Europe and not some separate thing.

Woops. Forgot to update the comment about 1PPS. We need it to supply only UTC seconds, and that's what the blob does.

However, it will be up to GPSD to notice -- and send a trap to -- PTPd or NTPd to inform it that the primary reference was lost or degraded.

I'd rather it DIDN'T pull the SHM1 segment away (127.127.28.1 to NTPD) so we can keep its more precise data; 127.127.28.0 is only good in microseconds, and only updated once per second, and that is the major weakness in NTP SHM + GPSD right now. For a GPSDO, drift is likely to be very low.

IEEE 1588 can tolerate this, in fact there is good data from Dr Leon Lobo showing that when the primary reference is re-acquired, the variance is on the nanosecond scale.

Using gpsctl to initialize the GPS is rather slow, due to the repeated auto-detection (even when the device type is forced).

Attempting to use the '-e' option is buggy, as it appends a debugging message in ASCII to the rendered binary blob:

# gpsctl -e -f -s 9600 -t u-blox -x '\x06\x16\x03\x07\x01\x00\x08\x00\x00\x00' /dev/cuaU0 | & xxd
00000000: b562 0616 0800 0307 0100 0800 0000 37f9  .b............7.
00000010: 2f64 6576 2f63 7561 5530 2069 6465 6e74  /dev/cuaU0 ident
00000020: 6966 6965 6420 6173 2061 2075 6e6b 6e6f  ified as a unkno
00000030: 776e 2c20 6174 2030 2062 6175 642e 0a    wn, at 0 baud..

It may be easier to fork gpsd and add the initialization(s) to its library, or to fix gpsctl to not do this. (The shell script needs a little rework to be usable.)

An apparently identical uBlox5 MiniPCIe module (on site at a client's0 stops responding after CFG-PRT to UBX only.

This is problematic for gpsctl -- even if the module were receiving commands and processing them, gpsctl insists on polling the device to determine its make & model, even if one specifies the -t <type> option.

I think there may also be (benign) bugs in the gpsmon monitor for UBX in gpsd.

I just swapped out a car antenna (SMA terminated) on my uBlox5 in the ALIX. The NAV-SOL (UBX analogue of the NMEA-0183 GGA sentence) solution is sane, but the space vehicle info (Azimuth, Elevation and SNR for each PRN in use) disappeared from the monitor. It took around a minute to reappear.

Pretty sure I saw this last week with the other uBlox5 unit. It's not a dealbreaker, but something to be aware of when troubleshooting a production deployment.

And now I've had gpsmon SIGSEGV on me. It doesn't happen often, but it has happened from time to time.

gpsd itself seems to be stable. As gpsmon requests a JSON feed from the underlying gpsd, this probably wouldn't affect any client-side front end for GPSD (much as exists for NTPD in pfSense right now).