A priori decoding, the good, the bad or the ugly?

By   10 November 2024 22:50

A priori is a Latin phrase which – in the context of FT8 etc. – translates to prior knowledge.

This article aims to explain what a priori decoding does. It also discusses how AP relates to a “valid QSO”. A little paragraph is dedicated to to information theory. The article also illustrates that AP is part of our daily life.

A Priori will further be abbreviated with “AP”. Both WSJT-X and JTDX can apply AP decoding.

Because of its superb decoding performance, I have been using JTDX for a long time with great success. Without it, I would have missed quite a few new entities.

How does AP work?
There are basically two sources of AP information. The first is a list of known call signs and locators, like PA2S JO21. The second source is based on the contents of a QSO.

As far as I know, only JTDX can use a list of known call signs. Both WSJT-X and JTDX can accumulate AP information from the QSO itself.

AP is based on expectation, which is simply put: what one expects during the next receive period. In general, when using AP, the decoder can predict parts of the bit pattern and that is used to guide the decoder towards likely candidates for the original message. It also reduces the number of iterations during decoding, which in turn leads to faster and better error correction.

I will use my initial 1976 callsign PE0HJS for the examples. Suppose you call a station like

PA2S PE0HJS JO20

When PA2S answers, he sends a report like

PE0HJS PA2S -15

Both callsigns are known, which implies that only the report needs to be decoded correctly. This reduces the number of possible candidates for the original message and improves error correction.

When PE0HJS has got the report, he will send a message like

PA2S PE0HJS R-18

And again, both callsigns are known and only the R-18 needs to be decoded correctly by PA2S. If the R-18 could not be decoded by PA2S, he sends the report again and PE0HJS will send the R-18 report again, which means an additional cycle. After the message was decoded, PA2S sends

PE0HJS PA2S RR73 (or RRR).

PE0HJS responds with

PA2S PE0HJS 73

The QSO is complete. You see, each time the callsigns are known, with every cycle an improvement in decoding.

Decoding CQ’s
JTDX is very good at decoding CQ’s from call signs that are in the CALL3.TXT file (which is a list of known stations).

I have often seen CQ’s with -24, which can be a bit tricky, because if you call a station with that strength, it is well possible that you miss the next cycle. Never tried is no Q, but it can be a bit annoying for others if the DX station sends a number of reports which are not decoded. I witnessed this fairly often on 6 m because signal strengths are often quite variable.

AP needs the TX message
It is important to emphasise that the transmitted message has to be taken into account as well. This enables the decoder to anticipate on the next message to receive. I remember a test where I tried to replay a JT65 QSO with the recorded audio files of a QSO with very weak signals. It did not decode, because I did not simulate transmitting. Another test, but that time with transmissons, responding to the received messages, the decodes were exactly as they were during the real QSO.

Does AP fail?
Yes it does. Although most AP decodes are correct, I had false decodes, which made me think that the DX station replied. The reported SNR (signal to noise ratio) was very low in those cases, in one case even down to -26 which is a clear indicator that the decoder was very aggressive. My experience is that AP decodes below -22 are less reliable and those below -24 are often false ones.

SNR “gain”
For propagation research, a small group of radio amateurs in Europe and New Zealand transmit almost daily during about 1.5 to 2 hours around sunrise in Europe. We use FT8, because the software saves the SNR of received messages and those files are processed to be put into a database. The data is used to create graphs of signal to noise ratios versus time. The goal is to gain insight into propagation in the ionosphere. Here is an article with some findings.

I am usually transmitting BEACON PA2S. Others are mostly calling CQ and making QSO’s from time to time. The web SDR in Marahau, New Zealand, is used to record signals from Europe. PG0DX calls usually CQ. Because the Beacon message is a random text message, AP is impossible (it could when the decoder was specifically programmed to recognise the beacon messages). The decoding threshold is around -20 dB SNR.

AP is possible for the PGoDX CQ transmissions and the decoding threshold is extended to about -23 dB, maybe a bit lower. The graphs below are histograms. The bars represent the sum of decodes for every SNR value. Decodes for non-AP information fail below -20 and The AP decodes (bottom graps) become sparse below -23.

These values are equivalent to a 1:2 power ratio or a single Yagi antenna versus a stack of two!

Please note that the graphs below were based on receptions during the first week of November 2024.

It is noted that for a precise comparison we would select one station. monitored with two instances of JTDX running simultaneously, one with AP and the other without. But I consider the histograms below good enough to indicate the difference between thresholds.

20241109_AP_Non_AP_Decoding

 

 

 

Valid QSO?
It depends how one looks at things. First of all, even with phone or CW, we use AP information. Take a CW DXpedition, for example. The signals are weak. You usually just need a part of your call sign to know you are getting a reply. You send a report, usually 599 even when signals are much lower. You listen for the TU. Even the U of TU will do. You note the DXpedition advancing to the next caller and you know that a QSO was completed. Are you 100% certain? Nowadays, using the internet, it can be a matter of seconds or minutes to see your call sign passing on the Clublog livestream or to check if you are in the log. Yes! In the log!

But with maybe only half of the message contents decoded. Did you realise that you used AP? You knew the call sign of the DXpedition. You expected certain patterns. Without even knowing, you were using AP information.

One can conclude that AP is “always on” for traditional phone or CW contacts and if they are considered good, why dismiss digimode QSO’s using AP?

A QSO with an unknown station has no AP information. Right? But as the QSO progresses, AP information is accumulated. We follow certain procedures and because we expect and recognise the patterns, we can fill the gaps.

AP in our daily life
Hmm, humans thus have good error correction capabilities! Many do not realise that we use AP information all the time.

Because the essence of the information theory says – simply put – that everything we already know, conveys no information. Only things we do not know, is information.

AP provides redundancy when communicating. Well known examples are words and phrases in languages. I am a l=cens=d =adi=amat==r. About one third of the characters is missing, but we can “decode” what was meant. The probability of two different words matching the pattern is very low. Do not forget that even the context adds redundancy.

Because languages have substantial redundancy, we understand the station transmitting, even under difficult conditions. Our brain does the AP decoding and error correction.

Information theory
As stated, information is only present when a message conveys something we do not know. AP information does not convey anything at all.

This basic principle has many uses. An important use is compression of data. When we record a video of a static image, every picture frame will be a copy of the previous one. That implies that after the first frame, no information is conveyed. This allows compression of video data.In general, compression removes or reduces redundancy.

I will refer to the literature, but it is noteworthy that most video transmissions have a huge amount of redundancy which allows high compression ratios.

An FT8 transmission has a lot of redundancy
The FT8 protocol adds redundancy wit the so called forward error correction method (FEC). Extra bits are added to the transmission in such a way, that the decoder can reconstruct missing bits. There are various methods to achieve this.

FT8 uses a LDPC (Low Density Parity Check) code. Details are here: chapter 3 of Joe Taylor, Steve Franke and Bill Sommerville (SK).

The LDPC codes were first introduced in 1960 by Robert G. Gallager. This method requires relatively much computation and that was not feasible at the time.

The actual FT8 message content consists of 77 bits. The transmitted message is 174 bits long, including a CRC checksum and 83 bits of FEC code. As a result, the probability of false decodes is very low, which in turn makes the communication reliable and efficient.