Thursday, 21 November 2013

Yet Another DSC Decoder (YaDD)

For the last couple of months I've been helping to beta test a new MF/HF DSC Decoder which is being written by the author of the excellent YaND (Yet another Navtex Decoder), Dirk Claessens.

The project has developed very quickly, after I asked Dirk on the 5th of September if he'd ever considered a DSC companion to YaND.

By the 17th of September I was trying out YaDD for myself, with Beta version B01.



This was the bare bones, the first of a long sequence of beta versions, and it gave only the raw symbol from the DSC signal, using the same demodulator code lifted from YaND, and it simply printed the received symbols on the screen.

Later versions would improve the demodulator performance, with changes to parameters that better suited the short duration DSC transmissions. Although the modulation of DSC and NAVTEX is the same, 170Hz shift / 100bd FSK, NAVTEX transmits much longer messages. A DSC decoder needs to find and lock onto a DSC signal much more quickly than a NAVTEX signal, and with NAVTEX there's time in a transmission to perhaps re-sync and only miss a few characters. DSC needs to get it all, in one go, to correctly decode a message with no errors - there's little room for human interpretation of a garbled message with DSC.

The improvements came thick and fast.

YaDD B02 changed the format of the displayed signal, and increased the range of DSC symbols that were decoded.



YaDD improved on each release. One thing we studied was the relative decode performance of YaDD compared to DSCDecoder and MultiPSK - the standard DSC Decoders used by the majority of DSC DXers. Each change in YaDD meant another comparison to see if we could spot any improvement in the numbers of successful messages decoded by YaDD.

Many of the improvements were adding more and more DSC formats to YaDD's repertoire - Selective Calls, Area Calls, All Ships Calls, etc. The ITU document (R-REC-M.439) was studied, and studied again (and again... and again), finding new in increasingly intricate combinations of symbols for encoding frequencies, positions, channel numbers, TEST Calls, and Distress Alerts and many more.

I made Perseus off-air recordings, and selected various 15 minute chunks which could be used to quickly measure the number of messages decoded by the three decoders. Along side this were long off-air sessions with multiple decoders, and lots of number crunching of the received messages, and trying to spot any systematic flaws in YaDD's performance.



At version B06, released on the 10th of October,  YaDD could now display the Coast Station name, by reference to a new MMSI-to-CoastStation lookup list that I compiled, using the ITU website, the Admiralty List of Radio Signals, the GMDSS Master Plan and the wonderful RWW Database of DSC signals reported via the DGNavlist, NDBList and NAVTEXDX yahoo groups.

Quickly came version B07 on the 24th of October, which added DSC Area Calls, and increased the amount of displayed  information (positions, frequencies etc,)  taken from decoded messages.


Three more Beta testers joined us, and we continued to test, and count, and test, and count. Problems in how YaDD coped with the convoluted mess of symbols that make up DSC messages were quickly spotted and Dirk speedily found a fix, time after time. The ITU's "Recommendation ITU-R M.493-13" is the bible, and has been our constant companion throughout all the testing and re-testing, but it's far from an easy read, and each new hiccup meant deciphering another obscure section.

Read it for yourselves, sometime...

http://www.itu.int/dms_pubrec/itu-r/rec/m/R-REC-M.493-13-200910-I!!PDF-E.pdf

With version B09,  published on the 1st of November, pretty much everything was in place - many DSC message types were now handled correctly and the weak-signal decode performance was on a par with DSCDecoder and MultiPSK.

After more comparative testing - which was getting more difficult now, since all three of the decoders showed that they coped differently on different signals - DSCDecoder might get a message which was missed by both MultiPSK and YaDD, but then YaDD would get one missed by the other two. Only after long overnight runs, with hundreds, even thousands, of messages could an overall assessment be made.

B0.91 came out, the final one before a public release.

This had some changes to modem parameters, and at first things seemed to be better, but comparing DSCDecoder with YaDD B09 and B0.91 side-by-side showed that perhaps it was a backwards step

Test 1)

2187.5kHz

YaDD b0.91  vs MultiPSK vs DSCDecoder

MultiPSK : 163
DSCDecoder : 163
YaDD b0.91 : 153

Test 2)

Overnight test 1622UTC Friday - 0741UTC Saturday
2187.5kHz

YaDD b0.90  vs  YaDD b0.91  vs DSCDecoder

DSCDecoder : 721
YaDD b0.90 : 752
YaDD b0.91 : 697


YaDD B09 seemed better, and for the release of YaDD V1.0 Dirk rolled back the detrimental changes, while keeping others that did improve things, and YaDD V1.0 was ready.

YaDD V1.0 was announced to the world on the 14th of November, 2013:


Hello all,

YaDD 1.0 is available for download; see instructions below as they were sent to the beta team : John Pumford Green, Dave Onley, Martin Augustus and Stephen "TheAppleMac". I wish to express my sincere thanks to the testers for all the work done. Without your help,  YaDD would not have been where it is today: V1.0 available in just over 2 months!

Special thanks go to John Pumford Green for providing invaluable information, test data and for the many interesting discussions.  

Regards - Dirk


YaDD v1.0 had a new colour scheme, and a real database of received signals, rather than a simple text file.





I tested it, one last time, and sent Dirk the good news

I've now had YaDD V1.0 running for almost 36 hours on 2MHz.

The scores so far:

YaDD v1.0 : 1920
DSCDecoder : 1923



Now that we are happy that YaDD is on a par with other decoders, Dirk is concentrating on improving the user features - watch out for Database filters, and many more things in the near future.

Here is a sneak preview of V1.1, with the ability to filter based on a correct or faulty Checksum, and on whether the sender was a Ship or a Coast Station.... 




More enhancements will surely come along quickly.

YaDD has gone from my question, on September the 5th :

Hello Dirk,

This is a very general enquiry... and please ignore it if you have no
interest.

YaND is the absolute pinnacle of NAVTEX software - extremely sensitive,
with features tailored to the DXer, especially the ability to create
Reports for posting to the NAVTEX list for inclusion in REU/RNA/RWW.

Have you ever considered a similar software for MF/HF DSC?

At present there are 2 options for DSC DXing: DSCDecoder and MultiPSK.

DSCDecoder isn't (from the accounts of others) as sensitive as MultiPSK.
MultiPSK is too large and not specific for DSC DXing.

Neither are really designed for DXers in the same way YaND is.

I also think that YaND is also probably much more sensitive than DSCDecoder.

I'd really like a "YaND for DSC" or "Yet another Dsc Decoder" and I'm
sure many more people would. I love being able to simply click the
database entries I want to send to REU and then magically there's a
properly formed report... DSCDecoder and MultiPSK require a lot of
manual intervention!

If you are not interested, then that's perfectly understandable, but I
thought I would just enquire.

Best wishes,

John
GM4SLV



to a fantastic DXers DSC decoder, in only 2 months.

Thanks Dirk!

YaDD Timeline:





First enquiry : 5 September 2013

Beta Releases:
B01 17 September 2013
B02 21 September 2013
B03 26 September 2013
  
B05  6 October 2013
B06 11 October 2013
B07 24 October 2013
B08 26 October 2013
B09  1 November 2013
B091 5 November 2013

Official Releases:
V1.0-prerelease  12 November 2013
V1.0 Public Release  14 November 2013
V1.1-prerelease  19 November 2013


You can download YaDD from the NBDList website:



5 comments:

  1. This is a truly amazing development in such a short time.
    Thanks to both Dirk and John and also to the beta testers.

    ReplyDelete
  2. I am very happy with this wonderful new tool. Thank you, Dirk! And thank you, John, as well! Calibration with the short DSC bursts is not easy. Taking the calibration value from YAND seems to work well.
    Fritz, DB4IW

    ReplyDelete
  3. Perfekt tool!! Thank you very much! I have used the YADD together with sdr perseus. I'am not sure, but is the offset 1700Hz considered?! The dsc frequencies are set to the sdr perseus, that's fine. E.g. to receive 2187.5kHz you have to dial 2185.8kHz (USB mode) right?!
    73s Andreas
    DK1ER

    ReplyDelete
  4. Offset must not be considered, when RTTY mode with 1700Hz pitch is used. That can be configured under Perseus software settings (RTTY note pitch).

    ReplyDelete
  5. I will give the program a run against DSC Decoder and let you know the results. I have recently created a DSC upload site where the latest HF DSC messages can be viewed. http://www.dscdecode.square7.ch/index.html My station QTH is Brisbane, Australia. Cheers, Ray VK4FTEN

    ReplyDelete