Monthly Archives: October 2010

Getting METAR weather through APRS

Looking on the APRS Google map one could see that there are not so many weather stations in Romania that hams could use.  Knowing that weather data could be gathered form the airports around the country, I decided to make possible that radioamateurs could use METAR weather information through APRS.

The idea is to make weather stations showing on the APRS map, presenting the METAR data. As one could see further down this article, not all METAR information could be sent through APRS but only the more interesting ones. The software will be presented and provided as open source package under GNU GPL license. The language used is C since the intention is to run the application on small, low power embedded Linux systems like home routers.

Couple of words about METAR, APRS and weather:

It seems to be gathering a large consensus that METAR come from french phrase:  message d’observation météorologique pour l’aviation régulière.

METAR is a way (one could say protocol) to convey weather information mainly for pilots use during pre-flight briefing. However, it is also used in other situations like moving weather data from one point to the other. It could contain many data fields that a pilot may be interested in like temperature, dew point, barometric pressure, visibility, wind speed and direction, runway visibility, etc.

APRS means Automatic Packet Reporting System and is a trademark of Bob Bruniga. It is a protocol able to convey various kind of information over AX25 protocol with primarily target being the radio networks.  Therefore APRS is a layer 3 OSI protocol over AX25 link layer. It uses AX25 somehow different in some details I will explain below.

Weather APRS message could include Latitude and Longitude, Wind direction/ Wind speed, Wind gusts speed, temperature, rain, last hour, rain last 24 hours, rain since midnight, humidity, barometric pressure.  Beside those information it could include a timestamp and possible some other arbitrarily message.

Looking at METAR and APRS weather messages one could see that some conversions are needed beside the fact that not all METAR information could fit in the APRS message. One example of such conversion is the wind speed which in METAR could be given in mph, knots, kmph but in APRS they should be given as mph.

Another mismatch between METAR and APRS is that METAR seems not to provide relative humidity (RH), therefore one need to calculate the RH from temperature and dew point. I used Magnus-Tetens formula to calculate RH and since it needs some floating point arithmetic it will need to be adapted when  running on embedded systems without hardware/software floating point.

The weather stations are identified using IDs given by different authorities. For example there could be ICAO (International Civil Aviation Organization) which is four letters ID (e.g. LROP) and WMO (World Meteorological Organization) which is a 5 digit number ID. Unfortunately, only weather data from stations having ICAO ID is provided on the NoAA servers (National Oceanic and Atmospheric Administration) in US. I would very much like to know how to access the data from stations listed only with WMO ID.

Being able to add text message to the normal APRS weather message made me add some more information from METAR like DEW point, prevail visibility and whether is CAVOK or not. CAVOK means Ceiling And Visibility OK.

How metarAPRS program works:

In short, the program accesses the NoAA ftp server, gets the latest METAR data from there, converts the relevant fields to APRS and sends it to the local PC Ax25 stack or directly to APRS-IS servers. When using the AX25 stack, the weather data could be output to a radio and when using APRS-IS servers connection, the data is fed directly to the servers through a TCP/IP connection.

A typical METAR message looks like:

date: 2010/10/31 14:30
METAR: LRBM 311430Z 00000KT CAVOK 14/02 Q1018

metarAPRS program has a configuration file ~/.metar-aprs/data.csv which needs to have listed all the station one needs to show on the APRS map. The ID (callsign in APRS parlance) of the station could be entered and also the position of the station needs to be added since it is not given by the METAR data. However, the position could be obtained by using following link: www.nws.noaa.gov/tg/siteloc.shtml

A typical line in the configuration file looks like this:

LRBM-1, LRBM, 4740.00N/02335.00E – where LRBM-1 is the APRS callsign given to the station and LRBM is the ICAO ID for the station.

System configuration for metarAPRS:

If using a direct connection to one APRS-IS server, there is no need for an IGate. Just the proper command line options needs to be given. For example if one need to connect to the server france.aprs2.net:14580, using password “1234” and callsign YO3IIU-10, the following command needs to be given:

metarAPRS -s france.aprs2.net -n 14580 -w 1234 -c YO3IIU-10

If want to use ax25 option in order to send the weather to a radio connected to your system, one needs to have an AX25 port configured. It is done by having axports (/etc/ax25/axports) filled in with the needed information (see man axports for details). For example, this is one line my axports file has:

sm0 YO3IIU 9600 256 7 144.800 APRS (9600 bps)

The port is sm0 and the callsign of the interface in this case is YO3IIU.

Now you need to initialize the port:

kissattach /dev/ttyS0 sm0 192.168.1.200

If all went smoothly you will see an interface called ax0 when listing existing interfaces on your system.

If you’ve already used soundmodem then you don’t need to do anything since the Ax25 is already configured.

Using an IGate:

One could use any IGate he/she has access to. I use aprsd but the most interesting application is XASTIR (www.xastir.org). It is not an IGate “per se” but it has this functionality among other features I like at it.  One of the best features it has is the ability to use the OSM (Open Street Maps) – www.openstreetmap.org directly in it’s latest version from svn.

For XASTIR you need to configure one internet interface and one Ax25 TNC interface with sm0 as port.

Options for metarPRS:

I’ve added couple of command line options to the program and they could be seen in the help.

$ metarAPRS –help
Usage: metar-aprs [OPTONS]

-h –help                  help
-V –verbose          verbose
-v –version            version
-p –port                   port
-d –destination    destination (default IDENT)
-i –interval             interval (minutes)
-a –ax25                  use ax25 stack instead of direct connection
-s –server                APRS-IS server name
-n –server-port     APRS-IS server port
-w –pass                   APRS-IS server password
-c –callsign             callsign used for server login

Practical results:

Screen shot of one METAR weather station when using the direct server connection ( you could see the TCPIP*,qAS construct which means data comes from a direct connection):

Screen shot of the same METAR weather station when using Ax25 stack. The data now comes from a radio link (see the qAR construct):

After setting data file, configuring the Ax25 and the IGate, one will just need to run metarAPRS. Here are some screenshots with METAR weather stations showing on APRS map in both web browser and XASTIR:

Screenshot showing the wisibility data:

XASTIR screenshot with Open Street Map:

Things TODO:

– use other sources of weather data (what about WMO source or RSS feed?)

– find a way to show more data

– daemonize the application

Source code released as Open Source under GNU GPL license:

metarAPRS application sourcecode (v01): metarAPRS.tar.gz

metarAPRS application sourcecode (v02): metarAPRS_02.tar.gz

metarAPRS application sourcecode (v03): metarAPRS_03.tar.gz


Digital Coded Squelch (DCS) codec part II

In the previous DCS article I presented a general overview of the technology together with the problems that an implementer of the decoder may fight with. In this article I will present the functional blocks of the decoder and how they interact each other. The complete information could be gathered from the implementation which is presented as open source packages in the end of the article.

Building blocks:


The goal of the project was set to decode the DCS signal in noisy conditions, having distorted signal and accepting a broad range of audio volume level. In the following sub-chapters each block is presented.

Aquisition of signal

ALSA is used for aquisition of signal at the sampling rate of 8Khz and S16_LE format. Note that libdcscodec uses a buffer as input and does not depend on ALSA or other input method. The acquired signal is looking like in the following picture:

LP Filter

The Low Pass Filter is implemented as a FIR filter and is designed using Parks-McClellan algorithm for optimum approximation of a generalized linear phase systems. Basically the algorithm aims for equiripple in both passband and stopband of the filter. The filter is designed to have maximum ripple as 0.5dB and 60dB attenuation in stopband. Giving this numbers together with the Fp=84.4Hz and Fs=134.4hz, parks McClellan algorithm will calculate a filter with order n=357 which is quite high and therefore the filter is a Type II FIR filter.

The signal after LP filtering is presented below:

DC filter

This filter is actually a combination of a differentiator followed by an integrator having therefore the differential equation like the following: y(t) = x(t) – x(t-1) +ay(t-1).  Its need comes from the observation that the signal from the receiver seems to be shifted up or below the zero level line. The coefficient a could be tweaked in order to adjust to the cut-on frequency.

Resample

Just to recap the input data we have, the sampling frequency is 8khz and the DCS signal has a rate of 134.4Hz. In order to correctly process the data and be able to find the moments when a switch from “0” to “1” and “1” to “0” is done, we must have a sampling frequency which is multiple of the 134.4Hz rate. That is why resampling is needed and the resampled rate is chosen 8*134.4Hz. Therefore we will have 8 samples per symbol. Resampling has been implemented using the SRC library which works very well and gives the proper results. However I plan to implement my own resampling function just to reduce the code size of the DCS codec.

Integrate and Dump

From the theory of digital communication it is know that a matched filter could be used to reduce the ISI (Inter Simbol Interference) and therefore to increase the BER. Integrate and dump is a simple matched filter that works by integrating (adding) the samples through a period, use the result, dump the result and then start all over. In our case the integration is done in a period of eight samples T=8*1/134.4[s] and therefore taking in consideration the new sampling frequency.

There is a discussion here regarding the start of the sampling period. In an optimum case the sampling period should have the peak of signal at its center so that the integrate and dump method should work proper. However, there is no clear start and stop for this integrating period and one big question arises: when to start the integrate period? Some methods have been developed for that among them being the “early late gate” which uses three samples in order to determine the proper start of the integrating period. However, what I’m currently doing is integrating the signal considering each of the 8th samples as starting point for the integration process. Each result of the integration will be sent to the detector algorithms and redundant data is eliminated. I know this is not optimum and this is one of the points to be changed in the near future.

AGC

One of the goals of making the DCS codec is to be able to use a broad range of audio volume level. This is important because the soundcard may have preamplifiers or dividers and also the radio receiver may output different audio level from discriminator. The AGC block is not properly an Automatic Gain Control but it is a mean to determine the threshold level above which the signal is considered to be a peak by the peak detector. The threshold level is calculated using the standard deviation (std) of a signal over the entire buffer.  See the source code for the implementation.

Peak detector

As previously stated, the peak detector is used to determine signal’s peaks and therefore the transition form “0” to “1” and “1” to “0”. Starting with a peak, the next peak should be at a time distance of k*1/134.4[s] with k being integer.  After the peak detector a bitstream of “0” and “1” will be output to the next processing block.

Golay decode

having the bitstream from peak detector as an input, this block does calculate the check bits taking in consideration the new arrived bit. If the calculated check bits match with the received check bits then we have a match and the corresponding Golay is looked up in a table containing all Golay codes.  The “standard” Golay code is taken from the table and then send to the next processing block.

Display of DCS code

This is simple console display of the DCS code. One more thing is done, due to the fact that polarity of the signal is not known by the software both positive and negative polarity codes are presented. It is users responsibility to decide which code to take in consideration. Normally the polarity should be communicated by the user of the transmitter.

Practical results

The tests were done using a Yaesu FT-857D as receiver and a Yaesu VX-7R portable as transmitter. All the above graphics were saved from the program and exported using gnuplot.

One interesting point to note regarding the VX-7R is when the portable is enabled as dual receive. It seems the DCS code it sends on transmission is the selected one together with another random code.

TODO

– optimize buffer utilization

– use own resampling method

– use early-late gate method for determining the proper start of integration

– implement transmitter

– optimize for embedded processors with no floating point

Sourcecode released as Open Source under GNU GPL license:

library that implements DCS codec – libdcscodec.tar.gz

test program for libdcscodec – dcscodec.tar.gz

libresample package needs to be installed in your system as it is a dependency.

Digital Coded Squelch (DCS) codec part I

Looking for a Digital Coded Squelch package that I could include in one of my projects I found out that there is no such package released for open source.  Looking for more information I found no DCS detailed information related to physical levels, how it is modulated etcaetera. Therefore I decided trying to implement a DCS codec that could be included as a library in any project. I present below the steps I followed in making the DCS codec library and the tests I’ve done for it.

Introduction

The DCS stands for Digital Coded Squelch and is a method of controlling the squelch of a radio by sending a digital code over the audio signal. It is a further development of the CTCSS method (Continuous Tone Coded Squelch System) ans it is called sometimes CDCSS (Continuous Digital Coded Squelch System). Various manufacturers calls this method different, for example Motorola (TM) calls it DPL (Digital Private Line), General Electric (TM) calls it DCG (Digital Channel Guard).

The DCS not only opens the squelch for a radio but it is also used to close the squelch before the carrier goes off in order to eliminate the squelch tail. It was an unfortunate decision to use the word “Private” on the squelch technologies because no privacy is added to the system when these are used. A lot of discussions were held regarding the “Private” meaning of these technologies many times ethical reasons being the major argument.

One will find on internet a lot of information about DCS including history and its preceding technologies but for what I needed I have found only two resources with valuable information. They are presented below:

http://onfreq.com/syntorx/dcs.html

http://mmi-comm.tripod.com/dcs.html

For short, DCS sends digital signals with 134.4Hz rate using golay (23, 12) codes over the audio signal. The golay (23, 12) means DCS uses 23 bit words with 12 bit data included the 23 bits.  The spectrum of the DCS signal contains major components in the 0 to 300hz band and therefore a LPF (Low Pass Filter) could be used on the receive side to get rid of the DCS signal and remain only with the audio signal. That’s what most of the receivers do noadays. Because of the existence of LPF in the receiver, it is indicated to get the signal for the DCS receiver just after the discriminator so that the DCS signal will be available together with audio signal.

Having available the mix of audio signal with DCS signal it is not a good thing for DCS decoding just because voice audio will interfere with the digital signal and therefore the DCS decoder will need a good low pass filter in order to eliminate the unwanted audio component. Some radio equipment vendors are doing voice filtering under 300hz before sending it to transmitter so that the receiver’s DCS decoder does not need to do LPF. However more vendors forget to do that and therefore the receiver decoder has to do it. I present below some captures of the DCS signal in varius conditions:

The picture above captures the signal after discriminator before and after the PTT on the transmitter is released. One could see three main zone:

1st zone – normal DCS code – in this specific case the code is 205.

2nd zone – the PTT is released, the transmitter sends squelch off signal. In this case it is a sine wave with frequency 134.4hz.

3rd zone – the carrier is gone and the receiver outputs white nose.

Given the 134.4hz rate of the digital signal it could be calculated that sending one word (23bits) takes up less than 0.2 seconds.

The major problem I encountered in implementing the DCS decoder was the big level of voice audio signal compared with the level of the DCS signal. This made the need for a good LP filter and other signal processing techniques in order to make the signal usable and correctly decoded with the DCS decoder. I present below an example of the signal when the user whistle into the microphone:

One could see the level of audio signal is three times more than the level of the DCS signal.

One word about modulation and Golay codes

The DCS signal is obviously a digital modulated analog signal. Each pulse represents a transition from 0 to 1 or reverse. There is no start or stop bit in the signal, therefore DCS decoder needs to look each bit it receives form the stream and decide whether it matches or not a Golay code together with the bits it received up to now.

Golay (23, 12) words contains 12bit of data represented in 8th base (octal). From this 12bits only 9bits are actual data, the rest 3bits are fixed as “100”. A Golay word is represented like this:

P11-P10-P9-P8-P7-P6-P5-P4-P3-P2-P1-100-C9-C8-C7-C6-C5-C4-C3-C2-C1

where Px are check bits and Cx are data bits.

Golay words are having some interesting properties, one of them being the fact that if a Golay word is circularly shifted left or right, another valid Golay word is obtained. This property together with the fact that no start/stop bits are given results in multiple matches depending on where the start bit is considered.  This made the maximum number of so called “standard” DCS codes be reduced so that just one match is taken in consideration.

Another property of Golay codes is the fact that if “1” and “0” is swapped into the bitstream of a valid code, then another valid code is obtained. This is in connection with what is considersd “1” and what is “0” when looking at the signal. Therefore one could talk about positive polarity and negative polarity for a DCS signal. For example +023 code produces the same pattern as -047 code. The current implementation gives both positive and negative matching.

It worth mentioning another property of the Golay codes that are not used for DCS. It is possible to recover up to three bits using the check bits.

In the next article I will present in detail how the DCS decoding is achieved in software and what software blocks I used for successfully doing this. To be continued.