An Alternative Setup for the WWD RPi

I have found an alternative item to use as the basis for the Web Weather Display (WWD).  I found some code for a Weatherflow PiConsole that uses information from my Tempest to display current conditions.  Chief URLs are as follows:

• https://community.weatherflow.com/t/weatherflow-piconsole/1933
• https://github.com/peted-davis/WeatherFlow_PiConsole
• https://www.raspberrypi.com/documentation/computers/config_txt.html#lcd-displays-and-touchscreens

I am proposing to use this code as a starting place for the WWD and then work out the mechanism for a web display later.  In this way I can get something up quickly without a lot of fuss.  I will probably use one of the RPis in my rack as the unit.

Using an RTL-SDR usb dongle to get other weather stations signals

I was futzing around last night repurposing some Raspberry Pis (bought before they were $180 apiece due to chip shortage) in my network rack. I took one of the RPis and installed RTL_433, node red, and plugged an RTL-SDR dongle into it along with a small antenna. I am going to use node red to push data to an MQTT service for use with my weather data extraction setup. I mostly wanted to see if I could pick up an Accurite Temperature sensor (606TX) on my balcony at 433.925 MHz. What I wasn’t expecting was picking up a couple other temperature sensors, one Accurite and one Oregon Scientific models.

I live in a retirement community and for the most part the residents are not really that tech-savvy, or computer-savvy for that matter. So I wouldn’t expect them to have a lot of weather station gear. My 606TX only gives back temperature on the balcony, but I noticed that it was reading a temperature about 2 degrees C higher than the Oregon Scientific model. The Oregon Scientific model also gave back humidity. I am thinking of gathering data from each of the other models to extract trends. My balcony faces north and that might be the cause of the temperature difference in the summer time.

I haven’t played with the gain of the dongle or looked into other frequencies yet. I am pretty sure that my Tempest, also on my balcony, transmits on the 915 MHz band, but I have not been able to capture it’s data set yet with the dongle.

WWD #03 - Setting up data_gather for Tempest weather station

 I have been extracting information from my Tempest weather station (TWS) via the WeatherFlow Tempest REST api.  It involves signing up for a key from their website for your particular TWS, then sending a crafted URI with the key, and sometimes your weather station id.  There is a limitation on the number of samples you can get over time unless you sign up for a business account, but at least you can get to your own data.

To sign up for the key you log into your account at TempestWx.com.  Then select the Settings icon on the upper right hand side.  That brings up a Settings page.  Go to the bottom, there you will find a link, "Data Authorizations".  When you select that you will see a create Token button on the top right corner.  Selecting that button will create a token key that can be used with the Tempest api to get information that was sent to TempestWx.com from your TWS.

You use the token as follows:  assume that my token is 12345678-1234-1234-1234-123456789098.  You can get the station information, which comes back in a JSON format by using the URI, https://swd.weatherflow.com/swd/rest/stations?token=12345678-1234-1234-1234-123456789098.  This will send back a JSON message similar to the following (showing a station_id of 12345 from stations[0].station_id) - 

{

  "stations": [
    {
      "location_id": 12345,
      "station_id": 12345,
      "name": "My TWS Name",
      "public_name": "My TWS Name",
      "latitude": 99.99999,
      "longitude": -99.99999,
      "timezone": "America/New_York",
      "timezone_offset_minutes": -240,
      "station_meta": {
        "share_with_wf": true,
        "share_with_wu": true,
        "elevation": 78.30184936523438
      },
      "last_modified_epoch": 1656712447,
      "created_epoch": 1630250472,
      "devices": [ ...

To get a sample, probably once every 1.0 minutes max, you would use a URI, https://swd.weatherflow.com/swd/rest/observations/station/12345?token=12345678-1234-1234-1234-123456789098. This will send back a JSON message similar to the one above, with the latest observation from your TWS under obs[0].

I will be prototyping this using an http request node within the data_gather container written in node-red.  I am going to push this to the MQTT server as a /weather/TWS/observation topic.  I am however going to reduce the number of items sent.

WWD #02 - Initial Architecture for the Web Weather Display

I am currently trying to thrash out the initial architecture for the Web Weather Display.  I know that it will be split into multiple parts and use Microservice patterns as the starting point.  This will take some time as I explore what items to add into the mix.  I do know that I will need the following Microservices as a start:

• NGINX to separate out the web front end
• A web front end that can be displayed via NGINX
• A Service Registry Microservice with a pattern allowing switching between Microservices on the fly, across platforms and vlans
• A Database back end to record changes as they happen
• A Microservice to push data to the Database back end
• A Microservice for each of:
• OpenWeatherMap data extraction
• Tempest weather station data extraction
• NWS data extraction
• NOAA data extraction
• WeeWx data extraction
• A Microservice for extracting current weather in a format that can be easily displayed
• A Microservice for merging current weather information
• A Microservice for building graphs of weather over time
• A Microservice for interacting with other sensors
• WeeWx (optional) to extract data from sensors

Each of these Microservices can be done with stubs as the overall fixed return.

So I have the following to start with:

• Defined Microservices
• service_def
• service_reg
• data_logger
• data_assemble
• data_gather
• web_assembly_proxy
• web_presentation
• MQTT
• Services that define the front end
• NGINX
• wordpress
• phpmyadmin
• mysql (mariadb)
• Services that provide temporal DB and graphs
• grafana
• influxdb
• Services for prototyping
• node-red-1
• node-red-2
• node-red-3
• Service for Docker control
• portainer

WWD #01 - Sources for Web Weather Display

 I was trying to pick through my mind and my notes to come up with a list of sources for the Web Weather Display.  I wanted to concentrate on free sources; this effectively eliminates any access to Weather Underground.  This is the list that I came up with:

• OpenWeatherMap - using a key from OpenWeatherMap you can download current predicted values for your given lat and long (https://openweathermap.org/api/one-call-api)
• NWS - the National Weather Service has a web services connection that allows you to gather information on current conditions in your area (https://www.weather.gov/documentation/services-web-api) as well as warnings and alerts for the area
• NOAA - NOAA has a web services connection that can gather satellite images and loop movies for the WWD (https://www.ncdc.noaa.gov/cdo-web/webservices/v2, https://www.ncdc.noaa.gov/cdo-web/token) along with other weather related information
• DarkSky - DarkSky is another source similar to OpenWeatherMap that provides world wide data; however it will be eclipsed and merged into an Apple product in 2023, so it doesn’t appear to be a good place to get information at this time
• Accuweather - Accuweather is another source of weather information; this one allows a limited set of 50 hits/day for free (https://developer.accuweather.com/apis)
• WeeWx - WeeWx is a general purpose posting and data gathering platform.  It is extensible with a number of different plug-ins and thus could be merged into the WWD as a data source/sink via MQTT interactions without transferring information to either CWOP or Weather Underground.  It does provide a good general purpose interface to most existing weather stations out there as well as RTL-SDR interfaces and other sensors (http://www.weewx.com/).
• Air Quality API - TBD
• Hurricane Models API - TBD
• Weather Models API - TBD

Project #27 - Developing a multi-source Web Weather Display using Microservices and DevOps

 I have been wanting to get into DevOps for a while.  Now that I am retired I should be able to find the time to immerse myself in the overall designs and setup for such a system.  I have also been studying Microservices and how that impacts the pipeline for a DevOps CI/CD approach.  Since I have a number of computer resources available to me, both RPi and Proxmox servers, I should be able to stand up a complete system fairly easily.  I also need to make it easy to modify as needed.  I am just a one person shop using a HomeLab for development, so much of this will not make sense to most people.

So for the general idea for the Web Weather Display:

• Most programming to be done with Node-Red, but allow for JavaScript and Python
• The Weather Display will be a Kiosk style web display
• Since I don’t have a Weather Station hooked into CWOP or Weather Underground, I need different sources of information
• Should be able to be updated with a display every 5 minutes
• Should show a forecast for at least 5 days
• Should allow for a display of radar data
• Should allow for a display of satellite data
• Should allow merging of micro-weather sensors to show alteration from the normal source of updates
• I want it to be able to display different models from different sources
• I want it to be able to display other environmental information such as earthquakes, hurricanes, tornado damage from different parts of the world
• I would like it to have a map display capability 
• I would like it to display data over a range of time from a backend storage medium
• I would like it to be self contained in a single RPi, or pulled to an RPi display from a remote source

For the general idea for Microservices:

• I would like to use some Microservices patterns for the development; those are TBD at this point
• I want to make the Weather Display components split into Microservices that make sense
• The Microservices should be able to be moved to a different host without causing disruption, including to different operating systems and processors
• Individual Microservices should be able to be changed without having to shut the entire set of Microservices down

For the general idea for DevOps:

• I would like to do the development on a separate Development vlan which works in conjunction with a Production vlan for the Weather Display
• The introduction of a newly tested and developed Microservice should be scripted so that I do not have to worry about forcing a change
• The movement of portions of the Microservice architecture should be fluid within the CI/CD pipeline

Some constraints for the Weather Display development:

• I should not have to buy anything additional in order to be able to complete the first version of this project
• I should be able to add additional sensors and merge their output into the displays without issue
• The first iteration of this project should use MQTT as the adhesion point, but allow for use of web services later on - so the mechanism should be hidden