Fancy synthesizing of bee-, weather- and environmental-data with Grafana*

while creating open-source hard- & software

Andreas Motl and Matthias Mehldau
The Hiveeyes Project

* and InfluxDB

5th Geospatial Sensor Web Conference
Münster, 3. September 2019

objectives






help bee-keeper's maintenance

study bees

create infrastructure

the whole mess




measure & collect

transmit & (re)distribute

store & query

enrich & represent

measure & collect

measured elements





hive's weight

temperatures & humidities

sound

technologies

InfluxDB & Grafana

measure & collect

ESP8266-board with C/Arduino-programming

bme1.takeForcedMeasurement();
temp = bme1.readTemperature();

#define MQTT_BROKER         "weather.hiveeyes.org"
#define MQTT_PORT           1883
#define MQTT_CLIENT_ID      "a2d46b9d"
#define MQTT_TOPIC          "umwelt/airrohr/research/node-8/data.json"
StaticJsonBuffer<250> jsonBuffer;
JsonObject& json_data = jsonBuffer.createObject();
json_data["temperature"]       = temp;
json_data["time"]              = measuretime;
int json_length = json_data.measureLength();
char payload[json_length + 1];
mqtt_publisher.publish(payload)

measure & collect

ESP32/FiPy-board with micropython

featuring WLAN, Bluetooth, LoRa, LTE(-NB/M1-IoT)

temp = rtd.read()

payload = "{ \"temperature\": " + str(temp) + ", \"time\": " + str(measuretime) + " } "
client = MQTTClient("ef3423be3", "weather.hiveeyes.org", port=1883, keepalive=6)
client.publish("umwelt/airrohr/research/pt100-wtf-02/data.json", payload)

transmit & (re)distribute

Message Queuing Telemetry Transport (MQTT)
& Kotori





Internet Protocol (IP), "uplink-agnostic"
MQTT: open-source pub/sub-message protocol
Kotori: attach MQTT to InfluxDB & Grafana

store & query

InfluxDB


Ingestion: line protocol 
dwd_temperatures,sta_name=Münster,sta_id=02342 2m=23.5,5cm=20.1,dew=14.4 1567505400000

Query v1: InfluxQL ("SQL-ish") 
> SELECT * FROM "dwd_temperatures" WHERE "2m" > 20.0

Query v2: Flux ("LISP-ish")

represent: weight

hive: Zentrum für Urbanistik, Berlin (no harvesting)

represent: bee weather

represent: Berlin's air-quality network

meteogram: past & future

luftdaten.info (LDI) import & worldmap panel

VMM/IRCELINE (Belgium): SOS Import

luftdaten.info: values need compensation

luftdaten.info: quality of humidity data

Grafana: Join()ing different result-tables

ldi_pm2_5 = from(bucket:"luftdaten_info")
  |> range(start:-7d)
  |> filter(fn: (r) =>
    r._measurement == "ldi_readings" and
    r._field == "pm-2-5" and
    r.sta_name == "Brussels Downdown"
  )
  |> aggregateWindow(every: 1h, fn: mean)

vmm_humidity = from(bucket:"vmm")
  |> range(start:-7d)
  |> filter(fn: (r) =>
    r._measurement == "irceline_readings" and
    r._field == "humidity" and
    r.sta_name == "Brussels Airport"
  )
  |> aggregateWindow(every: 1h, fn: mean)

join(tables: {pm2_5: ldi_pm2_5, hum: vmm_humidity}, on: ["_time"])
  |> map(fn: (r) => ({
    _time: r._time,
    _value:  r._value_pm2_5 / math.pow(x: 1.0-(r._value_hum*0.01), y: -0.3),
    _field: "PM2.5 (LDI), Hänel-corrected with humidity by VMM"
    }))

luftdaten.info: correction & comparison

Outlook

Thanks.

The Hiveeyes Project: https://hiveeyes.org/
andreas@hiveeyes.org, wetter@hiveeyes.org


image credits

  • https://unsplash.com/photos/jv9ATyWT0Bw
  • https://www.pexels.com/de-de/foto/bienenwabe-biene-gestalten-bienen-56876/
  • https://unsplash.com/photos/c3RWaj8L3M8
  • https://unsplash.com/photos/Cfo-q_4QQlA

    • Errate v1.1, Sept 4th 2019: Fixed missing sta_name-Filter in join()-example for Flux.