Trouble in Z-Wave paradise: Decoupling Z-Wave from Openhab via MQTT

In the previous post I described how I successfully connected Everspring AN158 (a non-Z-Wave Plus) device to Openhab. However the Z-Wave Plus devices (Aeon Smart Switch and Z-Wave.me WALLC-S) haven’t been that cooperative. It seems that the SECURITY class used by Z-Wave Plus devices is not quite that well supported by Openhab Z-Wave engine. I know that the issue is with Openhab, since I can easily turn the smart switch on and off with Z-Way server GUI.

After configuring the devices with Z-Way server I shut it down (as it cannot co-exist with Openhab since they both use exclusively the /dev/ttyAMA0 device which RazBerry binds to). Then I browesed through the Openhab Z-Wave logs after pushing manually the on/off button on Aeon switch:

2015-07-07 20:40:22.092 [DEBUG] [eController$ZWaveReceiveThread:1528]- Receive Message = 01 11 00 49 84 14 0B 04 10 01 5E 86 72 98 56 EF 5A 82 7A
2015-07-07 20:40:22.096 [DEBUG] [eController$ZWaveReceiveThread:1452]- Receive queue ADD: Length=1
2015-07-07 20:40:22.096 [DEBUG] [b.z.i.protocol.ZWaveController:1210]- Receive queue TAKE: Length=0
2015-07-07 20:40:22.100 [DEBUG] [o.b.z.i.protocol.SerialMessage:233 ]- Assembled message buffer = 01 11 00 49 84 14 0B 04 10 01 5E 86 72 98 56 EF 5A 82 7A
2015-07-07 20:40:22.103 [DEBUG] [b.z.i.protocol.ZWaveController:1211]- Process Message = 01 11 00 49 84 14 0B 04 10 01 5E 86 72 98 56 EF 5A 82 7A
2015-07-07 20:40:22.106 [DEBUG] [b.z.i.protocol.ZWaveController:190 ]- Message: class = ApplicationUpdate (0x49), type = Request (0x00), payload = 84 14 0B 04 10 01 5E 86 72 98 56 EF 5A 82
2015-07-07 20:40:22.107 [DEBUG] [.ApplicationUpdateMessageClass:44  ]- NODE 20: Application update request. Node information received.
2015-07-07 20:40:22.109 [WARN ] [.o.b.z.i.p.c.ZWaveCommandClass:221 ]- NODE 20: Unsupported command class ZWAVE_PLUS_INFO
2015-07-07 20:40:22.111 [DEBUG] [.o.b.z.i.p.c.ZWaveCommandClass:224 ]- NODE 20: Creating new instance of command class VERSION
2015-07-07 20:40:22.114 [DEBUG] [.o.b.z.i.p.c.ZWaveCommandClass:224 ]- NODE 20: Creating new instance of command class MANUFACTURER_SPECIFIC
2015-07-07 20:40:22.116 [WARN ] [.o.b.z.i.p.c.ZWaveCommandClass:221 ]- NODE 20: Unsupported command class SECURITY
2015-07-07 20:40:22.118 [DEBUG] [.o.b.z.i.p.c.ZWaveCommandClass:224 ]- NODE 20: Creating new instance of command class CRC_16_ENCAP
2015-07-07 20:40:22.121 [DEBUG] [.z.i.p.s.ZWaveCommandProcessor:63  ]- Sent message Message: class = SendData (0x13), type = Request (0x00), payload = 15 01 00
2015-07-07 20:40:22.124 [DEBUG] [.z.i.p.s.ZWaveCommandProcessor:64  ]- Recv message Message: class = ApplicationUpdate (0x49), type = Request (0x00), payload = 84 14 0B 04 10 01 5E 86 72 98 56 EF 5A 82
2015-07-07 20:40:22.125 [DEBUG] [.z.i.p.s.ZWaveCommandProcessor:65  ]- Checking transaction complete: class=ApplicationUpdate, expected=SendData, cancelled=false
2015-07-07 20:40:23.053 [DEBUG] [eController$ZWaveReceiveThread:1528]- Receive Message = 01 08 00 04 00 14 02 98 40 3D
2015-07-07 20:40:23.056 [DEBUG] [eController$ZWaveReceiveThread:1452]- Receive queue ADD: Length=1
2015-07-07 20:40:23.056 [DEBUG] [b.z.i.protocol.ZWaveController:1210]- Receive queue TAKE: Length=0
2015-07-07 20:40:23.060 [DEBUG] [o.b.z.i.protocol.SerialMessage:233 ]- Assembled message buffer = 01 08 00 04 00 14 02 98 40 3D
2015-07-07 20:40:23.062 [DEBUG] [b.z.i.protocol.ZWaveController:1211]- Process Message = 01 08 00 04 00 14 02 98 40 3D
2015-07-07 20:40:23.064 [DEBUG] [b.z.i.protocol.ZWaveController:190 ]- Message: class = ApplicationCommandHandler (0x04), type = Request (0x00), payload = 00 14 02 98 40
2015-07-07 20:40:23.066 [DEBUG] [ApplicationCommandMessageClass:38  ]- NODE 20: Application Command Request (ALIVE:DYNAMIC_VALUES)
2015-07-07 20:40:23.068 [DEBUG] [ApplicationCommandMessageClass:56  ]- NODE 20: Incoming command class SECURITY
2015-07-07 20:40:23.069 [DEBUG] [ApplicationCommandMessageClass:62  ]- NODE 20: Command class SECURITY not found, trying to add it.
2015-07-07 20:40:23.071 [WARN ] [.o.b.z.i.p.c.ZWaveCommandClass:221 ]- NODE 20: Unsupported command class SECURITY
2015-07-07 20:40:23.073 [ERROR] [ApplicationCommandMessageClass:75  ]- NODE 20: Unsupported command class SECURITY (0x98)

Also similar error messages can be seen with WALLC-S when I push one of its buttons:

2015-07-07 20:29:04.626 [DEBUG] [eController$ZWaveReceiveThread:1528]- Receive Message = 01 08 00 04 00 05 02 98 40 2C
2015-07-07 20:29:04.629 [DEBUG] [eController$ZWaveReceiveThread:1452]- Receive queue ADD: Length=1
2015-07-07 20:29:04.630 [DEBUG] [b.z.i.protocol.ZWaveController:1210]- Receive queue TAKE: Length=0
2015-07-07 20:29:04.633 [DEBUG] [o.b.z.i.protocol.SerialMessage:233 ]- Assembled message buffer = 01 08 00 04 00 05 02 98 40 2C
2015-07-07 20:29:04.635 [DEBUG] [b.z.i.protocol.ZWaveController:1211]- Process Message = 01 08 00 04 00 05 02 98 40 2C
2015-07-07 20:29:04.637 [DEBUG] [b.z.i.protocol.ZWaveController:190 ]- Message: class = ApplicationCommandHandler (0x04), type = Request (0x00), payload = 00 05 02 98 40
2015-07-07 20:29:04.638 [DEBUG] [ApplicationCommandMessageClass:38  ]- NODE 5: Application Command Request (ALIVE:PING)
2015-07-07 20:29:04.640 [DEBUG] [ApplicationCommandMessageClass:56  ]- NODE 5: Incoming command class SECURITY
2015-07-07 20:29:04.641 [DEBUG] [ApplicationCommandMessageClass:62  ]- NODE 5: Command class SECURITY not found, trying to add it.
2015-07-07 20:29:04.643 [WARN ] [.o.b.z.i.p.c.ZWaveCommandClass:221 ]- NODE 5: Unsupported command class SECURITY
2015-07-07 20:29:04.644 [ERROR] [ApplicationCommandMessageClass:75  ]- NODE 5: Unsupported command class SECURITY (0x98)

Documentation regarding Z-Wave Plus security is sparse, but OpenZWave (another z-wave stack NOT used by Openhab) has this very nice and informative page.

So, maybe it’s an issue with Z-Way server setting up a Network Key to manage encrypted communication with Plus devices but now Openhab doesn’t know the Network key and fails?
In /opt/z-way-server/config-zddx/ directory is a file named something like cb2ce85ed-DevicesData.xml that contains an entry such as:

   <data name="networkKey" invalidateTime="1435679337" updateTime="1435679338" type="binary" value="[ 35 c3 34 b2 44 e3 cf ca 23 db b1 3d 15 79 07 12 ]"/>

Bingo! Now, how to configure Openhab with it?

I browsed through the configuration files and their GitHub repository, but didn’t find any real references to implementation of Security Class. Quick search on their Google group provided a discussion about it. It seems (as of July 2015) the Z-Wave Security is still not supported. That’s a bummer.. Now, I could start using OpenZWave since it has preliminary support for it, but there’s no way to use it with Openhab, but instead I would be forced to switch to Domotiga, Domoticz, Jeedom or other system using OpenZWave. Since Z-Wave documentation is behind NDA and open-source implementation requires reverse-engineering it, I wonder why there are two different camps re-inventing the wheel (OpenZWave and Z-Wave stack of Openhab). For historical reasons? Openhab itself is a already a huge monolith, so the reason why they have decided to split the effort and code their own Z-Wave stack is just beyond me..

Anyway, to use Z-Wave Plus devices with Openhab there’s another way..

MQTT cavarly to the rescue!

From a DIY perspective, it’s best to use separate components that each do their own job very well instead of relying on a monolith that tries to do everything on its own. I decided to decouple Z-Wave from Openhab using MQTT, but I wasn’t sure which way to go. There’s one attempt with OpenZWave and MQTT bridge, but without any documentation and it has only one commit 6 months ago. Since I had already set up Z-Way server, I decided to do some hacking with it.

By browsing Z-Way API documentation one can see that the embedded web server offers JSON API, which you can use to configure Z-Wave settings, read device data and also control devices. For example to turn on and off the wall switch (here Z-Wave device #20), these simple HTTP requests can be issued with a browser:

http://192.168.1.80:8083/ZWaveAPI/Run/devices[20].instances[0].Basic.Set(1)
http://192.168.1.80:8083/ZWaveAPI/Run/devices[20].instances[0].Basic.Set(0)

Here we use the Basic command class to control the device. It’s little bit tricky to find out the exact JSON tree to read and control each device, but you can try to mesh the information seen on the GUI of Z-Way server with the Z-Way API doc, or you could decipher the JSON tree itself. You can read all the available command classes with command

http://192.168.1.80:8083/ZWaveAPI/Run/devices[20].instances[0].commandClasses

In my case the Aeon Smart Switch supports following command classes:

  • Basic (32)
  • SwitchBinary (37)
  • SwitchAll (39)
  • SceneActivation (43)
  • SceneActuatorConf (44)
  • Meter (50)
  • CRC16 (86)
  • AssociationGroupInformation (89)
  • ZWavePlusInfo (94)
  • Configuration (112)
  • ManufacturerSpecific (114)
  • PowerLevel (115)
  • FirmwareUpdate (122)
  • Association (133)
  • Version (134)
  • Security (152)
  • DeviceResetLocally (90)
  • Hail (130)

Here the SwitchBinary and Meter classes seem most interesting. It seems that Basic class is just re-mapped internally by the switch to SwitchBinary, since all Z-Wave devices must understand Basic class to facilitate interoperability between dumb sensors and actuators. Also you can reference the class by the number instead of name. Hence, all these four commands work equally:

http://192.168.1.80:8083/ZWaveAPI/Run/devices[20].instances[0].Basic.Set(1)
http://192.168.1.80:8083/ZWaveAPI/Run/devices[20].instances[0].SwitchBinary.Set(1)
http://192.168.1.80:8083/ZWaveAPI/Run/devices[20].instances[0].commandClasses[32].Set(1)
http://192.168.1.80:8083/ZWaveAPI/Run/devices[20].instances[0].commandClasses[37].Set(1)

mqttwarn – a swiss knife of MQTT

Now, how to best translate MQTT messages into JSON HTTP POSTs? There’s mqttwarn, an excellent piece of python software (on Github here), that not only does this, but can also convert MQTT messages into notifications on your cell phone (using Pushover or Instapush), send e-mail, post on your Twitter account, send e-mail, transfer your sensor data over to your remote EmonCMS server and tons more!

To install and configure mqttwarn, just follow the instructions at the end of the (very long) GitHub project description. Then, edit the mqttwarn.ini:

First add the http service provider:

; name the service providers you will be using.
launch   = file, log, http

Then configure the provider. Here the HTTP URL is modified with the MQTT message payload in following way:

[config:http]
timeout = 60

targets = {
                #method     #URL               # query params or None          # list auth
  'dev20set'    : [ "post", "http://192.168.1.80:8083/ZWaveAPI/Run/devices[20].instances[0].Basic.Set({payload})", None, None ],
  'dev21set'    : [ "post", "http://192.168.1.80:8083/ZWaveAPI/Run/devices[21].instances[0].Basic.Set({payload})", None, None ]
  }

Finally we add a list of MQTT messages that trigger the launch of the HTTP provider:

[#]
targets = {
        'home/zwave/Device20/switch/set' : 'http:dev20set',
        'home/zwave/Device21/switch/set' : 'http:dev21set'
        }

After you start mqttwarn, you can test it by sending MQTT messages like this:

mosquitto_pub -t home/zwave/Device20/switch/set -m 1

If you successfully installed mqttwarn you should now have all the required python dependencies set up (such as Paho MQTT Python module). Note that I tested few other python bindings and I remember having first some trouble connecting to the Mosquitto server, but the culprit was an old version of Mosquitto. Since the version in Raspberry Pi repositories is ancient, you should download it straight from Mosquittos repository (instructions here).

Supervisor – the big brother of small scripts

You can add the mqttwarn into your server init scripts, but I prefer Supervisor to handle all these small scripts, since it allows the administrator to monitor their status, see their output log and stop/start/restart them, all combined in a very nice web server UI.

To install it on Raspberry Pi, first install setuptools

wget https://bootstrap.pypa.io/ez_setup.py -O - | sudo python

Then install supervisor via easy_install:

easy_install supervisor

Update the /etc/supervisord.conf and check the http port and add username:password if needed. Also double-check that separate script config files are included:

[include]
files = /etc/supervisord.d/*.ini

Create /etc/supervisord.d/mqttwarn.ini and check that the mqttwarn installation directory and config files are referred correctly:

[program:mqttwarn]
directory = /home/pi/services/mqttwarn
command = /home/pi/services/mqttwarn/mqttwarn.py
user = pi
environment= MQTTWARNINI="/home/pi/services/mqttwarn/mqttwarn.ini"
stdout_logfile = /tmp/mqttwarn.log

Finally restart the supervisord

/etc/init.d/supervisor restart

And browse to http://192.168.1.80:9001/ and you should see the nice web GUI.

supervisord

From Z-Way to MQTT

We have now just configured one-way communication from MQTT to Z-Wave server. To be able for us to receive messages from various Z-Way devices, we must hack the Z-Way server Javascript a bit and bind to the JSON tree changes. I found about this trick from this EventGhost forum discussion and modified the scripts posted there (originals by Walter Krambring)

Edit /etc/opt/z-way-server/automation/main.js (depending on your installation directory) and add this snippet to the end:

var mqtt_host = '192.168.1.80';
var mqtt_port = 1883;
var mqtt_topic_prefix = 'home/zwave/';

// Here the 20 and 21 are hardcoded Z-Wave device ID's for wall plugs/power switches. Change them accordingly to your setup
var powerswitches = [20, 21];

function publish_mqtt (topic, key) {
    try {
        system(
            "mosquitto_pub",
            "-h",
            mqtt_host,
            "-p",
            mqtt_port,
            "-t",
            mqtt_topic_prefix + topic,
            "-m",
            key
        );
        return;
    } catch(err) {
        debugPrint("Failed to execute script system call: " + err);
    }
}

function switch_binary (device, instance, theValue) {
    console.log("MQTT plugin: dev#" + device + " (binary switch): " + theValue)
    state = 'on';
    key = 255;
    if (theValue == false){
        state = 'off';
        key = 0;
    }
    eventString = 'Device' + device + "/switch";
    publish_mqtt(eventString, state);
}

function switch_binary_meter (device, instance, theValue) {
    console.log("MQTT plugin: dev#" + device + " (binary switch meter): " + theValue)
    eventString = 'Device' + device + "/meter";
    publish_mqtt(eventString, theValue);
}

for (var i=0; i < powerswitches.length; i++) {

        var id = powerswitches[i];
        (function(devid) {
                console.log("MQTT plugin: Configure power switch  " + devid);
                zway.devices[ powerswitches[i] ].instances[0].SwitchBinary.data.level.bind(function() {
                        switch_binary (id, 0, this.value);
                });
                zway.devices[ powerswitches[i] ].instances[0].Meter.data[2].val.bind(function() {
                    switch_binary_meter (id, 0, this.value);
                 });
        })(id); // tie device ID so it is referenced correctly from callback funcs

}

This code will bind to changes in the Watt meter and on/off status of my Aeon and Evergreen wall switches and send MQTT message using external python script every time the data changes. The syntax for binding uses exactly the same data format as the Z-Way server JSON API, so we can use again the web interface to find the sub-item to bind with. However finding the correct item will indeed be bit like searching for a needle in a haystack if we use only the JSON tree. Instead of sifting through the giant data tree, we can use the Expert UI to find out clues where the interesting data is. For example in the case of Aeon switch we are interested in the Meter service:
z-way
Here we can see the scaleString of entry #2 is 'W' so .data2 should contain the watt meter reading. We confirm this by plugging in a known load (here 14W) and viewing the JSON tree:

http://192.168.1.80:8083/ZWaveAPI/Run/devices[20].instances[0].Meter.data[2]
returns
{"invalidateTime":1436389382,"updateTime":1436389383,"type":"empty","value":null,
"sensorType":{"invalidateTime":1435686498,"updateTime":1435686499,"type":"int","value":1},
"sensorTypeString":{"invalidateTime":1435686498,"updateTime":1435686499,"type":"string","value":"Electric"},
"val":{"invalidateTime":1435686498,"updateTime":1436389383,"type":"float","value":14.384},
"scale":{"invalidateTime":1435686498,"updateTime":1435686499,"type":"int","value":2},
"scaleString":{"invalidateTime":1435686498,"updateTime":1435686499,"type":"string","value":"W"},
"ratetype":{"invalidateTime":1435686498,"updateTime":1436389383,"type":"int","value":1},
"delta":{"invalidateTime":1435686498,"updateTime":1436389383,"type":"int","value":0},
"previous":{"invalidateTime":1435686498,"updateTime":1436389383,"type":"float","value":0}}

The '14.384' stored in 'val' sub-item looks awfully lot like our load. Thus the correct binding function will be

zway.devices[20].instances[0].Meter.data[2].val.bind

as shown in the example code above.

We must also allow the Javascript to invoke mosquitto_pub by adding .syscommands file to the same directory where main.js resides:

mosquitto_pub 

Openhab binding

Finally you can replace the z-wave binding with MQTT ones in the .items file:

Switch Kitchen_Coffee_Switch "Coffee machine" {mqtt=">[mosquitto:home/zwave/Device21/switch/set:command:*:MAP(switchToMqtt.map)]"} 
Number Kitchen_Coffee_Watts "Coffee machine power consumption [%.1f W]" {mqtt="<[mosquitto:home/zwave/Device21/meter:state:default]"}

Switch Livingroom_Power1_Switch "Living room power #1" {mqtt=">[mosquitto:home/zwave/Device20/switch/set:command:*:MAP(switchToMqtt.map)]"} 
Number Livingroom_Power1_Watts "Living room power #1 consumption [%.1f W]" {mqtt="<[mosquitto:home/zwave/Device20/meter:state:default]"}

We must also map the ON/OFF states to 1/0's that are understood by JSON API. Create configurations/transform/SwitchToMqtt.map:

ON=1
OFF=0

... aaand we're done! That was easy, wasn't it? 😀

Wall plug mania

To prevent my coffee pot from being burned every other weekday I had to be able to control wall outlets remotely and automatically via Openhab. For this I looked into into few possible solutions:

  • Build those outlets myself by modifying bunch of Kill-A-Watts, just like what Felix@Moteino Labs has done here with his WattMote or Mike on his Micromania Blog. I must say I was tempted to try experimenting with those, but since I eventually want quite a many of them, assembling those by myself seemed more like a chore. Also it is against the local Electrical Code to make permanent high voltage installations in here without proper license. Even though they are plug-in accessories, let’s be honest, they ARE permanent if one keeps them on 24/7. And besides, I wouldn’t want to sleep in a house full of DIY mains appliances – My sleeping disorder is bad enough as it is 🙂
Few pre-home automation era wall plugs. Don't these just scream for wireless connectivity!

Few pre-home automation era wall plugs. Don’t these just scream for wireless connectivity!

  • There’s a whole family of cheap 433 Mhz sensors, wall sockets/plug-ins, remote controls, doorbells and whatnot. Nexa is one of the manufacturers and their products are sold even here at local Clas Ohlsons. One could then buy for example Tellstick Duo or their more basic Tellstick USB stick, and connect it with RaspBerry Pi. Also Openhab supports Tellsticks natively. For complete device compatibility, see a list here. There’s also Tellstick Net, which is a ethernet connected stand-alone hub that doesn’t require another computer. I do like that modularity aspect, but the down-side is that it connects to their own Telldus Live! service (yuck!) and thus requires constant internet connection (another yuck!). Luckily there’s now an unofficial firmware available which removes that requirement.

    To my disappointment none of these 433 Mhz wall plug-ins (to my knowledge) are able to send their status or output power consumption to the hub i.e. you can only switch them on/off. Now that’s a bummer, since the use-case for turning off the coffee maker after some timeout period explicitly requires power consumption to be measured all the time (how else could the system know when the coffee maker has been manually turned on?)

    However the most concerning thing about these devices is that the security is a joke: all the wireless messages are transmitted without encryption so the attacker is able to just listen to the transmission outside your house and then replay the messages. Anyone still remember Wardriving? I can almost imagine script-kiddies driving around the city, looking for these devices and annoying the hell out of people by switching them on and off 🙂 In fact, there’s an interesting thesis written by students of Linkoping University that examines the security aspect of Tellstick Net. While it mainly concentrates on threats from (wired) network perspective and examines scenarios involving the Telldus Live! service, it’s interesting read. Also what’s funny is that they mention the wireless security only in one paragraph since the security flaw is so obvious they didn’t need the feel to research it more deeply 🙂

  • Wi-Fi 2.4Ghz remote wall plugs are available from D-Link, Belkin and Ankuoo (rebranded Netwjork here) just to name a few. The problem with these is that these look quite nice on brochures but actual user experience has been ranging from so-so to just horrible according to customer reviews on Amazon and other sites. The main annoyances have been flaky connectivity, pairing problems with WiFi router as well as 2.4 Ghz band collisions. I decided to opt out of this mess.

  • Finally there’s Z-Wave, a technology with a family of products that even remotely (pun intended) seemed usable. Using 868 Mhz band here in Europe (908 Mhz in the US) and providing encryption as well as better low-power support (compared to Wi-Fi products), this seemed the best pick out of options mentioned above. The only obvious down-side with Z-Wave devices is that they are quite expensive! Be it a simple flood switch, motion sensor or a wall switch, they all seem to cost a minimum of 40-50 euros / sensor. Since I wasn’t going to lock myself with only Z-Wave by using a proprietary hub, this wouldn’t be a problem. I can begin with wall plugs and then add more devices if needed should their prices come down.

Z-Wave jungle

For wall plugs there are quite many options available just to name a few:

In the end, they all seem quite alike, supporting power measurements beside remote switching. There’s also a new updated standard called Z-Wave Plus (another name for ‘Gen 5’ variety of Z-Wave products, such as the Aeon Smart Switch.)

According to the specs the Z-Wave Plus Features:

  • Significant increased range – up to 150m (clear air)
  • 50% improvement in battery life
  • 250% more bandwidth
  • Three F channels for improved noise immunity and higher bandwidth
  • New Plug-n-Play Network-wide Inclusion feature
  • Improved self-healing and fault tolerance with Explorer Frame feature
  • Standardised method for Over the Air firmware updates (OTA)
  • Improved product information capture for product certification database

This all looks good on paper, but there has been conflicting reports of success with Z-Wave Plus devices and Openhab and other DIY systems, so I decided to pick one older wall plug (AN158 by Everspring) without the Plus support as well as the newer Aeon Smart Switch Gen with Plus capability, both which are readily available here.

To enable communication between Z-Wave devices one needs a main controller that can include and exclude devices to and from the Z-Wave network as well as do bunch of other maintenance and configuration work in the network. When it comes to choosing the Z-Wave controller it all boils down to 2 options: Razberry and Aeon Labs Z-Stick Series 2. Both support Z-Wave Plus devices, but whereas the Z-Stick dangles outside Raspberry and occupies one of the USB slots, Razberry connects to Pis GPIO connector and slides nicely inside the enclosure. I chose the Razberry which seemed a nice idea at that time, but in hindsight Z-Stick maybe would have been better choice should I ever want to migrate away from Raspberry.

In a bit of shopping frenzy I bought also 2 wall switches WALLC_S Secure Wall Switch by Z-Wave.me, since there always will be some occasions when you cannot rely on automated scripts but want to do some manual switching instead. Even though these kind of switches could be easily built using a Moteino and 3.3V coin cell battery, I clearly lack the skill at this stage to create nice enclosures and thus didn’t want to stick any of my frankensteinos (Frankenmotes?) on walls or other visible surfaces 😀

First batch of Z-Wave devices to be tested

First batch of Z-Wave devices to be tested

Connecting with Openhab

For configuring Razberry to be used with Openhab I followed instructions from Home Automation For Geeks. I first tried HAbmin but I didn’t have any luck getting Z-Wave devices detected nor included in the network, so I resorted to the Z-Way server from Z-Wave.me (same manufacturer as Razberry). After fumbling around with the not-so-intuitive settings, all devices were finally set up nicely (or so I thought.. See the next post). After configuring devices the Z-Way server has to be shut down, since it cannot use Razberry simultaneously with Openhab.

First attempt was to use AN158 as coffee maker switch in kitchen. As before, items are defined in .items file (here Z-Wave device #3 is AN158)

Switch Kitchen_Coffee_Switch "Coffee machine" (GF_Kitchen) {zwave="3:command=switch_binary"} 
Number Kitchen_Coffee_Watts "Coffee machine power consumption [%.1f W]" (GF_Kitchen,GF_Energy) { zwave="3:command=meter" }

Then a rule for switching it off after 50 minutes can be set in coffee.rules. It automatically will detect when it has been switched on (consumption goes over pre-defined threshold value) and starts the timer.
import org.openhab.core.library.types.*
import org.openhab.core.persistence.*
import org.openhab.model.script.actions.*

var Integer coffee_watts_threshold = 5
var Integer coffee_timeout = 50
var Timer coffee_timer = null

rule "Coffee switch off"
    when
        Item Kitchen_Coffee_Watts received update
        then
        {
        logInfo("coffee switch", "consumption " + Kitchen_Coffee_Watts.state + " watts. " )         

        if ( ((Kitchen_Coffee_Watts.state as DecimalType).intValue() > coffee_watts_threshold) && (coffee_timer == null) )
            {
            logInfo("coffee switch", "Coffee on. consumption " + Kitchen_Coffee_Watts.state + " watts. Setting switch off timer to " + coffee_timeout + " minutes." )
            coffee_timer = createTimer(now.plusMinutes(coffee_timeout)) [|
                if ((Kitchen_Coffee_Watts.state as DecimalType).intValue() > coffee_watts_threshold)
                    {
                    logInfo("coffee switch", "coffee maker still on after timeout period. Switching off.. " )   
                    sendCommand(Kitchen_Coffee_Switch,OFF)
                    }
                    coffee_timer=null
                ]
            }
        else if ( (Kitchen_Coffee_Watts.state < coffee_watts_threshold) && (coffee_timer != null) ) 
            {
            logInfo("coffee switch", "Coffee off. Canceling switch off timer" )
            coffee_timer.cancel()
            coffee_timer=null
            }
        }
    end

To my surprise, it started to work out-of-the-box without any additional head scratching. No more burned coffee pots 🙂

Enter Moteino

Having played few days with Arduino Mini and the NRF24L01+ transceiver I came into conclusion that I wanted something more compact with lower power consumption. After some hours of coffee-powered research two top contesters surfaced: Jeenode and Moteino. Also there’s an upcoming ultra-low power arduino clone ULPNode which uses few quite interesting power saving techniques, that makes it possible to reach nanoampere scale basal power consumption when in deep sleep. Unfortunately prototypes are not yet available for purchasing but I will for sure try those out in the future!

Also one thing to consider is that these levels of ultra-low consumption can be implemented on a real-life sensor only when the sensor part itself can be put to sleep (for example with periodical temperature measurements). Since most of my sensor devices in planning will use at least one active sensor (PIR motion sensor mainly), it is irrelevant whether the board uses few microamps vs few dozen nanoamps when the PIR sensor itself uses 50 microamps in standby-mode.

Both Moteino and Jeenode use the low quiescent current MCP1703 voltage regulator and don’t include any kind of LED to indicate power on status (unnecessary power hogs), so low current consumption and long battery life could be obtained even without modifying the hardware.

In the end, I voted for Moteino for following reasons:

  • Compact and quite nice looking
  • RFM69W wireless chip (vs older RFM12B in Jeenode with no HW level encryption)
  • active user base and forum
  • few euros cheaper than JeeNode (not really a deciding factor, but nice to know if you are planning to spread your house full of these sensors..)

Testing

Testing Moteino on breadboard with FT232 USB adapter

Testing Moteino on breadboard with FT232 USB adapter

Here’s Moteinos sleep mode consumption after some testing:

  • 23 microamps (16 Mhz, BOD off, 5.0V to Vin)
  • 1.3 milliamps (16 Mhz, BOD off, 3.3V bypassing MCP1703).
    Wow, The regulator sure doesn’t like to be left idle 🙂
  • 22 microamps (16 Mhz, BOD off, 3.3V in, MCP1703 desoldered)
  • 20 microamps (8 Mhz, BOD off, 3.3V in, MCP1703 desoldered)

Note that also the RFM69 needs to be put to sleep (with the transmitter awake but idle I measured 1 milliamp higher values).

Whan transmitting packets the consumption jumps to few dozen milliamps, but this is inconsequential, since the transmitter and CPU will be in deep sleep 99% of the time, awakened only by interrupts from the sensor or hardware watchdog.

So, the setup and assumed current consumption is as follows (values for sensors are from datasheets, I didn’t test sensors separately):
* Moteino @ 16 Mhz: 23 uamps (I didn’t bother with 8 Mhz since at this stage the 2 microamp difference in consumption is negligible)
* 4xAA batteries (calculated later)
* DHT22 temperature and humidity sensor (40-50 uamps)
* PIR sensor (50 uamps)
* Voltage divider setup with a capacitor to measure 4-6V battery voltage (few microamps with 1 Mohm resistor)
* Motion detected LED (0 uamps off)

First prototype

I tossed the parts into left-over junction box. Better looking enclosure would have been nice, but unfortunately I don’t currently have tools nor working area in my current apartment to process suitable materials.

First prototype of bathroom motion sensor. This isn't going to win any beauty contests, but it works :)

First prototype of bathroom motion sensor. This isn’t going to win any beauty contests, but it works :)

Note that the 433 Mhz version of RFM69W seen here needs (or better said, is recommended to have) 17.2 cm antenna and is actually shipped with a 16,5 cm one since the PCB traces make up for the remainder. Now, in my tests I found out that I don’t need that long wire sticking out vertically, but can just route it inside around the edges of the enclosure. Even this gives it plenty of signal strength to transmit through 2 thick walls two rooms across. Good enough.

IMG_0053[1]

I measured the current consumption to be 104 microamps @ 5V, which is quite in the ballpark expected! With ideal situation (0% monthly battery discharge) this sensor would last for 4*2000 mAh / 0.104 = 76923 hours = 8.78 years 🙂 Of course now the self-discharge of NiMH batteries start to matter considerably: Cheap NiMH batteries have as high as 20-30% discharge rate, whereas better Sanyo Eneloops (3rd gen) are specified to hold 70% charge after 5 years! Now this would be an almost set-and-forget situation. Actually I just recently ordered a pack of Eneloops, so I think this is going to end up being a field-test lasting for few years 😀

Pairing with Openhab

Now, let’s concentrate on software a bit. Since MySensors supports only the NRF24L01+ and it’s libraries seem to be quite tightly coupled with it’s hardware functionality, I had to find a replacement. After some digging I came across this nicely done pack of libraries for MQTT Gateway and Sensor made by Computourist. With just a few tweaks I was able to fire up (not literally) the sensors and get them to communicate with Openhab. Unlike MySensors MQTT gateway, this one isn’t MQTT server but just normal client so no MQTT server bridging was necessary. Just configure the MQTT server IP address:port in the sketch and Moteino gateway connects automatically and will also keep on trying should it lose the connection to the MQTT server.

MQTT Gatewey will send and receive messages in a fixed notation such like these:

pi@raspberrypi:/opt/openhab# mosquitto_sub -v -t '#'
home/rfm_gw/nb/node02/dev02 -61
home/rfm_gw/nb/node02/dev04 5.2
home/rfm_gw/nb/node02/dev48 24.38

Here nb=northbound (messages from nodes/sensors to the server), sb=southbound (messages from server/openhab to nodes). Devices (items residing inside a node) will be defined per-node basis and could be anything from sensors, actuators to node setting containers. For example commands

pi@raspberrypi:/opt/openhab# mosquitto_pub -t home/rfm_gw/sb/node03/dev00 -m "READ"
pi@raspberrypi:/opt/openhab# mosquitto_pub -t home/rfm_gw/sb/node03/dev04 -m "READ"

could be used to read uptime and battery voltage of node #3.

pi@raspberrypi:/opt/openhab# mosquitto_pub -t home/rfm_gw/sb/node03/dev01 -m 60

Could be used to set periodic transmission interval to 60 seconds. Actual implementation depends on sensor, but these libraries give quite nice framework to work upon.

To accept messages from MQTT Gatewey we just define new items in the Openhab .items file to match the new sensors:

Contact Bathroom_Motion_Sensor {mqtt="<[mosquitto:home/rfm_gw/nb/node03/dev40:state:MAP(switchFromMqtt.map)]"}
Number Bathroom_Temperature  "Bathroom temperature [%.1f C]"   <temperature> {mqtt="<[mosquitto:home/rfm_gw/nb/node03/dev48:state:default]"}
Number Bathroom_Humidity  "Bathroom humidity [%.1f %%]" {mqtt="<[mosquitto:home/rfm_gw/nb/node03/dev49:state:default]"}
Number Bathroom_Sensor_Voltage  "Bathroom sensor voltage [%.1f V]"  {mqtt="<[mosquitto:home/rfm_gw/nb/node03/dev04:state:default]"}

We also need to map the state of the motion sensor (ON/OFF) to a state shat the "Contact" item understands (CLOSED/OPEN). That is done in switchFromMqtt.map (referenced above with MAP(switchFromMqtt.map):

ON=CLOSED
OFF=OPEN

PIR sensor and light rules

Then some timer magic is created to turn on the bathroom light when motion is detected:

import org.openhab.core.library.types.*
import org.openhab.core.library.items.*
import org.openhab.core.persistence.*
import org.openhab.model.script.actions.*

var Timer BathRoomTimer = null
var Integer timeOut = 120

rule "Bathroom Motion Sensor Trip"
        when
        Item Bathroom_Motion_Sensor received update
        then
        {
        logInfo("Bathroom motion Sensor", "sensor changed state")
        if (BathRoomTimer == null ) 
            logDebug("Bathroom Motion Sensor","timer==null")
        else                        
            logDebug("Bathroom Motion Sensor","timer!=null")
        if(Bathroom_Motion_Sensor.state == CLOSED)
            {
            if (BathRoomTimer == null ) 
                {                       
                // create timer
                logInfo("Bathroom Motion Sensor", "creating timer: " + timeOut + " sec")
                BathRoomTimer = createTimer( now.plusSeconds(timeOut) )
                    [ 
                        logInfo("Bathroom Motion Sensor", "timer expired, switching off ")
                        sendCommand(Bathroom_Light_Toggle,OFF)      
                        BathRoomTimer=null
                    ]               
                sendCommand(Bathroom_Light_Toggle, ON)
                sendCommand(Bathroom_Light_CT_Dimm, Light_Temperature.state as DecimalType)
                } 
            else
                {
                logInfo("Bathroom Motion Sensor", " rescheduling timer:" + timeOut + " sec" )
                BathRoomTimer.reschedule(now.plusSeconds(timeOut))
                }
            }
        }
    end

Everything seemed to work just fine, but when I took a shower in the morning, PIR sensor couldn't obviously detect any movement behind shower curtain and eventually the light was turned off again, me still in the shower 🙂 Hmm.. Either I could extend that delay period to few minutes (lazy option), OR I could use humidity sensor to detect rise in humidity above a certain threshold (when shower is being used) and extend the delay only then. I chose the latter, more elegant option:

var Integer timeOutWithShower = 300

rule "Bathroom Motion Sensor Humidity"
    when
        Item Bathroom_Humidity received update
    then
    {
        logInfo("Bathroom Humidity Sensor", "bathroom humidity changed")
        if( ( BathRoomTimer != null) && (Bathroom_Humidity.state > 60) )
        {
            logInfo("Bathroom Humidity Sensor", " shower on - rescheduling timer for bathroom in " + timeOutWithShower + "sec" )
            BathRoomTimer.reschedule(now.plusSeconds(timeOutWithShower))
        }
    }
    end             

This required also changes sensor-side, since in normal mode humidity would be measured only every few minutes. I implemented a "perceptive mode" to start measuring temperature and humidity every few seconds after motion is detected. Mode is then switched back to normal after 1 minute. This gives enough time for humidity sensors to react (in fact humidity goes over 50-60% in just 20 seconds after shower is turned on.)

Next coming: Dash of Z-Wave sorcery!

Let there be (dim) light!

There’s no fun in having sensors without being able to manipulate surrounding environment using their input. Like I explained in first post, my main motivation setting up home automation was to control lighting, preferably without minimal human intervention. Main illumination rules should be as follows:

    • Motion sensors should turn on lights automatically after person has entered a room. Lights should also be turned off after no presence has been detected after a while (5-10 minutes perhaps)
    • When on, lights should adjust to local time by dimming and if possible, changing color temperature. From early morning to 6 pm lights should be set to bright and neutral temperature (6000-7000 K), then declining towards the evening (around 2500-3000 K) and finally drop to 2000 K 1-2 hours before bedtime
    • If alarm is set to go off on mornings, bedroom lighting should be turned on automatically 30 minutes before, starting from very dim and slowly reaching full brightness wee bit before it’s time to get up.

Options, options…

I considered wireless dimmer modules (either off-the-shelf or even DIY Arduino hacks), but quickly scrapped the idea, since color temperature could not be then adjusted. Belkin has a family of WeMoÂŽ home automation appliances (using 2.4 GHzZigbee as wireless protocol), including their Smart LED Bulbs. Unfortunately their color temperature is fixed. There are however 3rd party bulbs available from OSRAM that are combatible with Wemo and have tunable color temperature. Osram manufactures also their own Zigbee compatible Lightway gateway. And finally there’s Philips Hue family of lights, that are both color temperature and color adjustable. Hue is also using Zigbee protocol and is the oldest of these three, having been around almost 2 years now already.

Since all three use Zigbee to communicate between hubs and light bulbs, I would assume there’s some interoperability between these products. It seems at least Osram Lightify bulbs can be paired with Philips Hue hub, but I didn’t found anyone mentioning compatibility between Belkin and Hue bulbs. However in the end it is the hub that the rest of the home automation framework is communicating with and not directly with the bulbs, so procurement was done mainly by comparing the existing support for the hubs in Openhab and other home automation systems. In the end I chose Philips Hue since Openhab supports it directly, its protocol is documented and there’s plethora of language bindings available.

philips-hue-blubs-and-station
There were also many success stories around using Philips Hue with DIY installations that encouraged me to shell out the 200 euros for the starter pack with the hub and 3 bulbs. I know, it’s a rip-off but then again, I have so many times found out that cheaping out on these things often cause major headaches later.

Philips Hue

philips hue app
There are many features on their mobile application (available for iOS and Android. Unofficial app also for Windows phones available). Not only can you control manually the lights with the application, but you can also set dimming functions based on timers and alarms which are saved into the hub, so the application doesn’t need to be left on. Then there are other lots of 3rd party apps available (Disco lights etc.) but those just feel utterly gimmicky. And besides, after one has been playing with a new toy such as this for awhile and the initial charm has worn off, who really wants to control his/her lights from a mobile phone?? Anyway, I won’t go into too much details with the features of Philips Hue itself since there are already many nicely done reviews around, such as this by Lester Chan.

Now, coming back to the initial objectives, using the timer feature I could fulfill some of the requirements above, but I still wanted to gather all the automation rules into one single hub (Openhab) to avoid conflicts later. In the end, Philips hub was converted into a dumb router. One last thing I did with the Hue application was to check the IP address of the hub after which I went on to configure Openhab.

Philips Hue hub, emitting exactly the light that would keep me awake at night should I spent too much time glancing it. Fortunately it can be put somewhere out of sight.

Philips Hue hub, emitting exactly the light that would keep me awake at night should I spent too much time glancing it. Fortunately it can be tossed somewhere out of sight.

Openhab coupling

Pairing Philips Hue hub with Openhab was quite straightforward (detailed instructions can be found here), but setting the lightning rules were little bit trickier. I’m not really a big fan of the XTend language that Openhab uses for its scripting, but that’s a topic for another post..

First one needs to define the light bulb items (e.g. in default.items)

// Simple ON/OFF toggles
Switch Kitchen_Light_Toggle    "left bulb"   (Switching) {hue="1"}
Switch Bathroom_Light_Toggle   "center bulb" (Switching) {hue="2"}
Switch Bedroom_Light_Toggle    "right bulb"  (Switching) {hue="3"}

// For setting color values
Color Kitchen_Light_Color    "left bulb"   (Colorize)  {hue="1"}
Color Bathroom_Light_Color    "center bulb" (Colorize)  {hue="2"}
Color Bedroom_Light_Color     "right bulb"  (Colorize)  {hue="3"}

// For setting dimmer values
Dimmer Kitchen_Light_Dimm     "left bulb"   (WhiteDimmer)   {hue="1;brightness;30"}
Dimmer Bathroom_Light_Dimm    "center bulb" (WhiteDimmer)   {hue="2;brightness;30"}
Dimmer Bedroom_Light_Dimm     "right bulb"  (WhiteDimmer)   {hue="3;brightness;30"}

// For setting color temperature values. Note that these accept values (0-100), where 0 is the coldest and 100 is warmest temperature
Dimmer Kitchen_Light_CT_Dimm      "left bulb"   (CTDimmer)  {hue="1;colorTemperature;30"}
Dimmer Bathroom_Light_CT_Dimm     "center bulb" (CTDimmer)  {hue="2;colorTemperature;30"}
Dimmer Bedroom_Light_CT_Dimm      "right bulb"  (CTDimmer)  {hue="3;colorTemperature;30"}

Lighting rules

Then we define a global color temperature setting (also in default.items) as well as target temperature for slow transition:

Number Color_Temperature "Light temperature [%.0f]"
Number Target_Color_Temperature

Next we define a function (I use light.rules files for this and the following rules) that gets called every time a light temperature is updated. It then controls each light bulb accordingly. Note that if a light is turned off, this setting is not propagated to the bulb itself so we must explicitly set the temperature each time when a light is turned on.

rule "Update color temperatures"
when 
    Item Color_Temperature received update  
then
    logInfo("Sensors","new color temperature: " + Color_Temperature.state)
    var Integer temp = Color_Temperature.state
    sendCommand(Kitchen_Light_CT_Dimm, temp)
    sendCommand(Bathroom_Light_CT_Dimm, temp)
    sendCommand(Bedroom_Light_CT_Dimm, temp) 
end

I also tried to address each dimmer in a group (set earlier in the items definition) like this…

//  CTDimmer?.members.forEach[dimmer|
//      Dimmer d = dimmer   // <-- Fails right here..
//      sendCommand(dimmer, Color_Temperature.state )
//      ]

... but it seems that Openhab doesn't store group items as real objects but as some kind of bastard string representation instead.. So, each light has to be addressed individually, which makes maintaining code cumbersome especially when one has many devices of same type in the setup.

Another annoyance has been with default item values. I haven't found a way to initialize values in items file, and when I modify the file and it gets reloaded, values are reset and left uninitialized. This can break many functions and adds overhead since there has to be more checkpoints all over the code to make sure no uninitialized value is addressed. I even wrote another rule to try to circumvent this:

rule "Startup"
    when 
        System started
    then
    if ( (Color_Temperature.state == Undefined) || (Color_Temperature.state == Uninitialized) )
    {
        postUpdate(Color_Temperature, 50)  // default value
    }
    end

, but this is called only during startup or when the rules file is reloaded, but NOT when items file is touched. This can be circumvented by saving first the .items file and then the .rules file. It is of course not a problem when running a stable system but when testing and building your rules, this can become quite a pain in the ass.

To implement a transition from current light temperature to target temperature slowly in 10 minutes, we implement a rule such as this:

rule "Temperature transition"
when
    Item Target_Color_Temperature received update
then
    var Number steps = 0
    logInfo("temp transition","temp:" + Color_Temperature.state + " -> " + Target_Color_Temperature.state )
    var Number delta = (Target_Color_Temperature.state as DecimalType).floatValue - (Color_Temperature.state as DecimalType).floatValue
    delta = delta / 60
    logInfo("temp transition","delta:" + delta)
    while(steps < 60) { 
            postUpdate(Color_Temperature, (Color_Temperature.state as DecimalType).floatValue + delta )
        steps = steps + 1 
            Thread::sleep(10000)
     }
end

Finally, we can add rules to start the transition at certain times (in this example at 8 pm and 7 am)

rule "Dusk"  
when
    Time cron "0 00 20 ? * MON-SUN "
then
    logInfo("Light rules","It's getting darker...")
    postUpdate(Target_Color_Temperature, 100)
end

rule "Dawn"
when 
    Time cron "0 00 07 ? * MON-SUN "  
then
    logInfo("Light rules","It's getting brighter...")
    postUpdate(Target_Color_Temperature, 10) 
end

Evolution of Architecture (Part 1)

My first contact with home automation came via MySensors, an project that combines Arduino with various sensors and NRF24L01+ wireless 2.4Ghz module. Lots of example code for both sensors and transmitter relays can be found on their site and thus deploying working prototype was quite straightforward. In just few hours of work I was able to construct simple wireless temperature sensor and it’s counterpart, a relay hub that receives temperature information wirelessly. I first used Dallas DS18B20 temperature sensor, but then decided to swap it to DHT22 that also gathers humidity values.

MySensors architecture supports plethora of home automation hubs where you can program the actual rules and behaviour of the system. Hub gathers data from sensors, can react to certain values and conditions and then manipulate lights, switches and other devices in the network. Also data storing and charting is possible. I decided to try out Openhab, a quite mature software package that supports various devices and protocols and seems to be quite popular in the DIY home automation scene. It’s written in Java and thus speed isn’t its best feature, but many people are running it in RaspBerry Pi so I figured it can’t be that slow. In fact running it wasn’t that bad and response times to stimuli were so-so but starting and re-starting the server took several minutes. So trying different settings by modifying openhab.cfg made me pull not-insignificant amount of hair out of my head. I will blame Openhab for my baldness later. Luckily Raspberry Pi 2 was released just few months ago and with its faster processor running Openhab turned out to be much nicer for my scalp.

MQTT magic

After some research I decided to pursue the first prototype:

Home automation architecture #1

Home automation architecture #1

MySensors supports most hubs via MQTT and the same goes for Openhab. There are lots of great tutorials around so I won’t go into details, but let’s just say that if there’s one protocol to consider, it’s MQTT. It’s a lightweight messaging protocol that follows publish-subscribe principle and allows devices on different platforms to talk to each other. Sensors can publish sensor data to MQTT message bus and automation hubs can subscribe to those messages. For example sensor might publish it’s temperature with following message:

topic: "/home/sensors/node02/temperature" msg:"23.50"

It’s also possible to control the devices (switching relays or turning on/off the lights) by sending MQTT messages as long as the device can talk MQTT (or there’s a translating gateway sitting between the MQTT bus and the target device). It’s nice to also configure sensors via MQTT: For example if you want to change the sensor update interval, you can just send a message to MQTT bus instead of going through the hassle of deploying a separate configuration tool that talks to the device, or even re-flashing the firmware via direct physical connection (there are wireless flashing solutions but that’s a subject of different post). In a nut-shell, MQTT is the glue that in optimal situation allows us to avoid tinkering with cumbersome ad-hoc DIY protocols as well as to prevent reliance on monolithic systems in which direct support for each device platform and protocol is required.

Do electromagnetic waves talk MQTT?

Now, how to get the sensors talking to Openhab via MQTT? Actually it’s best to not transmit MQTT messages wirelessly, but instead use some even simpler, fixed message structure from sensors to Sensor Relay, which then converts these messages to actual MQTT messages to be sent to MQTT server. This requires little bit more coding on the Sensor Gateway side, but then makes it possible to off-load all the MQTT message parsing and publish/subscribe logic from sensors to the relay. MQTT is simple but not THAT simple, and these low-power and low-memory sensors are best to be kept as simple as possible for obvious reasons. MySensors has a published a simple Arduino MQTT Gateway Broker sketch that can work almost straight out-of-the box with minimal code changes. It is actually a small MQTT server itself, but is not as robust as I would’ve liked, so I decided to install Mosquitto MQTT server on Raspberry Pi that works as a main MQTT hub. Mosquitto was then configured to work as a bridge with Arduino Gateway, i.e. it connects to the Arduino gateway as any normal client, subscribes to all relevant topics (sent by sensors) and publishes then all the messages coming from other devices. Bridging can be done by modifying /etc/mosquitto.conf and adding following snippet:

connection mysensors_mqtt
address 192.168.1.82
topic # both MySensors/ /

This subscribes to all topics (#) and replaces ‘/’ with ‘MySensors/’ in topics coming from Arduino Sensor Gateway and vice-versa. This topic translation makes it easier to distinguish MySensors MQTT messages from other MQTT traffic.

Adding the sensors to Openhab is then pretty straightforward: First install and configure mqtt add-on, then add each sensor sub-device as a MQTT source by editing /opt/openhab/configuration/items/default.items like this:

Contact Bathroom_Motion_Sensor {mqtt="<[mosquitto:MySensors/20/0/V_TRIPPED:state:default]"}
Number Bathroom_Temperature "Bathroom temperature [%.1f C]" {mqtt="<[mosquitto:MySensors/20/1/V_TEMP:state:default]"}
Number Bathroom_Humidity "Bathroom humidity [%.1f %%]" {mqtt="<[mosquitto:MySensors/20/2/V_HUM:state:default]"}

Of course these are just examples, and you have the  modify the MQTT topics above to match those that your sensors are sending.  Sniffing the MQTT traffic is easy with mosquitto_pub:

pi@raspberrypi /home/pi $ mosquitto_sub -v -t MySensors/#

For some reason “-t #” is not accepted, but “-t /#” is, but the latter does match only messages beginning with ‘/’, so if you want to subscribe to absolutely all messages, the ‘#’ must be closed inside quotation marks:

pi@raspberrypi /home/pi $ mosquitto_sub -v -t "#"

Next topic: Actually doing something practical with the sensors. Stay tuned!

Here we go!

Home automation, Internet of Things, MQTT, Arduino, Wireless communication. Raspberry Pi, Low Power computing. Web hosting on Digital Ocean. Docker…

There are vast number of technologies and solutions that I’ve been tinkering with these past few months after I decided to embark on a journey to automate my home. I’m that guy that quite often forgets to turn off the lights when leaving home, or leaves the coffee machine on when rushing to work in the mornings. Also I’m quite sensitive to white/blue light in evenings which dampens the endogenous melatonine production and thus messes up my circadian rhythm.  I have already utilized the magnificent F.Lux application to filter out the blue spectrum of light on my computer monitors, but I also wanted to do something about the in-house lightning by dimming the lights 2-3 hours before planned bed-time.

Of course, practical benefits aside, I do like to tinker and hack at various things. Buying an off-the-shelf product seemed boring, and I didn’t want to tie myself with any closed black-box solution with vendor lock-in, so I decided to use as much open-source parts as possible. Cost, open standards, security, flexibility and robustness were all evaluated.

Countless of hours of researching, planning, building, debugging, failing, re-building and re-failing, and sometimes also succeeding. After all the hard work, I wanted to document some of my findings, sob stories and victories on this site. Maybe readers can then avoid some of the pit-falls and time-wasters that I’ve had to go through. Of course, most of the information here is  duplicated on many forums and sites and countless home automation projects, but some of that information is either out-dated or scattered all over the web. Also I did come across few problems whose solution were not obvious or wasn’t applicable to my own configuration.

But this site is not only about home automation. I’ll try to cover also more general topics, such as electronics, computer science, networking solutions, product reviews and whatnot. I hope you find at least some of the content here helpful.

Welcome!