Things to do with your network-connected Denon/Marantz receiver

This post will focus on the things you can do with your network-connected Denon/Marantz receiver. I bought my Denon AVR-3808 in 2007, so it’s actually my very first IoT device. Unfortunately my receiver doesn’t support the newer HTTP/XML-based protocol or streaming services, but the old telnet-based AVR protocol still gives a lot of opportunities.

So without further ado, a walk-through of options I know of…

App control
Denon doesn’t support the AVR protocol in their app, so third party apps are the only options for me. If you have a newer receiver you won’t have that problem. I’ve been using AVR-Remote for Denon/Marantz for years. It’s a bit old, but still works.

Control with Yatse
Another use within Android is volume control and more within Yatse. Yatse is a full-featured Kodi remote control app and much more. If you have a Kodi media center and an Android device, you should definately give Yatse a chance. And if you use Yatse and own a Denon/Marantz receiver, you should try the Denon/Marantz plugin. Please note that this is a shameless recommendation, since I’m the developer of this plugin.

Control with Tasker
Into automation and using Tasker? Yet another use within Android. Use Denon/Marantz plugin to automate interaction with your receiver. Yes, this is the same plugin as mentioned in previous chapter. Example usages:

  • Mute receiver when phone rings.
  • Use as clockradio: Turn on receiver on radio at 6:00 in the morning.
  • Turn off the receiver when leaving home.
  • Well, whatever you can think of.

openHAB
Staying with automation, integrate your receiver into your home automation when using openHAB. I’m not using this binding myself, since it constantly occupies the only connection available for the AVR, so it would block all other usages. If you have a newer receiver, you won’t have that problem. Instead, I can integrate through the Logitech Harmony binding.

Linux shell script
I wrote this small script years ago and scheduled it to to be run by cron daemon on my server at midnight. Purpose was to save energy in case I’d forgotten to turn off my receiver. It demonstrates how to send a simple command from a shell script, but other than that it’s obsoleted today by openHAB rules or similar.

#!/bin/bash
# Written by Jacob Laursen , 2009-2011.
#
# This script turns off a Denon receiver if none of the listed
# hosts to be checked are up and running.
#
# Options:
# --force : Skip host checking, just turn it off.

HOST_CHECK=`echo {"ps3","smarttv"}`
HOST_DENON="denon"

standby ()
{
        {
                sleep 2
                echo -n -e "PWSTANDBY\r"
                sleep 2
        } | telnet >/dev/null 2>&1 $HOST_DENON
}

if [ "$1" == "--force" ]; then
        standby
        if [ $? == 0 ]; then
                echo "Power turned off"
        else
                echo "Already turned off"
        fi
        exit
fi

host_up=0

for host in ${HOST_CHECK}
do
        ping >/dev/null -c 1 $host

        if [ $? == 0 ]; then
                host_up=1
                break
        fi
done

if [ $host_up == 0 ]; then
        standby
fi

Logitech Media Server/Squeezebox plugin
For controlling volume from my Squeezebox Receiver I’m using DenonSerial v0.1.42 extension by Peter Watkins. I’m not sure where to find it today, as it’s quite old – but still working nicely. An alternative might be DenonAvpControl, but I haven’t tried that.

The purpose of this extension is to set fixed volume to 100% on the Squeezebox Receiver and instead control volume on the Denon receiver. Using this extension should work no matter how you control your Squeezebox Receiver. However, the app (or whatever) must support sending volume commands even for players with fixed volume configured. The Android app Squeeze Ctrl supports this. The ancient, but still excellent, Squeeze Commander does not.

Miele XGW3000 firmware 2.3.4

New firmware, this time noticed a bit late, since XGW3000_version.txt is still showing 2.3.0. I guess I should switch to http://GATEWAY/Rest/Update/ and check “AvailableVersion”, or once again reverse engineer how the gateway checks the version. Anyway, here goes…

Release 2.3.4 – 12.09.2018

  • Improved Miele@mobile behavior in conjunction with ZigBee appliances
  • Stability improvements and bugfixing

Good news is the this fixes the Miele@mobile app so it doesn’t crash anymore on startup. I guess this reveals that they changed the API between the app and the gateway in app version 2.9.4 and didn’t provide backwards compatibility for the existing gateway API. So the app was constantly crashing until the gateway firmware received an update. Not elegant.

First observation: It seems like phases for my oven (H 5581 BP) isn’t supported anymore. Running self-clean/pyrolysis:

Homebus

<information>
<key name="Appliance Type" value="Oven"/>
<key name="State" value="Running" type="state" raw="5"/>
<key name="Remaining Time" value="1:21" type="duration" raw="81"/>
</information>

Rest

{"SignalFailure":false,
"TargetTemperature":[0,0],
"Temperature":[0,0],
"SignalInfo":false,
"ProgramID":2348810240,
"InternalState":0,
"RemainingTime":[1,24],
"RemoteEnable":[0,0,0],
"Status":5,
"ProgramPhase":3072,
"ProgramType":1,
"ExtendedState":"000C100500110000000100000000110400000000000000000018010000000000000000000242000000000000400000000000000000000000000000",
"ElapsedTime":[0,42],
"SignalDoor":false,
"Light":0,
"ProcessAction":0,
"StartTime":[0,0]}

These are my notes for ProgramPhase 3072 and pyrolysis:

			case 3072:
				// Purpose unknown, seems to be set when the oven is just turned on.
				// Catch it here to avoid excessive logging.
				return setPhase(PHASE_UNKNOWN);
[...]
			case 3076:
				return setPhase(PHASE_PYROLYSIS);

Also, temperature is now reported as 0°C. Removing functionality for my oven is just sad, since it’s the appliance with least functionality already (for example, ProgramID’s are not supported). Here’s how it looks in Miele’s own app now:

Miele@mobile screenshot

Update/precision: This bug was probably caused by the fact that I upgraded the firmware on the gateway while the oven program was running. Since the phase didn’t change after the gateway rebooted, the gateway probably had some uninitialized values. I guess it should be possible to query the devices after a boot to get updated values, but I don’t know which limitations may exist. So the bug is there, but it’s rare.

Miele XGW 3000 gateway firmware 2.3.1

New firmware the other day. First it was 2.3.0, then quickly after 2.3.1:

Release 2.3.0 – 01.08.2018

  • Improved device sign on
  • Stability improvements and bugfixing

I wonder what’s in 2.3.1, but probably something was broken in 2.3.0, then quickly fixed.

No changes observed so far. Known bugs persist, new ones not discovered yet. That’s not entirely true. I still don’t know if Miele fixed the random change of name of some appliances. In my case it’s always the dishwasher (my only Wi-Fi device), and I have seen it change to “Oven” or “Washing machine”, i.e. names of my other appliances. Update: This bug is still present.

The app

The Miele@mobile app for Android also received an update a few days later (Wednesday, August 8th): version 2.9.4. This app has generally been of great quality, but this time it’s completely broken:

  • Always crashes on first startup. Never happened before.
  • Notifications doesn’t work anymore.
  • The appliance list sorting algorithm is changed from logical to random, e.g. turned off appliances at the top while other appliances are running.
  • Wrong reporting of power consumption. Reports 1.0 kWh when it should be 0.1 kWh (washing machine).
  • Oven program is reported as “APP_MISSING_LOCALIZATIONKEY_-1946157056 (H 5581 BP, XKM 3000 Z). The JSON value for ProgramID is weird (e.g. “2348810240”), don’t know if this bug is in the ZigBee module, the oven or the gateway. However, the app didn’t show this before.
  • When trying to edit product name to default: Object reference not set to an instance of an object.

It seems someone was back from vacation and decided it was time to push out a new release because of the new Miele logo. But forgot that it was still untested. Please fix. Rescue: Version 2.6.0

Miele XGW 3000 gateway firmware 2.2.0

I upgraded last week and today I investigated this firmware version a bit. From the release notes:

Release 2.2.0 – 21.02.2018

  • Stability improvements and bugfixing

Bugs I have found to be fixed

  • In the Homebus 1.0 protocol, Wi-Fi devices will no longer return HTTP code 500 (internal error) when in state “Waiting to Start” (numeric value: 4)

Bugs (still) not fixed

  • The additional name for my dishwasher (“Dishwasher”), keeps changing to “Washing machine” from time to time.
  • The ElapsedTime value for my Wi-Fi dishwasher is always:
    "ElapsedTime":[0,0]

New bug found: After a power outage today, my washing machine showed up as “mac-001d65feff02237b” (obscured a bit due to general paranoia) instead of its UID. I was still able to access it through http://GATEWAY/Rest/Devices/mac-001d65feff02237b/Ident, though. After another reboot, it was back to normal.

Controlling IKEA Trådfri bulbs with Philips Hue bridge

I wanted to be able to control a specific lamp with my Logitech Harmony remote, so I did bit of research. I wanted to join the Philips Hue ecosystem, but haven’t been able to find the bulbs I need so far. For this specific lamp I needed an E27 bulb with 1,000 lumen and preferable a CRI higher than 90. Otherwise I would compromise the quality of the light just for playing around. If I was to going to play, I should at least be able to maintain what I already had.

I found out Philips Hue as well as IKEA Trådfri actually implements the same standard: Zigbee Light Link (ZLL), and should be able to work together. Well, at least since recently. From what I learned, IKEA seriously fucked up in their first versions which supported only Zigbee Home Automation (ZHA) instead of ZLL. And at the other end, Philips removed support for ZLL to ZHA fallback. So, now I needed a pretty recent bulb, or a firmware update of the bulb with an IKEA Trådfri gateway I wasn’t planning on buying.

Products
I bought the following to get started:

Adding the bulb
I followed this guide to pair the IKEA bulb with the Hue bridge. My bulb was from batch 1746 (printed on the box close to the barcode) and is firmware version 1.2.214 which is good enough (should be at least 1.2.x). I had to retry the pairing process a few times, probably because I didn’t turn the bulb on/off fast enough (six times) when trying to reset it. But finally it showed up in the Hue app, and I was able to turn it on/off as well as dim it. Excellent!

Philips Hue Bridge 2.1
The bridge was super easy to configure and just works. I haven’t played with API’s etc. yet, but can enjoy the big Philips Hue ecosystem and endless integration possibilities. I see absolutely no reason to start building a system based on the relatively new IKEA gateway when you only need the slightly more expensive Hue bridge to use everything developed for the Hue system.

Integrations
First thing after being able to control the bulb with the Hue app and the Dimmer switch was to set up my Harmony remote. This turned out to be difficult because of bugs in the Logitech Harmony software. I already had Hue set up for a Hue emulator running on openHAB, so I removed this. After doing so, the Harmony app kept detecting the Hue bridge without being able to pair with it, even when disconnecting both openHAB and the Hue bridge from the network. The trick/work-around was to search for a Hunter Douglas PowerView Hub. Cancel the search, then try again to search for the Hue bridge. After this I was able to pair with it. The same problem occurred after changing IP address of the Hue bridge. Having two bridges is not supported and you can’t even configure the right one if you actually have two – so one of them must go offline before configuring Harmony. Great work, Logitech.

Anyway, after setting it up, it was very easy to configure the “Watch TV” activity to dim the bulb down using the pre-configured Nightlight scene. However, this immediately turned out to be a problem: If the bulb is already off when turning on the TV, it should stay off. It seems that even simple rules like this requires more sophisticated methods.

openHAB
In openHAB 2.2 it was very easy to get the Philips Hue binding working, and the first project was already defined: Dim the bulb to 30% when turning on the TV and the current dim value is higher than 30%. First thought was to use the Hue emulator in Hue, so the Harmony would control this and then build the logic in openHAB from there. However, I decided to simplify this while at the same time decoupling the logic from the Harmony remote and make it generic. By using the Samsung binding for my TV (alternatively a network binding would suffice), I could create an item-based rule like this:

rule "TV"
when
    Item TV_is_on changed from OFF to ON
then
    if (DimmableLight1_Brightness.state > 30)
    {
        DimmableLight1_Brightness.sendCommand(30)
    }
end

Power consumption

  • Philips Hue Bridge 2.1: 1.7 W
  • IKEA Trådfri E27 1000 lumen LED bulb: 13.0 W (on)
  • IKEA Trådfri E27 1000 lumen LED bulb: 0.3 W (off)

Future
Philips is planning on supporting Zigbee 3.0 in the bridges in first quarter of 2018. This will be backwards compatible with ZLL, so IKEA bulbs should continue to work, since they now got the ZLL implementation right with the latest firmware.

Playing with WeMo Insight Switch

I finally couldn’t resist getting my hands on a WeMo Insight Switch anymore. It’s a kind of expensive gadget, considering how clumsy it is and that all it can do is turn one thing on or off. Also, I read in some specifications that it was consuming a considerable amount of energy itself. However, I’m the unlucky owner of a Thomson modem from the Danish ISP Stofa, and this thing consumes about 12.4 W, so I’ve been wanting to do something about that for a while, since I’m only using it for my backup internet connection (WAN 2) and for IP telephony.

Unpackaging
Just kidding. After connecting the switch you have to install the 33 MB app from Belkin to get it up running. After this I searched the net for API’s and alternate ways to control it. Half an hour later I had a working curl command to turn it on or off, as well as a working Tasker configuration using RESTask for Tasker, which I just found for the job (since Tasker doesn’t seem to support setting custom headers in HTTP POST requests).

The hardware
Before jumping to my first project/solution, a few words on the hardware:

  • Power consumption: When the relay is off, it consumes approx. 1.5 W. When on, approx. 1.7 W.
  • Socket: The socket is a Schuko socket, which is problematic for the Danish market, since you cannot plug in the traditional Danish flat and round connectors – they physically don’t fit in.

Controlling WeMo from Tomato
Since my Tomato router firmware (by Shibby) comes with curl preinstalled, controlling WeMo from the router is not a problem. What I wanted to do was having my router turn on the modem for WAN 2 when WAN 1 is down. I decided to ping the DNS server to check if I’m connected to the internet, and to do this once a minute. So in Administration/Scheduler I added this custom script to run every minute:

# Turn on WeMo when WAN is down, hoping for WAN2...
if ! ping >/dev/null -c 1 8.8.8.8; then
  curl -d '<?xml version="1.0" encoding="utf-8"?><s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><s:Body><u:SetBinaryState xmlns:u="urn:Belkin:service:basicevent:1"><BinaryState>1</BinaryState></u:SetBinaryState></s:Body></s:Envelope>' -H 'SOAPACTION: "urn:Belkin:service:basicevent:1#SetBinaryState"' -H 'Content-Type: text/xml; charset="utf-8"' -X POST http://wemo.local:49153/upnp/control/basicevent1
fi

After switching over to WAN 2, DNS will still be unavailable for a while – and yes, this method will try to turn on the WeMo over and over again. However, this shouldn’t really cause any problems if it’s already turned on. On the other hand, it might actually save the day, if the first packet is lost (WeMo is on Wi-Fi).

Next project
Next project will be to set up openHAB on a Raspberry Pi (for starters) and try to teach it when at least one person is at home. Then I can have back my IP telephony (requiring the modem on) when arriving at home and until going to bed.

Miele XGW 3000 gateway firmware 2.1.0

None of the bugs mentioned in my previous post about 2.0.9 is fixed in this release. No new features discovered. See release notes:

Release 2.1.0 – 26.10.2017

  • Support for SMA Sunny Homemanager 1.x adapted
  • Improvement of the Miele@mobile remote access connection
  • Datasynchronisation improvements
  • Stability improvements and bugfixing

Oh, and yes, they changed the versioning and renamed previously released versions. So when I’m now referring to 2.0.9, which I called 2.09 in my previous post, it isn’t by mistake.

Miele XGW 3000 gateway firmware 2.09

Seconds after publishing my previous post, I was a bit confused after reading the firmware release notes again, while checking that I got the version number right. It seems that Miele pulled back version 2.09, which of course it not possible to do after people, including myself, have upgraded. The release notes was also edited to reflect this, so the notes about 2.09 are gone. http://GATEWAY/Rest/Update/:

{"AvailableVersion":"2.0.7","UpdateInProgress":false,"Type":"XGW3000","ReleaseNotes":{"href":"http://www1.miele.com/media/ex/int/service/downloads/xgw3000_release_notes.html"},"NewerAvailable":false,"CurrentVersion":"2.0.9","Rev":"1.4.6"}

Direct link to release notes.

Now I wonder if the bugs mentioned in my previous post was also present in version 2.07. Version 2.09 was released on September 27th. I know because I upgraded over VPN from CPH Airport while waiting for my flight – just after receiving a notification from my app that it was available.

Miele XGW 3000 firmware 2.09 and new dishwasher with Wi-Fi

I got a new dishwasher (G 6895) with Wi-Fi integrated and at the same time received a new firmware update the my XGW 3000 gateway. So I now have three ZigBee appliances (XKM 3000 Z) and one Wi-Fi applicance (EK039W). I was curious to see how this integration with the existing system would work.

Homebus 1.0
Integrated nicely – the devices shows up with id=hdm%3ALAN%3AUID%230 instead of id=hdm%3AZigBee%3AZigBee address%23210. However, when programmed, the gateway will return HTTP code 500, i.e. internal server error:

500 No message

Homebus 2.0 (Rest API)
The device is not included in http://GATEWAY/Rest/Devices/. However, when accessing http://GATEWAY/Rest/Devices/UID/ directly, JSON is returned with relevant ident/state information. Including water/power consumption in ExtendedState:

000702040F0000000000000080000000020026030001041B
                              ^^^^^^^^ 0.2 kWh/3.8 L (0x26/10)

I have not figured out how to interact directly with the dishwasher, and would also prefer not to, since this would complicate matters. However, it’s a shame that the Rest implementation is partly broken. For now I’ve hardcoded the new UID in the app I’m working on (as well as my water/power monitoring system running on my Linux server), but I’ll probably end up having to combine the two protocols in order to do what I need. I’ll probably do this anyway, since I also haven’t figured out how to perform actions using the Rest protocol.

Another thing that is broken when accessing the gateway through the Rest protocol is the ElapsedTime value, which is always:

"ElapsedTime":[0,0]

Actually, I think the value is dependent on the current value when booting the gateway. At one point it was always [1,14] until the next reboot, and this was quite possibly the first value it saw when first booting up.

So the value cannot be used at all, e.g. for creating a progress bar. Too bad, since this is one of the advantages of using the Rest protocol over the old XML protocol.

Protocol values

Values found so far:

	static final int PHASE_PRE_WASH    =  2;
	static final int PHASE_MAIN_WASH   =  3;
	static final int PHASE_RINSES      =  4;
	static final int PHASE_FINAL_RINSE =  6;
	static final int PHASE_DRYING      =  7;

	static final int PROGRAM_ECONOMIC         = 28;
	static final int PROGRAM_QUICK_POWER_WASH = 38;

Mapping to Homebus 2.0 phases:

@Override
boolean setProgramPhase(int programPhase) {
	super.setProgramPhase(programPhase);

	switch (programPhase) {
		case 1792:
			// Purpose unknown, observed when programmed (without phase) and off.
			return setPhase(PHASE_UNKNOWN);
		case 1794:
			return setPhase(PHASE_PRE_WASH);
		case 1795:
			return setPhase(PHASE_MAIN_WASH);
		case 1796:
			return setPhase(PHASE_RINSES);
		case 1798:
			return setPhase(PHASE_FINAL_RINSE);
		case 1799:
			return setPhase(PHASE_DRYING);
		default:
			setPhase(PHASE_UNKNOWN);
			return false;
	}
}

Miele XGW 3000 firmware 2.06

Another firmware upgrade released yesterday. First thing I’ve noticed is that my tumble dryer (TKR 350 WP) and oven (H 5581 BP) now also include numeric/raw values in the Homebus XML, i.e.:

<key name="State" value="Off" type="state" raw="1"/>

Excellent! So my previous theory about this feature missing in the XKM 3000 Z module firmware version 1.02 was wrong – it was missing for certain device types in the gateway firmware (2.0.3).