Provide asyncio interface for homeassistant #59

rytilahti · 2022-07-25T22:56:59Z

It likely doesn't make sense to convert the whole library to use asyncio, but providing a separate homeassistant-friendly interface could be fairly straightforward to implement.

See home-assistant/developers.home-assistant#1412 and home-assistant/developers.home-assistant#1412 (comment) for best practices.

TechHummel · 2022-08-08T17:46:50Z

I just read https://developers.home-assistant.io/docs/network_discovery/ but it is not 100% clear to me what is needed here.
Do I understand correctly that the BLEDevice instances need to be passed from homeassistant to the eq3bt library?

TechHummel · 2022-08-15T16:14:51Z

@rytilahti Friendly ping!

rytilahti · 2022-08-15T16:50:02Z

Hi @TechHummel, sorry for the delay and thanks for the remainder!

Yeah, my understanding is also that the way to handle this would be by passing the BLEDevice created by homeassistant to this library, use it for the update cycle, toss it, and re-do the same procedure on the next cycle. As that doesn't map nicely to the existing API (performing connect on __enter__), it might make sense to create a very lightweight pure-bleak, asyncio interface separate from the existing ones that wraps the synchronous thermostat.

I created previously home-assistant/core#74864 to modernize the code base, but I'm currently occupied with other things so I have no time to look into that until very late autumn or so...

TechHummel · 2022-08-19T14:47:50Z

I am looking into this, but I am currently blocked hbldh/bleak#946. Will let you know about any progress.

antoniomolram · 2022-09-11T15:39:38Z

Hi! I am hitting the same issue here.... I will start checking out this. Any progress on this?

TechHummel · 2022-09-11T15:44:55Z

Sadly not! Not sure what's causing the issue!

antoniomolram · 2022-09-11T16:08:42Z

do you have your implementation working on the latest homeassistant? it seems that they want that you don't use the BleakClient() directly, instead the BLEDevice(). I am quite new with the Bleak library too.

It seems that doings so, home assistant can handle multiple BLE python interfaces (what makes sense).

I can use your package as an standalone application (without Home assistant), but once that I integrate it I can't use the functionality, and now I am trying to learn how to debug it :)

TechHummel · 2022-09-11T17:27:11Z

No, I don't have a working implementation! I am considering just using a MQTT bridge though!

antoniomolram · 2022-09-13T10:33:29Z

mm, could me maybe a easier solution for the moment, but is quite unfortunate that after a home assistant update the things stopped working (my wife is quite piss off :P)

kaymueller · 2022-09-13T14:39:35Z

mm, could me maybe a easier solution for the moment, but is quite unfortunate that after a home assistant update the things stopped working (my wife is quite piss off :P)

It's super easy to downgrade HA via the SSH Plugin, worked great for me at least. Tho it's of course only a temporary fix. Make a full backup and download it, before sending this command to your HA.

ha core update --version 2022.6.7

Hope that helps to make your wife a bit happier again. ;-)

cheers,
Kay

batja84 · 2022-09-18T15:13:28Z

ha core update --version 2022.6.7

Hope that helps to make your wife a bit happier again. ;-)

I don't have a wife, but thank you very much. I'm happy now. Outside is cold and I need my thermostats working.

antoniomolram · 2022-09-18T15:15:31Z

I made the downgrade but then other things stop work as I wanted (specially the new zigbee stuff). I will check more in detail all this but time is limited... (wife again) xD

HandHsince1920 · 2022-09-19T11:17:23Z

I am using Home Assistant now on a downgraded version (2022.6.7) and everything works as it should. However, with every every release I feel the dire consequence of not being able to access the new features and also a bad feeling creeping in due to staying on an outdated version. Our whole apartment works fine with these thermostats and they consume almost no batteries, we recharge once every 18 months. I am more than happy and would love to support any kind of kickstarter request if any of the previous developers would be happy to assigns some valuable time to it.

ndurchx · 2022-09-19T12:31:49Z

This is no solution, but as a temporary fix, I switched to an bt-mqtt bridge, to work with home-assistant. https://github.com/zewelor/bt-mqtt-gateway

wjarka · 2022-09-21T12:34:34Z

This is no solution, but as a temporary fix, I switched to an bt-mqtt bridge, to work with home-assistant. https://github.com/zewelor/bt-mqtt-gateway

Is there a way to run this on Hassio OS or did you use a separate Pi / PC for hosting this?

LXR85 · 2022-09-21T12:42:41Z

This is no solution, but as a temporary fix, I switched to an bt-mqtt bridge, to work with home-assistant. https://github.com/zewelor/bt-mqtt-gateway

Is there a way to run this on Hassio OS or did you use a separate Pi / PC for hosting this?

I have the same issue and the same question.

ndurchx · 2022-09-21T12:54:31Z

This is no solution, but as a temporary fix, I switched to an bt-mqtt bridge, to work with home-assistant. https://github.com/zewelor/bt-mqtt-gateway

Is there a way to run this on Hassio OS or did you use a separate Pi / PC for hosting this?

I have had the same issue and the same question.

It is hosted separately. For me it was just another VM, because my HA is also virtualized. It is a bit of a downside, but even if the HA integration is fixed, I will stay at the mqtt bridge. It is much faster, and pops out more usable data, like valve state.

Edit: seems like there is a HA addon version: https://github.com/FaserF/hassio-addons/tree/master/bt-mqtt-gateway

HandHsince1920 · 2022-09-21T21:00:43Z

This is no solution, but as a temporary fix, I switched to an bt-mqtt bridge, to work with home-assistant. https://github.com/zewelor/bt-mqtt-gateway

Is there a way to run this on Hassio OS or did you use a separate Pi / PC for hosting this?

I have had the same issue and the same question.

It is hosted separately. For me it was just another VM, because my HA is also virtualized. It is a bit of a downside, but even if the HA integration is fixed, I will stay at the mqtt bridge. It is much faster, and pops out more usable data, like valve state.

Edit: seems like there is a HA addon version: https://github.com/FaserF/hassio-addons/tree/master/bt-mqtt-gateway

What I really liked about the solution via an integration was that it does not create more dependencies in form of further VM's and addons. It came out of the box and worked. 🥇

wjarka · 2022-09-22T07:37:42Z

Btw, I was able to get the valve state from the original integration. Can look it up later if you are interested

batja84 · 2022-09-22T15:52:57Z

like valve state

I can get valve level via template.

Everything from here can be gotten via templates.

batja84 · 2022-10-02T11:10:58Z

Hello guys! It there any good changes in 2022.9.7? My HACS integration is dead now "You need HA version 2022.8.0 or newer to use this integration". And I understand further will come more and more problems with obsolete HA Core. It's annoying.

HandHsince1920 · 2022-10-05T13:56:07Z

I am using Home Assistant now on a downgraded version (2022.6.7) and everything works as it should. However, with every every release I feel the dire consequence of not being able to access the new features and also a bad feeling creeping in due to staying on an outdated version. Our whole apartment works fine with these thermostats and they consume almost no batteries, we recharge once every 18 months. I am more than happy and would love to support any kind of kickstarter request if any of the previous developers would be happy to assigns some valuable time to it.

Just realised that one can actually support developers directly via github!

r-jordan · 2022-10-05T15:05:12Z

Hi all,

Meanwhile the issue is not resolved I found a temporal solution with a bluetooth to mqtt gateway. I hope it can help someone. I had mqtt configured previously. But if you don't have mqtt configured you should install the Mosquitto add-on previously. Then do the following:

Set add-on repository on Home Assistant OS to https://github.com/faserf/hassio-addons
Add user bt-mqtt-gateway in Mosquitto add-on or in Home Assistant with a password
Install bt-mqtt-gateway and put the following into the file /share/bt-mqtt-gateway.yaml:

mqtt:
  host: 192.168.1.2
  port: 1883
  username: bt-mqtt-gateway
  password: ***
  client_id: bt-mqtt-gateway
  availability_topic: thermostat/lwt_topic

manager:
  sensor_config:
    topic: homeassistant
    retain: true
  topic_subscription:
    update_all:
      topic: homeassistant/status
      payload: online
  command_timeout: 35
  workers:
    thermostat:
      args:
        devices:
          device1: 00:1A:22:xx:xx:xx
          device2: 00:1A:22:xx:xx:xx
        topic_prefix: thermostat
      topic_subscription: thermostat/+/+/set
      update_interval: 90

Configure the devices with their mac addresses and start the add-on

Thanks to @FaserF for the add-on development and to @zewelor for the bluetooth mqtt gateway development.

wjarka · 2022-10-06T15:43:01Z

I've been using the MQTT gateway and it works great. However, since HomeAssistant introduced Bluetooth Proxies, I would love to have EQ3 over BT Proxy. May want to try to debug myself.

dbuezas · 2022-10-17T16:47:03Z

Here you can find the HACS custom component.

https://github.com/dbuezas/eq3btsmart

I added some extra stuff like:

Bluetooth autodiscovery
Add via config flow
~~BTProxies support (just using the correct APIs)~~ (probably only in firmware < 1.20)
retries
ported everything to async
fixed setting operation mode
allowed to turn off by setting temp to 4.5°
fetching schedules in HA
(future) setting schedules via HA

My Apologies to @rytilahti for going this way instead of making a PR, I left my reasons in the readme (mostly that the changes I made were too many, and they break the CLI and other backends)

r-jordan · 2022-10-19T10:08:17Z

Hi @dbuezas ,

Thank you!! I have got 3 of these valves and it's working for two of them but it's not connecting to one of the devices and the following errors are showing:

Logger: homeassistant.helpers.entity
Source: custom_components/dbuezas_eq3btsmart/python_eq3bt/eq3bt/bleakconnection.py:97
Integration: dbuezas EQ3 Bluetooth Smart Thermostats
First occurred: 18 d’octubre de 2022, a les 23:14:42 (114 occurrences)
Last logged: 01:30:03

Update for climate.eq3_salon2 fails
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 515, in async_update_ha_state
    await self.async_device_update()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 718, in async_device_update
    await task
  File "/config/custom_components/dbuezas_eq3btsmart/climate.py", line 388, in async_update
    await self._async_thermostat_update()
  File "/config/custom_components/dbuezas_eq3btsmart/climate.py", line 381, in _async_thermostat_update
    await self._thermostat.async_update()
  File "/config/custom_components/dbuezas_eq3btsmart/python_eq3bt/eq3bt/eq3btsmart.py", line 239, in async_update
    await self._conn.async_make_request(PROP_WRITE_HANDLE, value)
  File "/config/custom_components/dbuezas_eq3btsmart/python_eq3bt/eq3bt/bleakconnection.py", line 111, in async_make_request
    raise ex
  File "/config/custom_components/dbuezas_eq3btsmart/python_eq3bt/eq3bt/bleakconnection.py", line 97, in async_make_request
    await conn.start_notify(
  File "/usr/local/lib/python3.10/site-packages/bleak/__init__.py", line 546, in start_notify
    raise BleakError(f"Characteristic {char_specifier} not found!")
bleak.exc.BleakError: Characteristic 1056 not found!

I know the device is not the cause because it works ok with the bt-mqtt-gateway.

dbuezas · 2022-10-19T10:31:31Z

@r-jordan please raise an issue in the repo to avoid polluting this issue further :)

andrei-micu-ro · 2022-10-20T12:38:52Z

Just as a conclusion, the integration works very well, there are a few improvement areas but dbuezas is handling them like a pro !!
Managed to integrate one thermostat and it works better/faster than the original integration.
The only remaining problem is that it won't connect through ESPHome's bluetooth proxy due the thermostat (ver > 1.2) asking for PIN Pairing (currently not supported by ESP)

rytilahti · 2022-10-21T02:48:17Z

My Apologies to @rytilahti for going this way instead of making a PR, I left my reasons in the readme (mostly that the changes I made were too many, and they break the CLI and other backends)

No worries, I have been maintaining this lib just in case someone is using it as I don't. I'm just glad that there's now a working solution for people who are using these devices in their homes! :-)

In case you want to make a push to make it into the homeassistant releases, it probably makes sense to create an async fork that removes all the cruft (we did similar for the tplink integration, pyhs100 -> python-kasa) and adjust the integration accordingly. I'm happy to help with the process if needed.

edit: just for the ref, although I'm not sure how useful it is, I did some ground work to modernize the integration at home-assistant/core#74864

batja84 · 2022-10-21T21:39:09Z

I have strange behavior one of my two thermostats. One of them is paired and working without problems with @dbuezas add-on, but another won't be paired via console. There is no message to input pair PIN.

And this one working without problems.

Does anybody can help me to solve this problem?

andrei-micu-ro · 2022-10-21T21:45:57Z

For me usually worked some restarts:

bluetooth service
thermostat (batteries out)
It may be because it's stuck in a waiting loop. BlueZ did that when I used RPi.

I'm on a linux server now with a BT Dongle - way more stable, though it's using the same stack (BlueZ based).

Also as a quick hint, if it's just not asking for the Pin, try long-pressing the wheel to trigger the pairing mechanism in the Thermostat - maybe due to battery savings it doesn't trigger the pin when RPi asks for it.

batja84 · 2022-10-21T21:50:32Z

For me usually worked some restarts:

bluetooth service

thermostat (batteries out)
It may be because it's stuck in a waiting loop. BlueZ did that when I used RPi.

I'm on a linux server now with a BT Dongle - way more stable, though it's using the same stack (BlueZ based).

Also as a quick hint, if it's just not asking for the Pin, try long-pressing the wheel to trigger the pairing mechanism in the Thermostat - maybe due to battery savings it doesn't trigger the pin when RPi asks for it.

Thanks for fast reply.
I have tried to take batteries out (for a minute o so) few times.
I have pushed button to trigger pair-mode on the thermostat and tried to pair in console just after that.
RPi restart does nothing too. I have external BT dongle connected to RPi.
No luck in all cases. :(

andrei-micu-ro · 2022-10-21T21:58:50Z

Sorry to hear that, I was expecting you to have tried all the normal stuff...
Recently I fix pairing issues with 2 simple steps under bluetoothctl:

remove 00:00:00:...
pair 00:00:00:...
If in a few seconds after that it's not asking for Pin I press the wheel to trigger and usually works.

jhass · 2022-10-22T04:55:50Z

Keep trying to pair, it can take a long time in my experience. What also seemed to help sometimes was toggling the BLE setting in the long press menu settings a couple of times.

batja84 · 2022-10-22T15:18:17Z

@jhass, @andrei-micu-ro
I was ready to give up, but want to try one more time. I've disconnected BT dongle from RPi, than connected again. Pushed pair button and entered in console "pair mac" command. And it worked.

But now I'm not sure, that earlier it wasn't there, because by new connection I've got 2 full screen of messages and haven't read them all.
Thank you guys for your help! And my personal thanks to @dbuezas for his great work!

P.S. Just for information. I have bought 2 these used thermostats about 2 years ago for 15€. Now in Germany price is starting from 60€ for one unit. It's insane.

dbuezas · 2022-10-22T16:06:54Z

I bought 7 for 20€ each last year, so I'll pass the thanks to @rytilahti for this repo :)

HandHsince1920 · 2022-10-23T20:04:17Z

that is somehow funny, I never needed to pair anything with codes with the old integration even if I needed the pin on the app. To me the pin requirement appeared rather like a superficial layer of "security" that was not really deep rooted.

HandHsince1920 · 2022-10-23T20:09:24Z

Here you can find the HACS custom component.

https://github.com/dbuezas/eq3btsmart

I added some extra stuff like:
* Bluetooth autodiscovery

* Add via config flow

* ~BTProxies support (just using the correct APIs)~ (probably only in firmware < 1.20)

* retries

* ported everything to async

* fixed setting operation mode

* allowed to turn off by setting temp to 4.5°

* fetching schedules in HA

* (future) setting schedules via HA
My Apologies to @rytilahti for going this way instead of making a PR, I left my reasons in the readme (mostly that the changes I made were too many, and they break the CLI and other backends)

that looks absolutely awesome. are you planning to get it into home assistant's normal releases? that would be great and due to your much improved autodiscovery it will certainly be of more use for less experiences users.

dbuezas · 2022-10-25T21:03:17Z

Thanks!
Yes, I'd wish to get it into home assistant core, but I'll wait until it ESPHome implements pairing. I will rhen probably ask @rytilahti for help reviewing my code (I'm not knowledgeable in Python)

HandHsince1920 · 2022-10-27T15:22:24Z

Thanks! Yes, I'd wish to get it into home assistant core, but I'll wait until it ESPHome implements pairing. I will rhen probably ask @rytilahti for help reviewing my code (I'm not knowledgeable in Python)

That would be really appreciated and also me the easiest option for many other users. Do you have a rough idea when the ESPHome pairing issue might be solved?

LXR85 · 2022-11-07T20:36:16Z

Thank you @dbuezas for providing us with a workaround/solution, and thank you @andrei-micu-ro for the tip with removing the thermostat before pairing it! I have got now all 4 of my thermostats working:

One minor issue, however: When I change the temperature through Siri voice command (via HomeKit Bridge), the thermostat switches from OFF to Auto Mode and not to Manual Mode. Is this something you can fix, @dbuezas?

dbuezas · 2022-11-07T21:16:30Z

Yey!
But that issue with siri is very weird. Can you grab from the logs all commands set to the thermostat?
If there are a sequence of "set temp", then "set auto", then I could probably add an exception

LXR85 · 2022-11-07T22:01:05Z

Yes, of course. I changed the thermostat (“Küche”) to 15 °C, then turned it off, and then back to 12 °C, all via Siri. This is the corresponding list of entries from the Logbook:

andrei-micu-ro · 2022-11-08T09:43:09Z

Strange with Siri... just tested it on my env and it works as expected (meaning that it keeps the "Manual" mode and just sets the temperature correctly).
I guess in your scenario in the Home app (iOS) it does show the different modes (Manual, Auto and Off), right ?
If so then it should keep the current mode and only set the temperature.

LXR85 · 2022-11-08T12:49:59Z

Yes, that is indeed how it looks in the Home app:

dbuezas · 2022-11-20T22:28:54Z

The custom EQ3 integration https://github.com/dbuezas/eq3btsmart is now a proper device with the entities you missed, and then some more.

HandHsince1920 · 2022-11-24T09:31:20Z

The custom EQ3 integration https://github.com/dbuezas/eq3btsmart is now a proper device with the entities you missed, and then some more.

Wow, that looks awesome! Thanks for the great work. Do you know when it will be available via home assistant's own integrations section?

dbuezas · 2022-11-24T17:53:06Z

It may take a while, but eventually I'll do it. I want to add all features and polish it well before I attempt to publish the library and open a PR to the core repo of home assistant.

dbuezas · 2022-11-25T19:13:24Z

I think I'm finished with the features

felixfischer · 2022-11-25T21:42:58Z

@dbuezas David, thank you so much for putting in all the effort to keep this piece of hardware supported by Home Assistant! It is very much appreciated! <3

wjarka · 2022-11-28T07:29:49Z

Looks awesome! @dbuezas any news on the proxies?

dbuezas · 2022-11-28T07:46:58Z

Unfortunately no news on the BTProxy.
I don't use them so somebody else should open an issue in EspHome. The issue is that EspHome cannot do pairing, and HA doesn't support pairing flows yet.
Note that even in older firmware where there is no real pin, pairing is still necessary

rytilahti added the help wanted label Jul 25, 2022

This was referenced Aug 15, 2022

Create event loop only when necessary #58

Merged

thermostat unavailable after update home-assistant/core#76040

Closed

kaymueller mentioned this issue Aug 26, 2022

I need some help understanding the ongoing issue with the Hassio/HA integration #60

Closed

Wave89 mentioned this issue Sep 2, 2022

[homeassistant.components.climate] eq3btsmart: Error on device update! home-assistant/core#77110

Closed

rytilahti mentioned this issue Oct 8, 2022

Eq3btsmart/bleak connection fix home-assistant/core#79888

Closed

23 tasks

rytilahti mentioned this issue Oct 28, 2022

eq3btsmart Detected integration that attempted to call BleakClient with an address instead of a BLEDevice. home-assistant/core#79221

Closed

rytilahti mentioned this issue Jun 21, 2023

Remove eq3btsmart integration home-assistant/core#94698

Merged

20 tasks

Provide asyncio interface for homeassistant #59

Provide asyncio interface for homeassistant #59

Comments

rytilahti commented Jul 25, 2022

TechHummel commented Aug 8, 2022

TechHummel commented Aug 15, 2022

rytilahti commented Aug 15, 2022

TechHummel commented Aug 19, 2022

antoniomolram commented Sep 11, 2022

TechHummel commented Sep 11, 2022

antoniomolram commented Sep 11, 2022

TechHummel commented Sep 11, 2022

antoniomolram commented Sep 13, 2022

kaymueller commented Sep 13, 2022 • edited Loading

batja84 commented Sep 18, 2022 • edited Loading

antoniomolram commented Sep 18, 2022

HandHsince1920 commented Sep 19, 2022

ndurchx commented Sep 19, 2022

wjarka commented Sep 21, 2022

LXR85 commented Sep 21, 2022 • edited Loading

ndurchx commented Sep 21, 2022 • edited Loading

HandHsince1920 commented Sep 21, 2022

wjarka commented Sep 22, 2022

batja84 commented Sep 22, 2022 • edited Loading

batja84 commented Oct 2, 2022 • edited Loading

HandHsince1920 commented Oct 5, 2022

r-jordan commented Oct 5, 2022

wjarka commented Oct 6, 2022

dbuezas commented Oct 17, 2022 • edited Loading

r-jordan commented Oct 19, 2022

dbuezas commented Oct 19, 2022

andrei-micu-ro commented Oct 20, 2022

rytilahti commented Oct 21, 2022 • edited Loading

batja84 commented Oct 21, 2022 • edited Loading

andrei-micu-ro commented Oct 21, 2022

batja84 commented Oct 21, 2022 • edited Loading

andrei-micu-ro commented Oct 21, 2022

jhass commented Oct 22, 2022

batja84 commented Oct 22, 2022 • edited Loading

dbuezas commented Oct 22, 2022

HandHsince1920 commented Oct 23, 2022

HandHsince1920 commented Oct 23, 2022

dbuezas commented Oct 25, 2022

HandHsince1920 commented Oct 27, 2022

LXR85 commented Nov 7, 2022

dbuezas commented Nov 7, 2022

LXR85 commented Nov 7, 2022 • edited Loading

andrei-micu-ro commented Nov 8, 2022

LXR85 commented Nov 8, 2022

dbuezas commented Nov 20, 2022

HandHsince1920 commented Nov 24, 2022

dbuezas commented Nov 24, 2022

dbuezas commented Nov 25, 2022 • edited Loading

felixfischer commented Nov 25, 2022

wjarka commented Nov 28, 2022

dbuezas commented Nov 28, 2022

kaymueller commented Sep 13, 2022 •

edited

Loading

batja84 commented Sep 18, 2022 •

edited

Loading

LXR85 commented Sep 21, 2022 •

edited

Loading

ndurchx commented Sep 21, 2022 •

edited

Loading

batja84 commented Sep 22, 2022 •

edited

Loading

batja84 commented Oct 2, 2022 •

edited

Loading

dbuezas commented Oct 17, 2022 •

edited

Loading

rytilahti commented Oct 21, 2022 •

edited

Loading

batja84 commented Oct 21, 2022 •

edited

Loading

batja84 commented Oct 21, 2022 •

edited

Loading

batja84 commented Oct 22, 2022 •

edited

Loading

LXR85 commented Nov 7, 2022 •

edited

Loading

dbuezas commented Nov 25, 2022 •

edited

Loading