Plugin update

[adorobis]

@tprelog as suggested let's have a separate discussion on the plugin update.
So you are saying that running the update plugin:

even without checking the "Update jail as well" option will update something more than just the plugin overlay files?

[tprelog]

That's correct. Though, I'm not exactly sure what "Update jail as well" does in addition vs leaving unchecked

Here's my understanding about the Plugin UPDATE process

1. It takes a snapshot
2. Clones the plugin repository
3. Applies the overlay folder to the plugin root fs
4. Run the pre_update.sh script, if it exists
5. Remove all user land installed packages
6. Install up to date versions of all packages listed in the plugin manifest
7. Run the post_update.sh script, if it exists

I know I've changed a few things since your version of the plugin but I think the overlay is still compatible, with a few changes. There's nothing that should affect Home Assistant (unless your plugin is over 2.5 years old) but for example the menu script location has changed. We would need to fix-up a few things so the new menu started when you enter the jail console

If you run the Plugin UPDATE I try to do all that in the pre_update.sh script. Actually, I think from your version it will try to just reinstall Home Assistant automatically. I know recreating the virtualenv was a hard requirement when version 5 came out. With version 5 I had to cut ties with all the previous crap I was trying to do. I really wanted everyone to do a clean install at that point but troubleshooting a failed Plugin UPDATE is still feasible, with a little time and patience of course.

The risk when Home Assistant, or the jail itself seems to break is usually when the system packages get updated.

When there is an issue with Home Assistant after such an update, the solution is usually to just recreate the virtualenv.

service homeassistant reinstall homeassistant

Do you want to try a Plugin UPDATE. Since you still have the 12.1-RELEASE, I think I would try the "Update Jail as well" to see if you get to the current 12.2-RELEASE-p10. Or we might need to try some iocage commands to achieve that RELEASE upgrade

We can also try to just do the overlay manually, if you want to try that.

Or if you're using an external dataset for your config the quickest way forward is honestly to just install a new plugin and mount your existing config to it...

I will try to help with any path forward you choose

[adorobis]

So I've found a bit of time to go through it. Installed HA plugin from scratch and mounted the configuration directory. And all works right away. The only thing was that it was a bit slow start after mounting the config directory.

Now I have plugin version v7-2022.2.3 and jail release 12.2-RELEASE-p11. Btw when creating empty jail a release 12.3-RELEASE is used - is it something you control in the plugin?
The only manual thing I have to do is to copy some scripts over to the new jail (used in command_line sensors). Have to move it to the config directory so no manual intervention would be required.

[adorobis]

Ah, and btw, again the pymysql module was not installed automatically and I've had to add it manually to the venv.

[tprelog]

again the pymysql module was not installed automatically and I've had to add it manually to the venv.

Yea, in the venv, if it's something extra you installed manually to begin with, it will always need re-installed manually as well. When it comes to reinstalling the virtualenv within an existing jail, there is a pip command (I'd have to look it up) that can be used to generate a list of everything installed in the venv. This list can then be passed as requirements to the install command.

service homeassistant install homeassistant -r /path/to/file

The only thing was that it was a bit slow start after mounting the config directory.

This is normal... What's happening behind the scenes is that Home Assistant is reinstalling all the integrations you have. This can take a while since lots of things may need to be compiled (aka building wheels). There's actually a trick you can use to watch these start-ups.

As you may know, the log level in Home Assistant is set to warning unless otherwise configured in configuration.yaml. For this reason nothing shows up until something is wrong. Basically, no news is good news. But you want to know the news regardless... So for this you need to set Home Assistant's logger integration to info

logger:
  default: info

You can also use this next command to view and "follow" the log. This can be useful to have open in a second window during an upgrade or when reinstall the virtualenv, like when python needs updated.

service homeassistant logs -f

I made a quick video (no audio) showing how this works using a service restart as an example.

Here I am logged into the jail twice. On the left I'm using the menu to restart the service. On the right, I have exited the menu and entered the command to view and "follow" the logfile. During the first restart I am just using the default configuration. On the right side of the screen you no additional information is printed.

Then I quickly edit the homeassistant/configuration.yaml to set the logger level to info and re-run the command to "follow" the logs.

When I restart Home Assistant a second time, you can see all the additional information about what's starting.

logger-info.mp4

BTW there's a daemon log: /var/log/homeassistant_daemon.log -- Basically it the same as home-assistant.log

service homeassistant logs -f

Is a shortcut for

tail -F /var/log/homeassistant_daemon.log

I threw a few others options in as well

---

Btw when creating empty jail a release 12.3-RELEASE is used - is it something you control in the plugin?

Sort of... maybe... but definitely not from the usual ix-system plugin repo. I'm not well versed in FreeBSD to understand any implications of this, but since the TrueNAS host is still running 12.2-RELEASE it is only that userland application that will be from 12.3-RELEASE. I can not say what effect, if any, this will have when Home Assistant is building the wheels.

I was running some tests on different jail types, trying to figure out options for user who do not upgrade to TrueNAS CORE 13 immediately upon release. This is when I noticed Home Assistant still appears to build wheels for the 12.2-RELEASE, even though the jail was created as 12.3 -- ix-plugin-hub/iocage-plugin-index#284 (comment)

At this time ix-systems has no plans to bump plugins to the 12.3-RELEASE. The next release will be for 13.0, but I have a separate plugin repo containing only the plugin manifest I maintain. I could try to publish a 12.3-RELEASE branch there and then try switching the repository on a plugin.

iocage set  plugin_repository=https://github.com/tprelog/truenas-plugin-index.git

From there try to somehow upgrade to the desired release. I'm not sure if, or how that might be possible. Definitely need to do some testing to figure out. I think we need to use iocage upgrade to change the release. I think, it should be the same as upgrading a regular jail

[tprelog]

What the heck, I just went and recorded in real time - reinstalling the Home Assistant virtualenv with HA open in the browser and a window "following" the log file... It's long.... It's boring but I think it gives a good idea of what to expect. With the logger still set to info, you see the file come alive as soon as the installation finishes. Around 3 min 10 sec we see home assistant is now listening on port 8123. About 10 seconds later you may notice the toast banner in home assistant changes from connection lost to Home Assistant is starting - this is followed a pause in activity... I know something's happening because I hear the fans on my computer kick up. Finally the logs start to roll again... It is not until around 6 min 50 secs that we finally see Home Assistant has started and it is still continuing to install additional modules for things as they are being discovered on my network.

logger-info-looong.mp4

[adorobis]

Wow that's a comprehensive response! Very useful for future troubleshooting. As the fresh install went ok, I'm not going to dig deeper now.
My only comment would be regarding the pymysql package: I have not installed it manually originally as far as I remember, I only need or for recorder supporting data to mariadb, which is a standard feature. Shouldn't HA install this package automatically?
Regarding TrueNAS 13, let me know if you'll need any testing to make your plugin work on it, I'll try to set up a test system.