Periodic information update

Scheduled tasks update information periodically

Panopticon does not have a “direct line” to all of your sites open all the time. Something like that would be incredibly taxing for the server running Panopticon, and the server(s) where your sites are on. Instead, Panopticon periodically pulls information from your sites. It does that by using two scheduled tasks, refreshsiteinfo and refreshinstalledextensions. Both tasks are scheduled to run at most every 15'.

The refreshsiteinfo task

This task retrieves the basic information for each of your sites: the connector version and API level, the CMS type and version, update availability for the CMS, PHP version information, template overrides to check, integration information with Akeeba Backup and Admin Tools and so on.

This information is queried in concurrent batches of 10 sites at a time, meaning that Panopticon will ask 10 sites to report this information at the same time. Once all ten sites respond, this information is saved to Panopticon's database. This task considers itself finished and reschedules itself after it has gone through al configured sites.

Please remember that Panopticon does not determine whether there is an update for Joomla!. This is something that Joomla! itself determines and conveys that information to Panopticon. This is deliberate feature. Joomla! can be configured to use a number of update sources, two of which are normally used on production sites: Current and Joomla! Next. The former will stick with the same major version of Joomla!, the latter will try to provide an update to the next major version when it's available. For example, in October 2023 Joomla! released two versions: 4.4.0 and 5.0.0. A site with Joomla! 4.3 would show Joomla! 4.4.0 as the available update when using the Current update source, whereas the exact same site would show Joomla! 5.0.0 as the available update source when using the Joomla! Next update source.

Further to that, each Joomla! version may have different minimum requirements for the PHP and database server version. For example, Joomla! 4 requires PHP 7.2 or later, and MySQL 5.6 or later, whereas Joomla! 5 requires PHP 8.1 or later, and MySQL 8.0 or later. A site on Joomla! 4.3 set up to use Joomla! Next would not offer Joomla! 5 as an update if the server it's running on uses PHP 8.0 or MySQL 5.7.

Moreover, the Joomla! Update component can be configured with a minimum stability. On production sites the recommended value is “stable”. In some cases, e.g. testing or development sites, this might be a less stable choices e.g. “beta”. Therefore, a site in early November 2023 might have shown an update to Joomla! 5.0.1-Beta1 instead of saying it's up to date with Joomla! 5.0.0, but this does not mean that every Joomla! 5 site you had at the time should be updated to the beta version!

That is to say, the concept of “latest version of Joomla!” does not exist per se. That's why Panopticon needs to ask Joomla! itself to tell it what is the latest version.

Lastly, please do remember that Joomla! itself caches the update information for a length of time which might be anywhere from one hour to 24 hours. Depending on when was the last time Joomla! checked for updates to itself and how long the caching time for the update information is set on that site, it is possible that one site "sees" an update to Joomla! itself, but another site does not "see" the update just yet.

Do keep in mind that when you click the user interface button to reload the site information you are telling Panopticon to ask Joomla! to override the updates cache, forcing it to recheck whether an update to itself is available. This is very slow and hits the Joomla! update servers hard. While it will show you updates faster you should not make a habit of using that button regularly. It not only slows down your site, it also causes operational problems to Joomla! itself if everyone overuses this feature. It's best for you to just wait for Joomla! to detect the updates when its updates cache expires, and Panopticon to eventually show them to you.

The refreshinstalledextensions task

This task retrieves the list of installed extensions of each site, and the availability of updates for each extension.

Panopticon does not determine by itself if an extension has an update. This is determined by Joomla! itself and conveyed to Panopticon. This is a deliberate choice. Remember that whether an update for the same extension and extension version is available on two different sites may differ depending on the Joomla! version, the PHP version, the database type, and the database version of each site. This may be the source of some confusion.

Firstly, each and every one of your sites must have the extension installed correctly, along with its update sites. If the update site for an extension is corrupt, missing, or disabled on one of your sites then that specific site will not show an update for a specific extension and extension version even though other seemingly identical sites do.

Secondly, Joomla! needs to connect to the extension's update site over the Internet to determine if an update is available. It is possible that one or more of your sites will be behind a firewall / proxy which will prevent that. In this case, no updates will be found for that extension.

Thirdly, please do remember that Joomla! itself caches the update information for each extension for a length of time which might be anywhere from one hour to 24 hours. Depending on when was the last time Joomla! checked for updates and how long the caching time for the update information is set on that site, it is possible that one site "sees" an update to an extension, but another site does not "see" the update just yet.

Do keep in mind that when you click the user interface button to reload the extension update information you are telling Panopticon to ask Joomla! to override the updates cache, forcing it to recheck whether an update is available for each and every extension Joomla! has an update site for. This is very slow and hits the update servers of third party extensions hard. While it will show you updates faster you should not make a habit of using that button regularly. It not only slows down your site, it also causes operational problems on third party extension developers' sites who have to cope with a higher load of update checks. It's best for you to just wait for Joomla! to detect the updates when its updates cache expires, and Panopticon to eventually show them to you.

Troubleshooting

A lot of people are impatient and submit discussion threads or issues claiming that Panopticon does not update the information “immediately” or for “some sites”. In the vast majority of cases it's a misunderstanding of how things work.

As noted above, Panopticon will only update this information at most once every 15 minutes. Moreover, the internal updates caching in Joomla! may delay updates from showing up from anywhere between 1 and 24 hours (the default is 6 hours). Finally, subtle differences in the server environment and site settings may mean that sites you perceive as being mostly the same detect vastly different updates.

Before assuming something is broken STOP AND WAIT for a day. If the updates now “magically” show you were just being impatient. Read the information on this page again until you understand why waiting makes sense. No disrespect here. It's counter-intuitive, we'll grant you that. Waiting really is the best policy as it will ensure that you will get informed about updates without unnecessarily taxing your servers and third party update servers. This is, after all, how all update systems work, be it Joomla!, WordPress, Microsoft Windows, macOS, or your favourite Linux distribution. Updates are never shown to everyone all at once; that would literally break the Internet.

If you have waited for at least a full day – 24 hours – and nothing happened, then and only then should you proceed to troubleshooting.

Viewing the scheduled tasks

Panopticon lets you see its scheduled tasks. Go to Administration, Tasks. In the “– Site –” drop-down select “System Task”.

You can now see the refreshinstalledextensions and refreshsiteinfo tasks. If one of the tasks is missing look towards the bottom of this page.

These tasks must be both Enabled (checkmark) and have a Status of OK (checkmark in a green circle).

The “Last Run / Next Run” column shows you when each task was last executed and when it is scheduled to execute next. If you see that the next execution is a time in the past, read the next section to understand what is going on.

Understanding delays in execution

Please read the CRON Jobs page, under “Busy installations will need more than one CRON job”.

As explained there, the CRON job you set up when installing Panopticon is a task runner. It runs tasks scheduled by Panopticon.

When you are installing updates, taking backups, or running the PHP File Change Scanner there is no magic involved, just some good old scheduled tasks. These tasks have a slightly higher priority than System Tasks, and take a heck of a lot more time to complete. While these tasks are executing, there is no time for Panopticon to run its System Tasks to pull information about your sites, their extensions, and their update information. This means that your site information will appear out-of-date and you will see that the System Tasks have a next execution time in the past.

As of Panopticon 1.0.5, when this lag becomes longer than 2 minutes on average (note that some tasks may have fallen behind an hour or more before the average hits two minutes!) it will show you a warning that you need more CRON jobs. Older versions of Panopticon do not do that, therefore you will have to check yourself if System Tasks are falling behind.

Either way, if you detect this kind of situation, you will need to set up more CRON jobs for Panopticon. These are identical to the CRON job you created when you first installed Panopticon. It runs every minute, and executes the exact same command line.

Adding more CRON jobs adds more capacity to Panopticon to execute tasks in parallel. The more CRON jobs you have, the more things can happen at the same time. The more things can happen at the same time, the more likely it is that System Tasks will execute on time.

There is no hard rule on how many CRON jobs you need to install. Typically, the more sites you have, and the more work you ask Panopticon to do (especially when it comes to long-running operations, like taking backups) the more CRON jobs you will need. Try to not overlap the backups / scanning of too many sites, and try to add a bit of a time buffer between executions (don't schedule everything back-to-back). Try to spread out the load throughout the day as much as possible.

What happens when these tasks disappear or get disabled

Normally, System Tasks will not disappear and will not get disabled. If you messed up your database, this might happen.

There's an easy way to fix that.

Log into Panopticon. Edit the URL in your browser to point to index.php?view=selfupdate&task=postinstall and press the ENTER key on your keyboard. For example, if you installed Panopticon on https://monitor.example.com log into it and then visit https://monitor.example.com/index.php?view=selfupdate&task=postinstall in your browser. This will check if the default system tasks are installed and enabled. If not, it will reinstall them and reactivate them.

If you prefer using the command line, use the Panopticon CLI command config:tasks:install which does the same thing.