A Few Thoughts on a Large Site

[lgrrealag]

Came across your plugin and was optimistic that perhaps it could solve some of the problems with WP search. Running a few tests on one of our test sites has provided me with some information that I would pass on for you to consider.

1. Had problems activating the plugin. PHP log showed it was running out of ram. Increasing the ram to 1gb of ram for WP solved that issue. Most likely related to the fact that it is a large website. We have just over 10k posts and well creating the search xml file with the base amount of 256mb of ram was not enough.

2. The plugin was created to offer search to WP sites that were turned static, so the search uses the xml file of the posts to search. Great idea, except with large sites that file is HUGE! Our search xml file was 170mb uncompressed. Since the search then loads that file on every page load, even delivering it gzipped it was roughly 15mb extra on every page load. That is not really practical on a site. If you only have a few hundred posts sure but the overhead of having to load the search xml file is to much. Perhaps if it was only loaded when the search box was in focus or some other way it would cut down on that extra download, but still really only good for maybe 1k posts.

3, Perhaps because the search xml file is so large in our test case but could not get the script to even return any results.

4. As mentioned in another post having the author data included in the xml file export is probably not needed. Considering it contains the login and the email address for the authors you probably don't want it in there. No reason to make that data public.

Anyways, interesting idea but not really practical for large sites.

Good luck.

[dbrw]

I am interested in this approach of searching.

If you implement this functionality, how would you handle this kind of problem, @lgrrealag ?

My only viable solution is to use more compact document format like JSON and not including all post content.

[dospuntocero]

i think json will help but the only way this will work for a medium to large site is adding the ability of choosing what to expose, maybe just titles and short content

[designeryusuf]

Hello. Was this issue solved already? I am curious.

[Erudition]

Looks like this can be solved with xpath filters as the XML is being saved?
https://www.w3schools.com/xml/xpath_syntax.asp