[Feature] Lazy loading #421
Comments
|
A (relatively) easy way to do this, compared to This way, we can Second, probably easier, option - the vast majority of the plugins we support are lua-based plugins. The majority of the loading time there is spent on the Either way, I will be looking into how lazy-loading in neovim has been traditionally done, since I don't have much experience with it, and will try to come up with some way it could work. |
|
Also, another feature |
|
Some references:
|
Great ! Thanks and good luck :) |
|
Yesterday I found out about nixvim. I'm already using home-manager. I'm not sure I would migrate though. I do some customization for neovim loading to greatly improve startup time and I'm not sure I can insert nixvim in that process. But the project looks interesting. Still I wanted to comment about lazy loading. What I think is most important for startup time are:
So even not doing any lazy-loading I have a decent startup time. Without all these optimizations |
|
Thank you so much @stasjok for this great insight of your config ! |
|
I'm not using nixvim either because I need the lazy-loading but I'd like to chip in an idea. If you're willing to outsource some of the code, I'd suggest using lazy.nvim seeing as it's pretty good at lazy-loading + it would reduce the maintenance burden for developing nixvim. Its variety of options and features could come in handy. It even has async execution and automatic caching and bytecode compilation of Lua modules for improved performance. You could provide options to configure it in nix and then translate that to lua like you do with the rest of nixvim. Lazy's I've written some examples below to illustrate the idea. {
plugins.lazy = {
enable = true;
# install any plugin available in the nix repos
nix = {
# this installs vimPlugins.neorg
neorg = {
build = ":Neorg sync-parsers";
dependencies = [
# these are all from the nix repos too
# they load when Neorg loads
plenary-nvim
nvim-treesitter
neorg-telescope = {
dependencies = [ telescope-nvim ];
config = ''
-- this is lua code
-- it does not get loaded until neorg-telescope is loaded
vim.wo.foldlevel = 99
'';
};
];
cmd = "Neorg";
ft = "norg";
keys = "<c-t>n";
# sometimes we like to keep the config in a different file
# so source it from elsewhere
config = import ./neorg.nix;
};
nvim-colorizer-lua = {
cmd = "ColorizerToggle"; # only load colorizer when I use :ColorizerToggle
config = ''
-- more lua code
vim.g.termguicolors = true,
require("colorizer").setup()
'';
};
};
# obscure plugin that isn't available in the nix repos
git."Vonr/align.nvim" = {
branch = "v2";
# only load align.nvim when the user presses <leader>a in visual mode
keys = [
{ "<leader>a"; mode = "v"; };
];
config = ''
vim.keymap.set(
'v',
'<leader>a',
function()
require'align'.align_to_string({
preview = true,
})
end,
{ noremap = true, silent = true }
)
'';
};
};
}This would install -- add lazy.nvim to the runtime path
vim.opt.rtp:prepend("/nix/store/rjkwbzdmdfp9gfrvi0rch5g5illqwbgf-vimplugin-lazy.nvim-2023-08-26")
require("lazy").setup({
{
name = "neorg",
dir = "/nix/store/iwxg2w4blbphyk7yvf7drqkmf8qfyv4x-vimplugin-neorg-2023-09-15",
build = ":Neorg sync-parsers",
dependencies = {
{ name = "plenary.nvim", dir = "/nix/store/4f17ss3v7lv580hmca8pc2q3zn33lqk4-lua5.1-plenary.nvim-2023-09-12" },
{ name = "nvim-treesitter", dir = "/nix/store/vp0lhrdzvknz7rb8pc6ndvv28cgsfy14-vimplugin-nvim-treesitter-2023-09-16" },
{
name = "telescope.nvim",
dir = "/nix/store/1ldgn56d6sgmk42n6dna3b91w3si3sn7-lua5.1-telescope.nvim-2023-09-16",
dependencies = { name = "neorg-telescope", dir = "/nix/store/rvg763iaqbx3czb52ghrsxhidvqqgwbk-vimplugin-neorg-telescope-2023-08-06" },
config = function()
-- this is lua code
-- it does not get loaded until neorg-telescope is loaded
vim.wo.foldlevel = 99
end,
},
},
cmd = "Neorg",
ft = "norg",
keys = "<c-t>n",
config = function()
require("path.to.neorg.config")
end,
},
{
name = "nvim-colorizer.lua",
dir = "/nix/store/03k72h7x4m543z7vrqjwbvqjx6j789qg-vimplugin-nvim-colorizer.lua-2023-02-27",
cmd = "ColorizerToggle",
config = function()
-- more lua code
vim.g.termguicolors = true,
require("colorizer").setup()
end,
},
{
"Vonr/align.nvim",
branch = "v2",
keys = {{ "<leader>a", mode = "v" }},
config = function()
vim.keymap.set(
'v',
'<leader>a',
function()
require'align'.align_to_string({
preview = true,
})
end,
{ noremap = true, silent = true }
)
end,
},
})Note that you shouldn't try to lazy-load lazy.nvim or the colourscheme. These should be loaded on startup. Cut me some slack if any of the nix or lua code above has mistakes. I'm pretty new to both languages and have so far not used them for anything besides configuring NixOS/home-manager and Neovim. |
|
Oh btw, I came across another project called nixneovim, which seems to have pretty much the same goal as you. I don't believe they do lazy-loading but maybe some of their stuff would interest you, such as their auto-updated plugins repo. |
|
Thanks for the link @koalagang ! |
you are right, no backporting. It's not critical software + we usually dont run the tests (when there are tests !) so backporting would be tricky. Plus the tooling is not great.
I am glad to learn about this, I've wanted to see this (not use it though) so that users can contribute to the nixpkgs plugin updater. Haven't seen much contribution from the usual contributors sadly. |
|
@teto if you are interested I have a different approach for my plugin management: |
|
plugins repo uses their own updater so I better understand why they dont contribute to upstream. Would be interesting to compare the features of both and see if we can merge both approaches. @traxys > I've seen that approach before. it's LGTM and to some extent, I wish we would leverage nix flake archive in the upstream updates like it's done for treesitter grammars already. Configuring neovim via nix is a bit tedious for most things so I use it only when the trade off is positive (plugins with complex dependencies and that) so I maintain a hybrid config with lazy to install hte other half of my plugins. |
|
@GaetanLepage I definitely agree that upstreaming plugins is ideal but surely users should have access to a way of installing plugins that haven't been upstreamed? If one wants a plugin immediately and said plugin is not in the nix repos then what would one do? Of course, you should then ask for this to be added and upstreamed so that all users can benefit but I don't think users should be forced to do this, especially considering that, like you rightfully said, those not on the unstable channel would have to wait for a whole new release to install those plugins. There should be some way of grabbing it straight from a git repo just like traditional vim plugin managers do. I think perhaps Anyway, this might be diverging slightly from the topic of how Nixvim is going to handle lazy-loading. |
If you're okay with having flakes as a requirement, you can use But I do like the idea of generating |
|
@musjj Haha I just discovered like 15 minutes after posting my comment that this is what flakes do 😅 But yeah, again I like the idea of using lazy.nvim. It should possible be to do it with home-manager rather than with nixvim, which I am considering. If, instead of using So for example: { pkgs, ... }:
{
programs.neovim.extraLuaConfig = ''
-- this is lua code
vim.opt.rtp:prepend(${pkgs.vimPlugins.lazy-nvim})
require("lazy").setup({
{
name = "nvim-colorizer.lua",
dir = "${pkgs.nvim-colorizer-lua}",
cmd = "ColorizerToggle",
config = function()
vim.g.termguicolors = true,
require("colorizer").setup()
end,
},
})
'';
}Note that The only difference between this and my suggestion earlier was basically that nixvim could do this but allowing the user to write it directly in nix rather than embedding lua code into a nix file. P.s. I've never done this nor have I seen it being done. It's just something that, logically, I think would work. |
|
Are plugin versions locked with |
|
lazy.nvim has a json lockfile that stores the branch and commit for each dependency that it pulls via git. I'm not sure how it handles local directory dependencies. |
|
As @bmanuel rightfully said, if pulling via git then it will use its own lockfile implementation ( |
|
@koalagang I agree, the goal is not to force the user to upstream packages.
|
This sounds like we could encapsulate the entire |
|
A lazy.nvim module sounds great, although I still think the original proposal of doing as much as possible in nix (at build time) is the ideal. That said, perfection is the enemy of good... |
|
Just popping back in here to add a datapoint for lazy.nvim: We used to have a module for a popular nvim package manager (packer? I forget...). It's what was used before to install packages outside of Nix, but had some issues. Either way, I'm not particularly opposed to having I don't really see a reason not to have both. |
Indeed! Otherwise, IIRC
We could trigger |
Not sure that's easily done given Nix's sandboxing. However we can definitely look at how lazy.nvim works under the hood, and even try to use some of its code. |
The original proposal was to use a nix config (with Though I do agree that a pure nix solution would be the most ideal. |
|
I'd like to clarify the idea more explicitly. The proposal with lazy.nvim was not for it to install plugins (although that certainly could be exposed as an option for those who want it). Plugins can be installed by nix at build time (including lazy.nvim itself using Additionally, lazy.nvim's For example, if the user sets {
name = "nvim-colorizer",
-- path to plugin in the nix store (installed at build time)
dir = "/nix/store/03k72h7x4m543z7vrqjwbvqjx6j789qg-vimplugin-nvim-colorizer.lua-2023-02-27",
config = function()
-- config generated by nixvim's nvim-colorizer module
require 'colorizer'.setup {
'css';
'javascript';
}
end,
},As you can see, we're installing the plugin using nix and then just pointing lazy.nvim to the location of the plugin in the nix store so that it can handle the lazy-loading and we're generating the plugin's lua configuration at build time. If the user, for instance, wants to lazy-load it with the I'm not sure if there's a more pure nix way of doing it since you'll have to generate the lua code anyway as Neovim doesn't natively understand nix but I agree with everyone that we should use as much nix as possible. |
I'm not really sure there's any need for that in that case. Seems like we're just using lazy.nvim as a library at this point, which I have no real objection to, though in that case it might be better to fork it ourselves and create a more special-purpose thing. Still though, as was said before, perfect is the enemy of good. If we go with this approach I don't really have any objections, especially considering it can be spun off later fairly transparently to the end-user. IMO the course of action for this would therefore be:
CC: @nix-community/nixvim. If everyone agrees, I can create a project board and the required issues. |
|
Perhaps we could have a "loading method" option which is an enumeration of:
This approach means we have a central way of selecting how plugins are loaded, seeing as (I assume) each approach will conflict the others. Alternatively, these could each be separate enable options, and we could panic if more than one is enabled. The proposed Having a centralised way of specifying the loading backend means supporting one method ( |
|
I agree with the enum, IMO it's the best option. Removes a lot of need for assertions. |
Entirely possible, though as you stated further down in the comment, we should probably wait until later before forking. I'd suggest taking a look at lazy's plugin spec for ideas. The most important options for lazy-loading imho are From lazy.nvim's README:
Note that, traditionally, the user has to manually mark plugins as dependencies but you could perhaps make nixvim aware of what plugins are dependencies of other plugins. I would suggest also making it possible to manually mark dependencies though as this can be useful in some situations where a plugin isn't necessarily an actual dependency of another. For example, in my config I have oil.nvim marked as a dependency of telescope-zoxide so that oil loads when I load telescope-zoxide. You could argue that this is a bit of a hack, though, and that this should be handled differently. Useful lazy.nvim options that are unrelated to lazy-loading but more so about how to install plugins include Edit: come to think of it, maybe these options are handled by upstream, at least
Yup. It would also modularise and future-proof the project more because you can really easily just keep adding more methods if you want to / if particular methods are requested. Also, remember that, regardless of what method the user goes for, you should set the colourscheme plugin to be the first plugin (other than the plugin manager itself) to load to avoid race conditions where some plugins are loaded before the colourscheme, causing them not to have the right colours. With lazy.nvim this can be done by setting |
Keeping a special-purpose Alternatively, we could try to merge our required features into |
That item in the enum wasn't supposed to hint at a fork (poor naming on my part). Rather it was supposed to represent the original proposal of this issue; lazy loading without using a plugin management plugin, or in other words a solution implemented at build time. As has been discussed above, this should be possible with minimal lua code. It'll just take a fair bit more time & effort than a solution that utilizes
All installation should be handled by nix, not at runtime. Therefore these options wouldn't be used. Instead the |
But what do you mean by lazy-loading at build time? You need a plugin of some sorts to listen for input (such as commands, keys, etc.) whilst the user is using Neovim. That's not something you can do at build time. Do you mean like generating autocmds?
Yes, I agree that it should be handled by nix. The point I was making was not that we should use those lazy.nvim options but I was just suggesting that perhaps the user being able to choose, for example what branch they want, could perhaps be a good thing. This would likely require them to use Anyway, regardless of which route nixvim ultimately takes, it might be best to start with a lazy.nvim module that provides all of its options and then aim to develop an in-house method later down the line if it's deemed necessary. |
|
Seeing how this is going, I'm going ahead and creating an issue for Even if we end up going with another route, it's still a nice to have. |
|
Yes! |
|
Btw, in regards to lazy.nixvim (I'm assuming that's the placeholder name we're going with for nixvim's in-house lazy-loading) do you believe it would make sense to have some sane defaults? I'm thinking perhaps there could be some option like I know this feature isn't exactly the next step in the plan but I figured I may as well post it here now since I just had the thought and perhaps you could look back on the issue later down the line. |
|
When it comes to the sane defaults, yeah, not really oposed to that. Honestly I think some plugins already sort of do that, with an option to disable it per-plugin, so maybe we could do it that way. Don't know, though. |
|
I have just came across this: https://github.com/b-src/lazy-nix-helper.nvim |
It is a neovim plugin that provides a function to search the Essentially you set If If All this happens at runtime (in lua), so it isn't the approach nixvim would take. It is useful for people who want to maintain a neovim config that can be used with or without a nix installation. |
|
Ok, I see. Thank you for the explanation. |
|
I've been experimenting after finding this: https://github.com/azuwis/lazyvim-nixvim/blob/master/flake.nix. |
|
Have done some more digging through the code base, I'm assuming this is where we could hook into ensuring only the plugin-manager is passed on. Obviously we'd have to then still use the list to ensure the configurations are applied however the chosen plugin-manager needs them... |
|
Recently we introduced new helpers to create plugin modules ( |
|
So... it looks like to use the new lazy module we ("we" being me, someone just using nixvim who doesn't really know what's going on) have to re-do how we setup plugins so they can be loaded with lazy? Is that something I should be doing, or is it going to change soon and we're in kindof an interim state at the moment? |
Currently, yes.
If you want to try out lazy loading now, yes. If you don't like re-doing your config, no.
The plan is to add a |
I had an idea what if lazy.nvim had an option that affects all plugins to lazy load the option could be |
|
This library could be useful: https://github.com/nvim-neorocks/lz.n |
I was just thinking this actually; if I could just enable it then set lazy loading options based on filetype or something and then just enable |
-->Summary: https://notes.glepage.com/rA_fyfKESvC9OxGGNHOYXw#Lazy-Loading <--
Currently, nixvim relies on the neovim built-in way of loading plugins.
This is implemented by nixpkgs neovim's module. It has the advantage of existing and leveraging the collection of plugins packaged in nixpkgs.
With that said, the very popular
lazy.nvimplugin manager proposes a more performant way of loading plugins.Imitating their lazy loading process could be a very valuable addition to nixvim.
I would guess that this feature should be optional so that conventional loading would stay the default.
As of today, no-one has started to work on this issue.
Feel free to go and make a PR, even if the work is in draft-stage. I will personally be glad to review and discuss a proposed implementation.
The text was updated successfully, but these errors were encountered: