Getting direnv.vim to work properly

[mdietrich16]

I am using Nix flakes with devShells for development environments, which get activated with direnv automagically. I think the way NixCats provides Nix-provided executables like LSPs by adding their binaries to the Neovim PATH is a problem when trying to integrate with direnv via direnv.vim.

When starting nvim in a directory that has an .envrc and is loaded by direnv, everything works at first, but when changing the directory, the direnv.vim plugins unloads the environment and with it, the modifications the Neovim wrapper applies to Neovim's PATH variable. This does not happen, when starting Neovim in a non-direnv directory, changing dir to one (and thus loading the env) and changing dir out of it again. Then PATH is properly reset, and at any point all binaries are available.

The concrete way this works (I believe) is that direnv (somehow) saves the initial shell state, opening neovim triggers direnv to look up if something needs to be done, it doesn't since the env is already loaded. But changing dirs outside unloads the environment, meaning a reset to the initial shell state. This means that the nixCats modification of PATH are also removed, since they were not there at the initial shell state.

Starting Neovim in a non-direnv directory an :cd-ing into one works as desired, because then the env has to be loaded from scratch and the Neovim-internal shell state is now properly tracked.

Any ideas on how to fix this?

[BirdeeHub]

This is not cleanly solveable with nixCats as it is.

I am honestly not really sure how this could be solveable at all from within a wrapper script for any program, not just nvim. At least not cleanly.

It might be, but I am not sure how. Basically, the plugin issues an event, and you would need to restore the environment from nvim after that event. But to do that, you have to know all the things that could be in that environment. NixCats allows you to provide arbitrary arguments to makeWrapper, so there isn't exactly a great way to get all the things that were added, its not just PATH additions, someone could theoretically change anything about the environment of the launched nvim.

Maybe it would be possible to printenv both at the start and end of the wrapper and find the difference, but then you would be reimplementing half of direnv in order to work around direnv.

Maybe a second wrapper script can be made which is designed to be sourced and provide everything but do nothing else?

The problem with either of those would be, you can't then just source that from within nvim, you would need to get direnv to source that. Because nvim can't edit the environment of the shell that called it. I guess it might be worth asking if a new nix wrapper generator can be created that does this environment editing directly within nvim?

But also, Im not sure that would even solve it, it still changes the environment without re-running any other config, which would probably still overwrite the changes even if done within your nvim config. It probably removes things added by mason even, I would assume.

Is there a global config for direnv? I haven't used it much, is there a global config for it that lets you add extra things to any shell you enter? Because if not, if you wanted to add that stuff back you would need to add whatever packages to every single shell config.

---

On top of that, it honestly feels like a bug in direnv.

Direnv works by running the stuff in a subshell and then propagating that to the host. It doesn't directly run in the host shell. When you swap over shells to a different one, shouldn't it be able to undo its modifications, rather than just setting the shell to its exact state from prior to the direnv command?

This could happen with a lot of things. There is a lot of stuff in nixpkgs wrapped simply for functionality. If you wanted to keep any of those running through multiple direnv calls, you would need to solve this problem. But text editors are likely to be the most obvious.

But say, for example, you wrapped npm-watch. If you then changed direnv like this, the wrapped npm-watch that is running would also break.

---

Is there a way to rewrite the direnv.vim plugin to make it so the new direnv goes on top of the current shell rather than replacing it? Replacing it is obviously ideal, but unless they fix that in direnv, I would think that stacking any extra ones on top of the first would be the next best option.

Or, maybe you just make the wrapper script check at the start if it is in a direnv shell, and if so, cancel it before entering nvim and then restarting it maybe? But that also feels messy.

[mdietrich16]

I thought that maybe unloading direnv before entering the wrapper shell or before setting anything custom at least could be a solution? I'm not sure how to do that though, I've read that unsetting DIRENV_DIR and DIRENV_WATCHES may be one way to do that, but I find debugging direnv to be not straightforward and I currently don't have enough time to really dive deep there.

[BirdeeHub]

Thats more or less what I suggested in my last sentence of my comment yeah.

TBH I have no idea how you would do it, I generally just type nix shell.

But if you could prevent direnv from knowing you are already in a direnv shell, it should avoid unloading stuff. So that does make sense. If you figure out a concrete way to do it, I would be interested to know though!

Otherwise, I'm in a similar boat to you, except I rarely change directories out of the one with the shell in it.

If I go into a project dir and load the dev shell and open neovim, I tend to leave it there and if I want a new directory I open a new tmux pane. So, this plugin probably wouldnt be something I would ever use the capabilities of, but now that I know it both exists and doesn't work, if I start using direnv more I will check it out.

[BirdeeHub]

Also, the same thing holds true of my new neovim wrapper in nix-wrapper-modules, although it is a lot better in a lot of other ways, and one likely has a better chance of solving this problem properly there than here.

[zenoli]

If I understand you correctly @mdietrich16, this is what is going on?

I'd argue that direnv and nix-cats are both "working as intended" and your workflow unfortunately uncovers in this unintuitive behavior.

I guess you are using neovim as a "long running session" and change directories to switch projects for example?
I share a similar view as @BirdeeHub that I cd to a directory, quickly shoot up a neovim instance and close it again once I'm done. This workflow is unaffected by the issue, which is probably why this is not a widespread problem in the community.

For the same reason I guess disabling the option autoload_direnv is not really an option? If it only disables the auto loading/unloading but keeps the remaining features of this plugin, this might be a compromise?

If nix cats collects all paths it injects in an env variable (NIX_CATS_PATH) a "hack" would be to make nix cats ensure that everything in NIX_CATS_PATH remains in $PATH after cd-ing by using some callback from the neovim api?

[BirdeeHub]

If nix cats collects all paths it injects in an env variable (NIX_CATS_PATH) a "hack" would be to make nix cats ensure that everything in NIX_CATS_PATH remains in $PATH after cd-ing by using some callback from the neovim api?

For the record, I personally will not be adding such a thing to nixCats, you would have to parse all the pkgs.makeWrapper args, it would be pretty crazy.

We might be able to do this with nix-wrapper-modules, because we already parse them out of the options and have helpers for that, we could put them in a separate file, or maybe even a lua file and source that at startup, and give a handle to it so you can run it again. But if we do this, it probably won't be soon unless someone else takes the initiative, and it doesn't really interact well with the function form of argv0type for the nix wrapper implementation... but I suppose we could just generate both the main file, and also an extra one for sourcing at runtime. But they would need to be in the same format as each other, so if the main one is bash, the other one has to be too. Maybe it could be added as a "lua" wrapper implementation where it basically all happens in lua both at startup and for the runtime sourceable one, and only create that file if using that implementation.

But, yes, this is the only solution I was able to think of last time I looked at the problem, but doing it in nixCats would be a lot of work, more so than in the new project. And it would not be all that easy in the new one either. But it would at least be able to do a better job of it.

If ALL you wanted was PATH, then it is not that hard. The moment you want to generalize it, and then have the escaping and all that work consistently everywhere, it gets hard.