x/tools/cmd/gopls: Return Newly Added Modules in KnownPackages

[marwan-at-work]

KnownPackages is a gopls command that returns a list of all packages a workspace file can add to its "import" statement.

It is currently used by VSCode's "Add Import" functionality.

However, if you had just ran a "go get" command on a new module, that new module's packages will not show up. This is because it is a "newly added module", meaning it is recorded in your go.mod file but none of your Go files import a package within this module just yet. This is usually the typical time you'd want to list the available packages to add one to your import statement.

The reason for KnownPackages not returning newly added module packages is that the implementation of KnownPackages does two things:

1. It considers all import paths that are cached in its session (which obviously wouldn't yet have the newly added module)
2. It randomly searches all of your GOMODCACHE through the GetAllCandidates function but only limits itself to 80 milliseconds and returns the partial results. This is because it would take too long to search all of your GOMODCACHE directory. Therefore it is a hit or miss and often returns completely irrelevant results.

Proposal

I propose that we change the above implementation to something more guaranteed and would also fix the "newly added module" issue above:

When a client asks for KnownPackages relative to a file, we simply get the go.mod that belongs to that file by calling snapshot.GoModForFile then parse the file to get its dependencies. From there, we can simple loop over the "require" statements and find their location and return the valid packages within each module@version.

This achieves a few things:

1. It only take into consideration your newly added module and show its packages.
2. It should be quite fast because we don't run any "go list" or search all of the GOMODCACHE, but only what is in your go.mod file. Also worth noting that "go list" analyzes your import paths (IIRC) which means it would skip any newly added modules.
3. With Go1.17 in your go.mod file, we know all of your dependencies whether direct or indirect are recorded there, so we'll always return exactly every single package that is truly importable within your module. For versions less than that, we can either keep the old behavior or apply the new behavior as arguably you don't usually import indirect packages that you may not even know is in your transitive dependencies.

Alternatives:

1. Running "go list" from the root go.mod directory (will not pick up newly added modules)

Edge cases:

1. This of course would not work in GOPATH mode, but in that case we can just default to the old behavior or just show the std lib.
2. In neither GOPATH/Module modes, we can just show the standard library packages.

[gopherbot]

Change https://golang.org/cl/351650 mentions this issue: internal/lsp/source: Deduce KnownPackages from the go.mod file

[pavlelee]

I use gopls 0.8.1, list_known_packages return incomplete package list. I modify 80 milliseconds to 30s will return complete package list

[pavlelee]

any response? Add imports is a very useful action

[pavlelee]

go list -f "{{.Name}};{{.ImportPath}}" all is incomplete in go mod, nearly two-thirds less.

go list -f "{{.Name}};{{.ImportPath}}" all
gopkgs -format '{{.Name}};{{.ImportPath}};{{.Dir}}' -workDir ./