vulncheck: remove isLocal check from fetchVulnerabilities

isLocal check was added to improve efficiency by avoiding
fetch of data that's not going to be used. Version info is
inaccurate or unavailable for modules that are in writable
local directories so vuln check for those modules are skipped
anyway.

With the check, fetchVulnerabilities excludes vulnerabilities
for modules if their source files are outside the module cache.
The location of the module cache was determined by querying
GOMODCACHE and GOPATH environment variables of the govulncheck
process. That worked well for govulncheck when it is used
for source scanning.

The logic was copied to vulncheck API internal. However,
relying on process's GOMODCACHE/GOPATH environment variables
limit the API's utility. For example, Gopls may use different
GOMODCACHE/GOPATH for each workspace it's processing and they
can be different from the Gopls's own GOMODCACHE/GOPATH env vars.
Test data can be loaded with a fake GOMODCACHE that's different
from the GOMODCACHE env var of the test process.

There was an escape flag to skip this check to work with
the test environment where the module cache and GOPATH
are different from the test process's. But that is unexported;
external packages cannot utilize it and that prevents
writing tests from external packages.

This CL proposes to remove the isLocal check. There is already
a cache that reduces volume of data fetch over network, and
vulncheck can potentially address the efficiency issue in different
ways. Users and applications that need to exclude
vulnerabilities of local modules, may utilize
golang.org/x/vuln/client.Client
and implement filtering from GetByModule. Or, if this problem
is common, we may consider an explicit setting in the
vulncheck.Config.

Cherry-picked: https://go-review.googlesource.com/c/exp/+/391914

Change-Id: Ib591f27563a7cd0ab00723d54d5a55e265e35752
Reviewed-on: https://go-review.googlesource.com/c/vuln/+/395063
Trust: Julie Qiu <julie@golang.org>
Run-TryBot: Julie Qiu <julie@golang.org>
Reviewed-by: Jonathan Amsterdam <jba@google.com>
4 files changed
tree: 997dcf484509a37201d026b618010e66a2d6e2b4
  1. client/
  2. devtools/
  3. internal/
  4. osv/
  5. vulncheck/
  6. .gitignore
  7. all_test.go
  8. AUTHORS
  9. checks.bash
  10. CONTRIBUTING.md
  11. CONTRIBUTORS
  12. go.mod
  13. go.sum
  14. LICENSE
  15. PATENTS
  16. README.md
  17. tools_test.go
README.md

Go Vulnerability Database

Go Reference

This repository contains code for hosting the Go Vulnerability Database. The actual reports can be found at x/vulndb.

Neither the code, nor the data, nor the existence of this repository is to be considered stable. See the Draft Design for details on this project.

Accessing the database

The Go vulnerability database is rooted at https://storage.googleapis.com/go-vulndb and provides data as JSON. We recommend using client.Client to read data from the Go vulnerability database.

Do not rely on the contents of the x/vulndb repository. The YAML files in that repository are maintained using an internal format that is subject to change without warning.

The endpoints the table below are supported. For each path:

  • $base is the path portion of a Go vulnerability database URL (https://storage.googleapis.com/go-vulndb).
  • $module is a module path
  • $vuln is a Go vulnerabilitiy ID (for example, GO-2021-1234)
PathDescription
$base/index.jsonList of module paths in the database mapped to its last modified timestamp (link).
$base/$module.jsonList of vulnerability entries for that module (example).
$base/ID/index.jsonList of all the vulnerability entries in the database
$base/ID/$vuln.jsonAn individual Go vulnerability report

Note that these paths and format are provisional and likely to change until an approved proposal.

Packages

Some of these packages can probably be coalesced, but for now are easier to work on in a more segmented fashion.

License

Unless otherwise noted, the Go source files are distributed under the BSD-style license found in the LICENSE file.

Database entries available at https://storage.googleapis.com/go-vulndb/ are distributed under the terms of the CC-BY 4.0 license.