| # Tests | |
| The tests directory contains integration tests that require a pre-seeded | |
| database. | |
| Each tests/<group> contains a seed.txt file at tests/<group>/seed.txt, which is | |
| used by the docker seeddb container to setup the test database. | |
| The tests can be run using tests/<group>/run.sh. | |
| ## Search Tests | |
| The tests/search/scripts directory contains test scripts \*.txt that are run | |
| using tests/search/run.sh. | |
| These scripts follow the format: | |
| ``` | |
| Test name A | |
| search query A | |
| Expected result A1 | |
| Expected result A2 | |
| Test name B | |
| search query B | |
| Expected result B1 | |
| Expected result B2 | |
| ... | |
| ``` | |
| Each test set is separated by a newline. Comments (lines that being with a #), | |
| are ignored. | |
| The imported by counts for a dataset can also be updated in | |
| search/importedby.txt. The file has the format: | |
| ``` | |
| <package-path>, <imported-by-count> | |
| ``` | |
| It is expected that the modules for these packages are in | |
| tests/search/seed.txt. | |
| ## Symbol History API Tests | |
| The tests/api/scripts directory contains tests that are run | |
| using tests/api/run.sh. | |
| The tests compare API data for a module `tests/api/testdata` with the API | |
| history on the frontend versions page. The frontend that is used for comparison | |
| must have `GO_DISCOVERY_SERVE_STATS` set so that the | |
| `/<path>?tab=versions&content=json` endpoint is available. | |
| ### Adding a package | |
| Data in `tests/api/testdata` is generated by running: | |
| ``` | |
| go run tests/api/main.go generate [module path]:[package path suffix] | |
| ``` | |
| For example, `go run tests/api/main.go generate github.com/julieqiu/api-demo:pkg` for package | |
| `github.com/julieqiu/api-demo/pkg`. | |
| This data can now be used to check the API history on a frontend by running: | |
| ``` | |
| go run tests/api/main.go compare [module path]:[package path suffix] | |
| ``` | |
| ## Screentest | |
| The screentest/ directory contains visual diff tests for pages on pkg.go.dev. | |
| Tests are written for the screentest tool from golang.org/x/website. | |
| See the [command](https://pkg.go.dev/golang.org/x/website/cmd/screentest) docs for | |
| more information. | |
| ### Running the tests | |
| In order to run the tests, run this command from the root of the repository: | |
| ``` | |
| ./tests/screentest/run.sh <env> | |
| ``` | |
| `./tests/screentest/run.sh ci` sets up a series of docker containers that run a | |
| postgres database, frontend, and headless chrome, and runs the screentest using | |
| headless chrome. | |
| Tests in `./tests/screentest/testcases.ci.txt` run only for CI and local runs of the | |
| screentest. These tests may rely on mocked data, like the vuln page tests. | |
| Tests in `./tests/screentest/testcases.txt` run for all environments and | |
| are suitable for tests with data that is consistent between environments or | |
| static pages. | |
| ### Updating the tests | |
| Update screen shot diffs using the following: | |
| 1. Run `./tests/screentest/run.sh -update ci` to collect the updated screenshots. | |
| 2. Optionally use `./tests/screentest/run.sh -update -run <test> ci` to update specific tests. | |
| 3. Copy the new screenshots from `./tests/screentest/output` into `./tests/screentest/testdata/testcases` | |
| ### Understanding Test Failures | |
| If the tests fail with diffs, a file with the diffs will be created in the | |
| testdata directory that show the cause of the failure. | |
| Use `./devtools/docker/compose.sh logs` to get detailed logs of the docker containers. | |
| ### Troubleshooting | |
| If docker fails with "connect: cannot assign requested address" during `run.sh`, | |
| try running it again. It will eventually succeed. Do not pass the '-rm' flag so that | |
| prior state is preserved. | |
| You can also try reducing the value of cmdgonetlimit in devtools/docker/compose.yaml. |