tests/README.md

# Tests

The tests directory contains integration tests that require a pre-seeded
database.

Each tests/<group> contains a seed.txt file at tests/<group>/seed.txt, which is
used by the docker seeddb container to setup the test database.

The tests can be run using tests/<group>/run.sh.

## Search Tests

The tests/search/scripts directory contains test scripts \*.txt that are run
using tests/search/run.sh.

These scripts follow the format:

```
Test name A
search query A
Expected result A1
Expected result A2

Test name B
search query B
Expected result B1
Expected result B2

...
```

Each test set is separated by a newline. Comments (lines that being with a #),
are ignored.

The imported by counts for a dataset can also be updated in
search/importedby.txt. The file has the format:

```
<package-path>, <imported-by-count>
```

It is expected that the modules for these packages are in
tests/search/seed.txt.

## Symbol History API Tests

The tests/api/scripts directory contains tests that are run
using tests/api/run.sh.

The tests compare API data for a module `tests/api/testdata` with the API
history on the frontend versions page. The frontend that is used for comparison
must have `GO_DISCOVERY_SERVE_STATS` set so that the
`/<path>?tab=versions&content=json` endpoint is available.

### Adding a package

Data in `tests/api/testdata` is generated by running:

```
go run tests/api/main.go generate [module path]:[package path suffix]
```

For example, `go run tests/api/main.go generate github.com/julieqiu/api-demo:pkg` for package
`github.com/julieqiu/api-demo/pkg`.

This data can now be used to check the API history on a frontend by running:

```
go run tests/api/main.go compare [module path]:[package path suffix]
```

## Screentest

The screentest/ directory contains visual diff tests for pages on pkg.go.dev
In order to run the tests, run this command from the root of the repository:

```
./tests/screentest/run.sh <env>
```

`./tests/screentest/run.sh ci` sets up a series of docker containers that run a
postgres database, frontend, and headless chrome, and runs the screentest using
headless chrome.

### Understanding Test Failures

If the tests fail with diffs, a file with the diffs will be created in the
testdata directory that show the cause of the failure.

If the tests fail with "context deadline exceeded", you may have forgotten to
seed your local DB. Run with with `-seeddb` flag.

Use `./devtools/docker/compose.sh logs` to get detailed logs of the docker containers.

### Adding Testcases

Tests are written for the screentest tool from golang.org/x/website.
See the [command](https://pkg.go.dev/golang.org/x/website/cmd/screentest) and
[package](https://pkg.go.dev/golang.org/x/website/internal/screentest) docs for
more information.