commit	1d8c3548c0bbc477bdf7cb400b72a4cf9c7d362e	[log] [tgz]
author	Dmitri Shuralyov <dmitshur@golang.org>	Fri Sep 14 16:38:33 2018 -0400
committer	Dmitri Shuralyov <dmitshur@golang.org>	Fri Sep 14 21:44:00 2018 +0000
tree	2fb0c9eadc2c423301cfcf1254f067f5edffd29a
parent	431df8c332a83eb21f868aefbdea03eaa63f902b [diff]

internal/gophers: restore valid Gerrit emails

CL 132995 made a lot of changes to the gophers table based on
the output of gopherstats -mode=find-gerrit-gophers, which is
a new cmd/gopherstats mode added in that very CL.

The current implementation of Person.mergeIDs makes it so that
the first email seen is considered the Gerrit email.

Many of the additions of CL 132995 did not take that into account,
causing the Gerrit field to change for some gophers incorrectly.

Overall, that CL caused the following changes to gophers:

	Github field changes:
	tal@whatexit.org: GitHub="" -> "TomOnTime"
	Carl Henrik Lunde: GitHub="" -> "chlunde"
	Dieter Plaetinck: GitHub="" -> "Dieterbe"
	Diogo Pinela: GitHub="" -> "dpinela"
	Frank Schroeder: GitHub="" -> "magiconair"
	Gregory Man: GitHub="" -> "gregory-m"
	Jan Berktold: GitHub="" -> "JanBerktold"
	Jean de Klerk: GitHub="" -> "jadekler"
	Josselin Costanzi: GitHub="" -> "josselin-c"
	Martin Garton: GitHub="MartinGarton" -> "mjgarton"
	Matt Harden: GitHub="" -> "nerdatmath"
	Michael Darakananda: GitHub="" -> "pongad"
	Mostyn Bramley-Moore: GitHub="" -> "mostynb"
	Nicholas Maniscalco: GitHub="" -> "nicholasmaniscalco"
	Roland Illig: GitHub="" -> "rillig"
	Yasha Bubnov: GitHub="" -> "ybubnov"
	Zheng Xu: GitHub="" -> "Zheng-Xu"
	ltnwgl: GitHub="" -> "gengliangwang"
	oiooj: GitHub="" -> "oiooj"
	thoeni: GitHub="" -> "thoeni"

	Gerrit field changes:
	Andrew Bonventre: Gerrit="andybons@golang.org" -> "365204+andybons@users.noreply.github.com"
	Carl Mastrangelo: Gerrit="notcarl@google.com" -> "carl.mastrangelo@gmail.com"
	Chris McGee: Gerrit="sirnewton_01@yahoo.ca" -> "newton688@gmail.com"
	Eric Lagergren: Gerrit="ericscottlagergren@gmail.com" -> "eric@ericlagergren.com"
	Filippo Valsorda: Gerrit="filippo@golang.org" -> "6195@62eb7196-b449-3ce5-99f1-c037f21e1705"
	Guillaume J. Charmes: Gerrit="guillaume@charmes.net" -> "gcharmes@magicleap.com"
	Harshavardhana: Gerrit="hrshvardhana@gmail.com" -> "harsha@minio.io"
	Jean de Klerk: Gerrit="jadekler@gmail.com" -> "deklerk@google.com"
	Joe Tsai: Gerrit="joetsai@google.com" -> "joetsai@digital-static.net"
	Martin Möhrmann: Gerrit="moehrmann@google.com" -> "martisch@uos.de"
	Matthew Dempsky: Gerrit="mdempsky@google.com" -> "matthew@dempsky.org"
	Olivier Poitrey: Gerrit="rs@dailymotion.com" -> "rs@netflix.com"
	Paul Jolly: Gerrit="paul@myitcv.org.uk" -> "paul@myitcv.io"
	Ralph Corderoy: Gerrit="ralph@inputplus.co.uk" -> "ralph.corderoy@gmail.com"
	Raul Silvera: Gerrit="rsilvera@google.com" -> "rauls5382@gmail.com"
	Richard Miller: Gerrit="miller.research@gmail.com" -> "millerresearch@gmail.com"
	Sebastien Binet: Gerrit="seb.binet@gmail.com" -> "binet@cern.ch"
	Tobias Klauser: Gerrit="tobias.klauser@gmail.com" -> "tklauser@distanz.ch"
	Vitor De Mario: Gerrit="vitordemario@gmail.com" -> "vitor.demario@mendelics.com.br"

	Googler field changes:
	Aaron Kemp: Googler=false -> true
	Jason Hall: Googler=false -> true
	Jean de Klerk: Googler=false -> true

	(It also caused many emails to be added, but I'm not considering
	those changes since they're not relevant to golang/go#27517 and
	aren't causing harm.)

All of the Github and Googler field changes are good,
but not all of the Gerrit field changes are good.
I've manually checked them against the
go-review.googlesource.com Gerrit server,
and classified them as follows:

	bad  Andrew Bonventre: Gerrit="andybons@golang.org" -> "365204+andybons@users.noreply.github.com"
	bad  Carl Mastrangelo: Gerrit="notcarl@google.com" -> "carl.mastrangelo@gmail.com"
	ok   Chris McGee: Gerrit="sirnewton_01@yahoo.ca" -> "newton688@gmail.com"
	bad  Eric Lagergren: Gerrit="ericscottlagergren@gmail.com" -> "eric@ericlagergren.com"
	bad  Filippo Valsorda: Gerrit="filippo@golang.org" -> "6195@62eb7196-b449-3ce5-99f1-c037f21e1705"
	bad  Guillaume J. Charmes: Gerrit="guillaume@charmes.net" -> "gcharmes@magicleap.com"
	bad  Harshavardhana: Gerrit="hrshvardhana@gmail.com" -> "harsha@minio.io"
	ok   Jean de Klerk: Gerrit="jadekler@gmail.com" -> "deklerk@google.com"
	bad  Joe Tsai: Gerrit="joetsai@google.com" -> "joetsai@digital-static.net"
	bad  Martin Möhrmann: Gerrit="moehrmann@google.com" -> "martisch@uos.de"
	bad  Matthew Dempsky: Gerrit="mdempsky@google.com" -> "matthew@dempsky.org"
	ok   Olivier Poitrey: Gerrit="rs@dailymotion.com" -> "rs@netflix.com"
	bad  Paul Jolly: Gerrit="paul@myitcv.org.uk" -> "paul@myitcv.io"
	bad  Ralph Corderoy: Gerrit="ralph@inputplus.co.uk" -> "ralph.corderoy@gmail.com"
	bad  Raul Silvera: Gerrit="rsilvera@google.com" -> "rauls5382@gmail.com"
	ok   Richard Miller: Gerrit="miller.research@gmail.com" -> "millerresearch@gmail.com"
	bad  Sebastien Binet: Gerrit="seb.binet@gmail.com" -> "binet@cern.ch"
	bad  Tobias Klauser: Gerrit="tobias.klauser@gmail.com" -> "tklauser@distanz.ch"
	bad  Vitor De Mario: Gerrit="vitordemario@gmail.com" -> "vitor.demario@mendelics.com.br"

I considered any @google.com -> non-@google.com changes as bad.
For the rest, it was based on which email was recognized by the
Gerrit server and had more activity overall, as well as recently.

This CL undoes all the bad Gerrit field changes, reverting them
to their original pre-CL 132995 values. It also changes the Gerrit
email for Gopherbot, and cleans up my own entry. That leaves just:

	Gerrit field changes:
	Chris McGee: Gerrit="sirnewton_01@yahoo.ca" -> "newton688@gmail.com"
	Jean de Klerk: Gerrit="jadekler@gmail.com" -> "deklerk@google.com"
	Olivier Poitrey: Gerrit="rs@dailymotion.com" -> "rs@netflix.com"
	Richard Miller: Gerrit="miller.research@gmail.com" -> "millerresearch@gmail.com"

In future CLs, we'll need to be careful with the order in which
emails are added, until golang/go#27631 is resolved.

Updates golang/go#27631.
Fixes golang/go#27517.

Change-Id: I6bd289af6ea2c50c293c4576de3873658994b98a
Reviewed-on: https://go-review.googlesource.com/135456
Reviewed-by: Andrew Bonventre <andybons@golang.org>

internal/gophers/gophers.go[diff]

1 file changed

tree: 2fb0c9eadc2c423301cfcf1254f067f5edffd29a

README.md

Go Build Tools

This subrepository holds the source for various packages and tools that support Go's build system and the development of the Go programming language.

Report Issues / Send Patches

This repository uses Gerrit for code changes. To contribute, see https://golang.org/doc/contribute.html.

The main issue tracker for the blog is located at https://github.com/golang/go/issues. Prefix your issue with “x/build/DIR: ” in the subject line.

Overview

The main components of the Go build system are:

The dashboard, in app/, serves https://build.golang.org/. It runs on App Engine and holds the state for which builds passed or failed, and stores the build failure logs for post-submit failures. (Trybot build failure logs are stored elsewhere). The dashboard does not execute any builds on its own.
The coordinator, in cmd/coordinator/, serves https://farmer.golang.org/. It runs on GKE and coordinates the whole build system. It finds work to do (both pre-submit “TryBot” work, and post-submit work) and executes builds, allocating machines to run the builds. It is the owner of all machines.
The Go package in buildenv/ contains constants for where the dashboard and coordinator run, for prod, staging, and local development.
The buildlet, in cmd/buildlet/, is the HTTP server that runs on each worker machine to execute builds on the coordinator's behalf. This runs on every possible GOOS/GOARCH value. The buildlet binaries are stored on Google Cloud Storage and fetched per-build, so we can update the buildlet binary independently of the underlying machine images. The buildlet is the most insecure server possible: it has HTTP handlers to read & write arbitrary content to disk, and to execute any file on disk. It also has an SSH tunnel handler. The buildlet must never be exposed to the Internet. The coordinator provisions buildlets in one of three ways:
1. by creating VMs on Google Compute Engine (GCE) with custom images configured to fetch & run the buildlet on boot, listening on port 80 in a private network.
2. by running Linux containers (on either Google Kubernetes Engine or GCE with the Container-Optimized OS image), with the container images configured to fetch & run the buildlet on start, also listening on port 80 in a private network.
3. by taking buildlets out of a pool of connected, dedicated machines. The buildlet can run in either listen mode (as on GCE and GKE) or in reverse mode. In reverse mode, the buildlet connects out to https://farmer.golang.org/ and registers itself with the coordinator. The TCP connection is then logically reversed (using revdial and when the coordinator needs to do a build, it makes HTTP requests to the coordinator over the already-open TCP connection.
These three pools can be viewed at the coordinator's http://farmer.golang.org/#pools
The env/ directory describes build environments. It contains scripts to create VM images, Dockerfiles to create Kubernetes containers, and instructions and tools for dedicated machines.
maintner in maintner/ is a library for slurping all of Go's GitHub and Gerrit state into memory. The daemon maintnerd in maintner/maintnerd/ runs on GKE and serves https://maintner.golang.org/. The daemon watches GitHub and Gerrit and apps to a mutation log whenever it sees new activity. The logs are stored on GCS and served to clients.
The godata package in maintner/godata/ provides a trivial API to let anybody write programs against Go's maintner corpus (all of our GitHub and Gerrit history), live up to the second. It takes a few seconds to load into memory and a few hundred MB of RAM after it downloads the mutation log from the network.
pubsubhelper in cmd/pubsubhelper/ is a dependency of maintnerd. It runs on GKE, is available at https://pubsubhelper.golang.org/, and runs an HTTP server to receive Webhook updates from GitHub on new activity and an SMTP server to receive new activity emails from Gerrit. It then is a pubsub system for maintnerd to subscribe to.
The gitmirror server in cmd/gitmirror/ mirrors Gerrit to GitHub, and also serves a mirror of the Gerrit code to the coordinator for builds, so we don't overwhelm Gerrit and blow our quota.
The Go gopherbot bot logic runs on GKE. The code is in cmd/gopherbot. It depends on maintner via the godata package.
The developer dashboard at https://dev.golang.org/ runs on GKE. Its code is in devapp/. It also depends on maintner via the godata package.
cmd/retrybuilds: a Go client program to delete build results from the dashboard

Adding a Go Builder

If you wish to run a Go builder, please email golang-dev@googlegroups.com first. There is documentation at https://golang.org/wiki/DashboardBuilders, but depending on the type of builder, we may want to run it ourselves, after you prepare an environment description (resulting in a VM image) of it. See the env directory.