internal/lsp: restructure user options (CL 278433 continued)

This CL copies Heschi's structural changes to the options from CL 278433
and makes the necessary adjustments in the JSON and documentation
generation. Nested settings are grouped together and the "status" of a
given setting is also listed. Currently the only possible statuses are
"experimental" and "debug", but I will add "advanced" in a follow-up (to
indicate that a setting is only for advanced users).

The options "set" function still expects flattened settings to avoid
fundamentally changing people's current configurations, so VS Code Go
will just have to make sure to flatten the settings before sending them
to gopls (which should be easy enough).

No names of any settings are changed (Heschi's earlier CL adjusted the
experimental prefixes). As discussed offline, we've decided to prefix
any setting that we expect to delete with "experimental", and so we'll
leave existing setting names as they are.

Updates golang/go#43101

Change-Id: I55cf7ef09ce7b5b1f8af06fcadb4ba2a44ec9b17
Reviewed-on: https://go-review.googlesource.com/c/tools/+/280192
Trust: Rebecca Stambler <rstambler@golang.org>
Run-TryBot: Rebecca Stambler <rstambler@golang.org>
gopls-CI: kokoro <noreply+kokoro@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
Reviewed-by: Suzy Mueller <suzmue@golang.org>
diff --git a/internal/lsp/source/api_json.go b/internal/lsp/source/api_json.go
index e2444e1..85d91cd 100755
--- a/internal/lsp/source/api_json.go
+++ b/internal/lsp/source/api_json.go
@@ -4,104 +4,6 @@
 
 var GeneratedAPIJSON = &APIJSON{
 	Options: map[string][]*OptionJSON{
-		"Debugging": {
-			{
-				Name:       "verboseOutput",
-				Type:       "bool",
-				Doc:        "verboseOutput enables additional debug logging.\n",
-				EnumValues: nil,
-				Default:    "false",
-			},
-			{
-				Name:       "completionBudget",
-				Type:       "time.Duration",
-				Doc:        "completionBudget is the soft latency goal for completion requests. Most\nrequests finish in a couple milliseconds, but in some cases deep\ncompletions can take much longer. As we use up our budget we\ndynamically reduce the search scope to ensure we return timely\nresults. Zero means unlimited.\n",
-				EnumValues: nil,
-				Default:    "\"100ms\"",
-			},
-		},
-		"Experimental": {
-			{
-				Name: "annotations",
-				Type: "map[string]bool",
-				Doc:  "annotations specifies the various kinds of optimization diagnostics\nthat should be reported by the gc_details command.\n",
-				EnumValues: []EnumValue{
-					{
-						Value: "\"bounds\"",
-						Doc:   "`\"bounds\"` controls bounds checking diagnostics.\n",
-					},
-					{
-						Value: "\"escape\"",
-						Doc:   "`\"escape\"` controls diagnostics about escape choices.\n",
-					},
-					{
-						Value: "\"inline\"",
-						Doc:   "`\"inline\"` controls diagnostics about inlining choices.\n",
-					},
-					{
-						Value: "\"nil\"",
-						Doc:   "`\"nil\"` controls nil checks.\n",
-					},
-				},
-				Default: "{\"bounds\":true,\"escape\":true,\"inline\":true,\"nil\":true}",
-			},
-			{
-				Name:       "staticcheck",
-				Type:       "bool",
-				Doc:        "staticcheck enables additional analyses from staticcheck.io.\n",
-				EnumValues: nil,
-				Default:    "false",
-			},
-			{
-				Name:       "semanticTokens",
-				Type:       "bool",
-				Doc:        "semanticTokens controls whether the LSP server will send\nsemantic tokens to the client.\n",
-				EnumValues: nil,
-				Default:    "false",
-			},
-			{
-				Name:       "expandWorkspaceToModule",
-				Type:       "bool",
-				Doc:        "expandWorkspaceToModule instructs `gopls` to adjust the scope of the\nworkspace to find the best available module root. `gopls` first looks for\na go.mod file in any parent directory of the workspace folder, expanding\nthe scope to that directory if it exists. If no viable parent directory is\nfound, gopls will check if there is exactly one child directory containing\na go.mod file, narrowing the scope to that directory if it exists.\n",
-				EnumValues: nil,
-				Default:    "true",
-			},
-			{
-				Name:       "experimentalWorkspaceModule",
-				Type:       "bool",
-				Doc:        "experimentalWorkspaceModule opts a user into the experimental support\nfor multi-module workspaces.\n",
-				EnumValues: nil,
-				Default:    "false",
-			},
-			{
-				Name:       "experimentalDiagnosticsDelay",
-				Type:       "time.Duration",
-				Doc:        "experimentalDiagnosticsDelay controls the amount of time that gopls waits\nafter the most recent file modification before computing deep diagnostics.\nSimple diagnostics (parsing and type-checking) are always run immediately\non recently modified packages.\n\nThis option must be set to a valid duration string, for example `\"250ms\"`.\n",
-				EnumValues: nil,
-				Default:    "\"250ms\"",
-			},
-			{
-				Name:       "experimentalPackageCacheKey",
-				Type:       "bool",
-				Doc:        "experimentalPackageCacheKey controls whether to use a coarser cache key\nfor package type information to increase cache hits. This setting removes\nthe user's environment, build flags, and working directory from the cache\nkey, which should be a safe change as all relevant inputs into the type\nchecking pass are already hashed into the key. This is temporarily guarded\nby an experiment because caching behavior is subtle and difficult to\ncomprehensively test.\n",
-				EnumValues: nil,
-				Default:    "true",
-			},
-			{
-				Name:       "allowModfileModifications",
-				Type:       "bool",
-				Doc:        "allowModfileModifications disables -mod=readonly, allowing imports from\nout-of-scope modules. This option will eventually be removed.\n",
-				EnumValues: nil,
-				Default:    "false",
-			},
-			{
-				Name:       "allowImplicitNetworkAccess",
-				Type:       "bool",
-				Doc:        "allowImplicitNetworkAccess disables GOPROXY=off, allowing implicit module\ndownloads rather than requiring user action. This option will eventually\nbe removed.\n",
-				EnumValues: nil,
-				Default:    "false",
-			},
-		},
 		"User": {
 			{
 				Name:       "buildFlags",
@@ -109,6 +11,8 @@
 				Doc:        "buildFlags is the set of flags passed on to the build system when invoked.\nIt is applied to queries like `go list`, which is used when discovering files.\nThe most common use is to set `-tags`.\n",
 				EnumValues: nil,
 				Default:    "[]",
+				Status:     "",
+				Hierarchy:  "build",
 			},
 			{
 				Name:       "env",
@@ -116,6 +20,62 @@
 				Doc:        "env adds environment variables to external commands run by `gopls`, most notably `go list`.\n",
 				EnumValues: nil,
 				Default:    "{}",
+				Status:     "",
+				Hierarchy:  "build",
+			},
+			{
+				Name:       "directoryFilters",
+				Type:       "[]string",
+				Doc:        "directoryFilters can be used to exclude unwanted directories from the\nworkspace. By default, all directories are included. Filters are an\noperator, `+` to include and `-` to exclude, followed by a path prefix\nrelative to the workspace folder. They are evaluated in order, and\nthe last filter that applies to a path controls whether it is included.\nThe path prefix can be empty, so an initial `-` excludes everything.\n\nExamples:\nExclude node_modules: `-node_modules`\nInclude only project_a: `-` (exclude everything), `+project_a`\nInclude only project_a, but not node_modules inside it: `-`, `+project_a`, `-project_a/node_modules`\n",
+				EnumValues: nil,
+				Default:    "[]",
+				Status:     "",
+				Hierarchy:  "build",
+			},
+			{
+				Name:       "expandWorkspaceToModule",
+				Type:       "bool",
+				Doc:        "expandWorkspaceToModule instructs `gopls` to adjust the scope of the\nworkspace to find the best available module root. `gopls` first looks for\na go.mod file in any parent directory of the workspace folder, expanding\nthe scope to that directory if it exists. If no viable parent directory is\nfound, gopls will check if there is exactly one child directory containing\na go.mod file, narrowing the scope to that directory if it exists.\n",
+				EnumValues: nil,
+				Default:    "true",
+				Status:     "experimental",
+				Hierarchy:  "build",
+			},
+			{
+				Name:       "experimentalWorkspaceModule",
+				Type:       "bool",
+				Doc:        "experimentalWorkspaceModule opts a user into the experimental support\nfor multi-module workspaces.\n",
+				EnumValues: nil,
+				Default:    "false",
+				Status:     "experimental",
+				Hierarchy:  "build",
+			},
+			{
+				Name:       "experimentalPackageCacheKey",
+				Type:       "bool",
+				Doc:        "experimentalPackageCacheKey controls whether to use a coarser cache key\nfor package type information to increase cache hits. This setting removes\nthe user's environment, build flags, and working directory from the cache\nkey, which should be a safe change as all relevant inputs into the type\nchecking pass are already hashed into the key. This is temporarily guarded\nby an experiment because caching behavior is subtle and difficult to\ncomprehensively test.\n",
+				EnumValues: nil,
+				Default:    "true",
+				Status:     "experimental",
+				Hierarchy:  "build",
+			},
+			{
+				Name:       "allowModfileModifications",
+				Type:       "bool",
+				Doc:        "allowModfileModifications disables -mod=readonly, allowing imports from\nout-of-scope modules. This option will eventually be removed.\n",
+				EnumValues: nil,
+				Default:    "false",
+				Status:     "experimental",
+				Hierarchy:  "build",
+			},
+			{
+				Name:       "allowImplicitNetworkAccess",
+				Type:       "bool",
+				Doc:        "allowImplicitNetworkAccess disables GOPROXY=off, allowing implicit module\ndownloads rather than requiring user action. This option will eventually\nbe removed.\n",
+				EnumValues: nil,
+				Default:    "false",
+				Status:     "experimental",
+				Hierarchy:  "build",
 			},
 			{
 				Name: "hoverKind",
@@ -143,14 +103,9 @@
 						Doc:   "",
 					},
 				},
-				Default: "\"FullDocumentation\"",
-			},
-			{
-				Name:       "usePlaceholders",
-				Type:       "bool",
-				Doc:        "placeholders enables placeholders for function parameters or struct fields in completion responses.\n",
-				EnumValues: nil,
-				Default:    "false",
+				Default:   "\"FullDocumentation\"",
+				Status:    "",
+				Hierarchy: "ui.documentation",
 			},
 			{
 				Name:       "linkTarget",
@@ -158,34 +113,8 @@
 				Doc:        "linkTarget controls where documentation links go.\nIt might be one of:\n\n* `\"godoc.org\"`\n* `\"pkg.go.dev\"`\n\nIf company chooses to use its own `godoc.org`, its address can be used as well.\n",
 				EnumValues: nil,
 				Default:    "\"pkg.go.dev\"",
-			},
-			{
-				Name:       "local",
-				Type:       "string",
-				Doc:        "local is the equivalent of the `goimports -local` flag, which puts imports beginning with this string after 3rd-party packages.\nIt should be the prefix of the import path whose imports should be grouped separately.\n",
-				EnumValues: nil,
-				Default:    "\"\"",
-			},
-			{
-				Name:       "gofumpt",
-				Type:       "bool",
-				Doc:        "gofumpt indicates if we should run gofumpt formatting.\n",
-				EnumValues: nil,
-				Default:    "false",
-			},
-			{
-				Name:       "analyses",
-				Type:       "map[string]bool",
-				Doc:        "analyses specify analyses that the user would like to enable or disable.\nA map of the names of analysis passes that should be enabled/disabled.\nA full list of analyzers that gopls uses can be found [here](analyzers.md)\n\nExample Usage:\n```json5\n...\n\"analyses\": {\n  \"unreachable\": false, // Disable the unreachable analyzer.\n  \"unusedparams\": true  // Enable the unusedparams analyzer.\n}\n...\n```\n",
-				EnumValues: nil,
-				Default:    "{}",
-			},
-			{
-				Name:       "codelenses",
-				Type:       "map[string]bool",
-				Doc:        "codelenses overrides the enabled/disabled state of code lenses. See the \"Code Lenses\"\nsection of settings.md for the list of supported lenses.\n\nExample Usage:\n```json5\n\"gopls\": {\n...\n  \"codelenses\": {\n    \"generate\": false,  // Don't show the `go generate` lens.\n    \"gc_details\": true  // Show a code lens toggling the display of gc's choices.\n  }\n...\n}\n```\n",
-				EnumValues: nil,
-				Default:    "{\"gc_details\":false,\"generate\":true,\"regenerate_cgo\":true,\"tidy\":true,\"upgrade_dependency\":true,\"vendor\":true}",
+				Status:     "",
+				Hierarchy:  "ui.documentation",
 			},
 			{
 				Name:       "linksInHover",
@@ -193,6 +122,48 @@
 				Doc:        "linksInHover toggles the presence of links to documentation in hover.\n",
 				EnumValues: nil,
 				Default:    "true",
+				Status:     "",
+				Hierarchy:  "ui.documentation",
+			},
+			{
+				Name:       "usePlaceholders",
+				Type:       "bool",
+				Doc:        "placeholders enables placeholders for function parameters or struct\nfields in completion responses.\n",
+				EnumValues: nil,
+				Default:    "false",
+				Status:     "",
+				Hierarchy:  "ui.completion",
+			},
+			{
+				Name:       "completionBudget",
+				Type:       "time.Duration",
+				Doc:        "completionBudget is the soft latency goal for completion requests. Most\nrequests finish in a couple milliseconds, but in some cases deep\ncompletions can take much longer. As we use up our budget we\ndynamically reduce the search scope to ensure we return timely\nresults. Zero means unlimited.\n",
+				EnumValues: nil,
+				Default:    "\"100ms\"",
+				Status:     "debug",
+				Hierarchy:  "ui.completion",
+			},
+			{
+				Name: "matcher",
+				Type: "enum",
+				Doc:  "matcher sets the algorithm that is used when calculating completion\ncandidates.\n",
+				EnumValues: []EnumValue{
+					{
+						Value: "\"CaseInsensitive\"",
+						Doc:   "",
+					},
+					{
+						Value: "\"CaseSensitive\"",
+						Doc:   "",
+					},
+					{
+						Value: "\"Fuzzy\"",
+						Doc:   "",
+					},
+				},
+				Default:   "\"Fuzzy\"",
+				Status:    "advanced",
+				Hierarchy: "ui.completion",
 			},
 			{
 				Name: "importShortcut",
@@ -212,27 +183,9 @@
 						Doc:   "",
 					},
 				},
-				Default: "\"Both\"",
-			},
-			{
-				Name: "matcher",
-				Type: "enum",
-				Doc:  "matcher sets the algorithm that is used when calculating completion candidates.\n",
-				EnumValues: []EnumValue{
-					{
-						Value: "\"CaseInsensitive\"",
-						Doc:   "",
-					},
-					{
-						Value: "\"CaseSensitive\"",
-						Doc:   "",
-					},
-					{
-						Value: "\"Fuzzy\"",
-						Doc:   "",
-					},
-				},
-				Default: "\"Fuzzy\"",
+				Default:   "\"Both\"",
+				Status:    "",
+				Hierarchy: "ui.navigation",
 			},
 			{
 				Name: "symbolMatcher",
@@ -252,12 +205,14 @@
 						Doc:   "",
 					},
 				},
-				Default: "\"Fuzzy\"",
+				Default:   "\"Fuzzy\"",
+				Status:    "advanced",
+				Hierarchy: "ui.navigation",
 			},
 			{
 				Name: "symbolStyle",
 				Type: "enum",
-				Doc:  "symbolStyle controls how symbols are qualified in symbol responses.\n\nExample Usage:\n```json5\n\"gopls\": {\n...\n  \"symbolStyle\": \"dynamic\",\n...\n}\n```\n",
+				Doc:  "symbolStyle controls how symbols are qualified in symbol responses.\n\nExample Usage:\n\n```json5\n\"gopls\": {\n...\n  \"symbolStyle\": \"dynamic\",\n...\n}\n```\n",
 				EnumValues: []EnumValue{
 					{
 						Value: "\"Dynamic\"",
@@ -272,14 +227,107 @@
 						Doc:   "`\"Package\"` is package qualified symbols i.e.\n\"pkg.Foo.Field\".\n",
 					},
 				},
-				Default: "\"Dynamic\"",
+				Default:   "\"Dynamic\"",
+				Status:    "advanced",
+				Hierarchy: "ui.navigation",
 			},
 			{
-				Name:       "directoryFilters",
-				Type:       "[]string",
-				Doc:        "directoryFilters can be used to exclude unwanted directories from the\nworkspace. By default, all directories are included. Filters are an\noperator, `+` to include and `-` to exclude, followed by a path prefix\nrelative to the workspace folder. They are evaluated in order, and\nthe last filter that applies to a path controls whether it is included.\nThe path prefix can be empty, so an initial `-` excludes everything.\n\nExamples:\nExclude node_modules: `-node_modules`\nInclude only project_a: `-` (exclude everything), `+project_a`\nInclude only project_a, but not node_modules inside it: `-`, `+project_a`, `-project_a/node_modules`\n",
+				Name:       "analyses",
+				Type:       "map[string]bool",
+				Doc:        "analyses specify analyses that the user would like to enable or disable.\nA map of the names of analysis passes that should be enabled/disabled.\nA full list of analyzers that gopls uses can be found [here](analyzers.md)\n\nExample Usage:\n\n```json5\n...\n\"analyses\": {\n  \"unreachable\": false, // Disable the unreachable analyzer.\n  \"unusedparams\": true  // Enable the unusedparams analyzer.\n}\n...\n```\n",
 				EnumValues: nil,
-				Default:    "[]",
+				Default:    "{}",
+				Status:     "",
+				Hierarchy:  "ui.diagnostic",
+			},
+			{
+				Name:       "staticcheck",
+				Type:       "bool",
+				Doc:        "staticcheck enables additional analyses from staticcheck.io.\n",
+				EnumValues: nil,
+				Default:    "false",
+				Status:     "experimental",
+				Hierarchy:  "ui.diagnostic",
+			},
+			{
+				Name: "annotations",
+				Type: "map[string]bool",
+				Doc:  "annotations specifies the various kinds of optimization diagnostics\nthat should be reported by the gc_details command.\n",
+				EnumValues: []EnumValue{
+					{
+						Value: "\"bounds\"",
+						Doc:   "`\"bounds\"` controls bounds checking diagnostics.\n",
+					},
+					{
+						Value: "\"escape\"",
+						Doc:   "`\"escape\"` controls diagnostics about escape choices.\n",
+					},
+					{
+						Value: "\"inline\"",
+						Doc:   "`\"inline\"` controls diagnostics about inlining choices.\n",
+					},
+					{
+						Value: "\"nil\"",
+						Doc:   "`\"nil\"` controls nil checks.\n",
+					},
+				},
+				Default:   "{\"bounds\":true,\"escape\":true,\"inline\":true,\"nil\":true}",
+				Status:    "experimental",
+				Hierarchy: "ui.diagnostic",
+			},
+			{
+				Name:       "experimentalDiagnosticsDelay",
+				Type:       "time.Duration",
+				Doc:        "experimentalDiagnosticsDelay controls the amount of time that gopls waits\nafter the most recent file modification before computing deep diagnostics.\nSimple diagnostics (parsing and type-checking) are always run immediately\non recently modified packages.\n\nThis option must be set to a valid duration string, for example `\"250ms\"`.\n",
+				EnumValues: nil,
+				Default:    "\"250ms\"",
+				Status:     "experimental",
+				Hierarchy:  "ui.diagnostic",
+			},
+			{
+				Name:       "codelenses",
+				Type:       "map[string]bool",
+				Doc:        "codelenses overrides the enabled/disabled state of code lenses. See the \"Code Lenses\"\nsection of settings.md for the list of supported lenses.\n\nExample Usage:\n\n```json5\n\"gopls\": {\n...\n  \"codelens\": {\n    \"generate\": false,  // Don't show the `go generate` lens.\n    \"gc_details\": true  // Show a code lens toggling the display of gc's choices.\n  }\n...\n}\n```\n",
+				EnumValues: nil,
+				Default:    "{\"gc_details\":false,\"generate\":true,\"regenerate_cgo\":true,\"tidy\":true,\"upgrade_dependency\":true,\"vendor\":true}",
+				Status:     "",
+				Hierarchy:  "ui",
+			},
+			{
+				Name:       "semanticTokens",
+				Type:       "bool",
+				Doc:        "semanticTokens controls whether the LSP server will send\nsemantic tokens to the client.\n",
+				EnumValues: nil,
+				Default:    "false",
+				Status:     "experimental",
+				Hierarchy:  "ui",
+			},
+			{
+				Name:       "local",
+				Type:       "string",
+				Doc:        "local is the equivalent of the `goimports -local` flag, which puts\nimports beginning with this string after third-party packages. It should\nbe the prefix of the import path whose imports should be grouped\nseparately.\n",
+				EnumValues: nil,
+				Default:    "\"\"",
+				Status:     "",
+				Hierarchy:  "formatting",
+			},
+			{
+				Name:       "gofumpt",
+				Type:       "bool",
+				Doc:        "gofumpt indicates if we should run gofumpt formatting.\n",
+				EnumValues: nil,
+				Default:    "false",
+				Status:     "",
+				Hierarchy:  "formatting",
+			},
+			{
+				Name:       "verboseOutput",
+				Type:       "bool",
+				Doc:        "verboseOutput enables additional debug logging.\n",
+				EnumValues: nil,
+				Default:    "false",
+				Status:     "debug",
+				Hierarchy:  "",
 			},
 		},
 	},