)]}' { "commit": "6f52790a20a2432ae61e0ec9852a3df823a16d40", "tree": "c41cd5b507900f53872798b8d1447d43272c641b", "parents": [ "6ea19bb668ee603d1a41d3fd5ab77e130118fd8b" ], "author": { "name": "Filippo Valsorda", "email": "filippo@golang.org", "time": "Thu Apr 02 17:10:13 2020 -0400" }, "committer": { "name": "Filippo Valsorda", "email": "filippo@golang.org", "time": "Thu May 07 19:22:19 2020 +0000" }, "message": "crypto/x509: use Security.framework without cgo for roots on macOS\n\n+----------------------------------------------------------------------+\n| Hello, if you are reading this and run macOS, please test this code: |\n| |\n| $ GO111MODULE\u003don go get golang.org/dl/gotip@latest |\n| $ gotip download |\n| $ GODEBUG\u003dx509roots\u003d1 gotip test crypto/x509 -v -run TestSystemRoots |\n+----------------------------------------------------------------------+\n\nWe currently have two code paths to extract system roots on macOS: one\nuses cgo to invoke a maze of Security.framework APIs; the other is a\nhorrible fallback that runs \"/usr/bin/security verify-cert\" on every\nroot that has custom policies to check if it\u0027s trusted for SSL.\n\nThe fallback is not only terrifying because it shells out to a binary,\nbut also because it lets in certificates that are not trusted roots but\nare signed by trusted roots, and because it applies some filters (EKUs\nand expiration) only to roots with custom policies, as the others are\nnot passed to verify-cert. The other code path, of course, requires cgo,\nso can\u0027t be used when cross-compiling and involves a large ball of C.\n\nIt\u0027s all a mess, and it broke oh-so-many times (#14514, #16532, #19436,\n #20990, #21416, #24437, #24652, #25649, #26073, #27958, #28025, #28092,\n #29497, #30471, #30672, #30763, #30889, #32891, #38215, #38365, ...).\n\nSince macOS does not have a stable syscall ABI, we already dynamically\nlink and invoke libSystem.dylib regardless of cgo availability (#17490).\n\nHow that works is that functions in package syscall (like syscall.Open)\ntake the address of assembly trampolines (like libc_open_trampoline)\nthat jump to symbols imported with cgo_import_dynamic (like libc_open),\nand pass them along with arguments to syscall.syscall (which is\nimplemented as runtime.syscall_syscall). syscall_syscall informs the\nscheduler and profiler, and then uses asmcgocall to switch to a system\nstack and invoke runtime.syscall. The latter is an assembly trampoline\nthat unpacks the Go ABI arguments passed to syscall.syscall, finally\ncalls the remote function, and puts the return value on the Go stack.\n(This last bit is the part that cgo compiles from a C wrapper.)\n\nWe can do something similar to link and invoke Security.framework!\n\nThe one difference is that runtime.syscall and friends check errors\nbased on the errno convention, which Security doesn\u0027t follow, so I added\nruntime.syscallNoErr which just skips interpreting the return value.\nWe only need a variant with six arguments because the calling convention\nis register-based, and extra arguments simply zero out some registers.\n\nThat\u0027s plumbed through as crypto/x509/internal/macOS.syscall. The rest\nof that package is a set of wrappers for Security.framework and Core\nFoundation functions, like syscall is for libSystem. In theory, as long\nas macOS respects ABI backwards compatibility (a.k.a. as long as\nbinaries built for a previous OS version keep running) this should be\nstable, as the final result is not different from what a C compiler\nwould make. (One exception might be dictionary key strings, which we\nmake our own copy of instead of using the dynamic symbol. If they change\nthe value of those strings things might break. But why would they.)\n\nFinally, I rewrote the crypto/x509 cgo logic in Go using those wrappers.\nIt works! I tried to make it match 1:1 the old logic, so that\nroot_darwin_amd64.go can be reviewed by comparing it to\nroot_cgo_darwin_amd64.go. The only difference is that we do proper error\nhandling now, and assume that if there is no error the return values are\nthere, while before we\u0027d just check for nil pointers and move on.\n\nI kept the cgo logic to help with review and testing, but we should\ndelete it once we are confident the new code works.\n\nThe nocgo logic is gone and we shall never speak of it again.\n\nFixes #32604\nFixes #19561\nFixes #38365\nAwakens Cthulhu\n\nChange-Id: Id850962bad667f71e3af594bdfebbbb1edfbcbb4\nReviewed-on: https://go-review.googlesource.com/c/go/+/227037\nReviewed-by: Katie Hockman \u003ckatie@golang.org\u003e\n", "tree_diff": [ { "type": "add", "old_id": "0000000000000000000000000000000000000000", "old_mode": 0, "old_path": "/dev/null", "new_id": "359694fabf2694eac06fe725fa67893b40d64f93", "new_mode": 33188, "new_path": "src/crypto/x509/internal/macOS/corefoundation.go" }, { "type": "add", "old_id": "0000000000000000000000000000000000000000", "old_mode": 0, "old_path": "/dev/null", "new_id": "8f6be47e4b57e32f91b05ad4cfbb0fc96c67dd6b", "new_mode": 33188, "new_path": "src/crypto/x509/internal/macOS/corefoundation.s" }, { "type": "add", "old_id": "0000000000000000000000000000000000000000", "old_mode": 0, "old_path": "/dev/null", "new_id": "64fe20639019d9cc339a586935aba3ca11d3d363", "new_mode": 33188, "new_path": "src/crypto/x509/internal/macOS/security.go" }, { "type": "add", "old_id": "0000000000000000000000000000000000000000", "old_mode": 0, "old_path": "/dev/null", "new_id": "1630c55bab256b42303836f2dea24272b79a75ed", "new_mode": 33188, "new_path": "src/crypto/x509/internal/macOS/security.s" }, { "type": "rename", "old_id": "784470bb3b2e7fc7e4a2749ac9cac08dc10fa512", "old_mode": 33188, "old_path": "src/crypto/x509/root_cgo_darwin.go", "new_id": "bec57eb8367dc2a4b4bc375f36b3f31bd448ee7a", "new_mode": 33188, "new_path": "src/crypto/x509/root_cgo_darwin_amd64.go", "score": 97 }, { "type": "delete", "old_id": "2f6a8b8d60762703679a6e3d60a6c33d5156793a", "old_mode": 33188, "old_path": "src/crypto/x509/root_darwin.go", "new_id": "0000000000000000000000000000000000000000", "new_mode": 0, "new_path": "/dev/null" }, { "type": "add", "old_id": "0000000000000000000000000000000000000000", "old_mode": 0, "old_path": "/dev/null", "new_id": "3ddd3a46f45f9a7f56f6e9385bdebcb156a4e468", "new_mode": 33188, "new_path": "src/crypto/x509/root_darwin_amd64.go" }, { "type": "modify", "old_id": "639c6ae7dec1f4996aaa001b04a7b3bfec5fd6b2", "old_mode": 33188, "old_path": "src/crypto/x509/root_darwin_arm64.go", "new_id": "2fb079ba6695a47d186fa2c9b4763cdcad6e316b", "new_mode": 33188, "new_path": "src/crypto/x509/root_darwin_arm64.go" }, { "type": "rename", "old_id": "cba950fcc941b7c1fdcacf2d1308c33ff679eefe", "old_mode": 33188, "old_path": "src/crypto/x509/root_darwin_arm_gen.go", "new_id": "e7e312e8820be05f44da2b521d00fd0deee96220", "new_mode": 33188, "new_path": "src/crypto/x509/root_darwin_arm64_gen.go", "score": 91 }, { "type": "modify", "old_id": "bd14d34b90abe36d8edd7f46c4de32c82b3696f5", "old_mode": 33188, "old_path": "src/crypto/x509/root_darwin_test.go", "new_id": "2c773b91203bd055c312b28cb9ed8684f7be8cbe", "new_mode": 33188, "new_path": "src/crypto/x509/root_darwin_test.go" }, { "type": "delete", "old_id": "2ac4666aff65b6c010fe94caefa2b485f8240f17", "old_mode": 33188, "old_path": "src/crypto/x509/root_nocgo_darwin.go", "new_id": "0000000000000000000000000000000000000000", "new_mode": 0, "new_path": "/dev/null" }, { "type": "modify", "old_id": "f466e24dce8c1dbdd5e9886326bf286ded22a5f5", "old_mode": 33188, "old_path": "src/crypto/x509/root_omit.go", "new_id": "b757ea81c20d5c89d9294b8d4f5d174e9e7b402c", "new_mode": 33188, "new_path": "src/crypto/x509/root_omit.go" }, { "type": "modify", "old_id": "d980781416b35233bd529f5596d70e32e2fd287c", "old_mode": 33188, "old_path": "src/go/build/deps_test.go", "new_id": "d3bbf087c3565c8fd96ff564b99975bf48d1dd47", "new_mode": 33188, "new_path": "src/go/build/deps_test.go" }, { "type": "modify", "old_id": "1b136f88a8e4e430cc1c39036225896c0cd6e6d0", "old_mode": 33188, "old_path": "src/runtime/sys_darwin.go", "new_id": "28c500a71005acd352959ca34f83c5ff1499a77c", "new_mode": 33188, "new_path": "src/runtime/sys_darwin.go" }, { "type": "modify", "old_id": "a45ea42e5d91c6ce547bae90aa587ad25c197071", "old_mode": 33188, "old_path": "src/runtime/sys_darwin_amd64.s", "new_id": "825852d673975969ef14a16cb43252d3c4e623bc", "new_mode": 33188, "new_path": "src/runtime/sys_darwin_amd64.s" } ] }