|
<!DOCTYPE html> |
|
<html lang="en" data-theme="auto"> |
|
<head> |
|
|
|
<link rel="preconnect" href="https://www.googletagmanager.com"> |
|
<script >(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start': |
|
new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0], |
|
j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src= |
|
'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f); |
|
})(window,document,'script','dataLayer','GTM-W8MVQXG');</script> |
|
|
|
<meta charset="utf-8"> |
|
<meta name="viewport" content="width=device-width, initial-scale=1"> |
|
<meta name="theme-color" content="#00add8"> |
|
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Material+Icons"> |
|
<link rel="stylesheet" href="/css/styles.css"> |
|
<link rel="icon" href="/images/favicon-gopher.png" sizes="any"> |
|
<link rel="apple-touch-icon" href="/images/favicon-gopher-plain.png"/> |
|
<link rel="icon" href="/images/favicon-gopher.svg" type="image/svg+xml"> |
|
<link rel="me" href="https://hachyderm.io/@golang"> |
|
|
|
<link rel="alternate" title="The Go Blog" type="application/atom+xml" href="/blog/feed.atom"> |
|
|
|
|
|
<script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start': |
|
new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0], |
|
j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src= |
|
'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f); |
|
})(window,document,'script','dataLayer','GTM-W8MVQXG');</script> |
|
|
|
<script src="/js/site.js"></script> |
|
<meta name="og:url" content="https://go.dev/blog/execution-traces-2024"> |
|
<meta name="og:title" content="More powerful Go execution traces - The Go Programming Language"> |
|
<title>More powerful Go execution traces - The Go Programming Language</title> |
|
|
|
<meta name="og:description" content="New features and improvements to execution traces from the last year."> |
|
<meta name="description" content="New features and improvements to execution traces from the last year."> |
|
|
|
<meta name="og:image" content="https://go.dev/doc/gopher/gopher5logo.jpg"> |
|
<meta name="twitter:image" content="https://go.dev/doc/gopher/runningsquare.jpg"> |
|
<meta name="twitter:card" content="summary"> |
|
<meta name="twitter:site" content="@golang"> |
|
</head> |
|
<body class="Site"> |
|
|
|
<noscript><iframe src="https://www.googletagmanager.com/ns.html?id=GTM-W8MVQXG" |
|
height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript> |
|
|
|
|
|
|
|
<header class="Site-header js-siteHeader"> |
|
<div class="Header Header--dark"> |
|
<nav class="Header-nav"> |
|
<a href="/"> |
|
<img |
|
class="js-headerLogo Header-logo" |
|
src="/images/go-logo-white.svg" |
|
alt="Go"> |
|
</a> |
|
<div class="skip-navigation-wrapper"> |
|
<a class="skip-to-content-link" aria-label="Skip to main content" href="#main-content"> Skip to Main Content </a> |
|
</div> |
|
<div class="Header-rightContent"> |
|
<ul class="Header-menu"> |
|
<li class="Header-menuItem "> |
|
<a href="#" class="js-desktop-menu-hover" aria-label=Why Go aria-describedby="dropdown-description"> |
|
Why Go <i class="material-icons" aria-hidden="true">arrow_drop_down</i> |
|
</a> |
|
<div class="screen-reader-only" id="dropdown-description" hidden> |
|
Press Enter to activate/deactivate dropdown |
|
</div> |
|
<ul class="Header-submenu js-desktop-submenu-hover" aria-label="submenu"> |
|
<li class="Header-submenuItem"> |
|
<div> |
|
<a href="/solutions/case-studies"> |
|
Case Studies |
|
|
|
</a> |
|
</div> |
|
<p>Common problems companies solve with Go</p> |
|
</li> |
|
<li class="Header-submenuItem"> |
|
<div> |
|
<a href="/solutions/use-cases"> |
|
Use Cases |
|
|
|
</a> |
|
</div> |
|
<p>Stories about how and why companies use Go</p> |
|
</li> |
|
<li class="Header-submenuItem"> |
|
<div> |
|
<a href="/security/"> |
|
Security |
|
|
|
</a> |
|
</div> |
|
<p>How Go can help keep you secure by default</p> |
|
</li> |
|
</ul> |
|
</li> |
|
<li class="Header-menuItem "> |
|
<a href="/learn/" aria-label=Learn aria-describedby="dropdown-description"> |
|
Learn |
|
</a> |
|
<div class="screen-reader-only" id="dropdown-description" hidden> |
|
Press Enter to activate/deactivate dropdown |
|
</div> |
|
</li> |
|
<li class="Header-menuItem "> |
|
<a href="#" class="js-desktop-menu-hover" aria-label=Docs aria-describedby="dropdown-description"> |
|
Docs <i class="material-icons" aria-hidden="true">arrow_drop_down</i> |
|
</a> |
|
<div class="screen-reader-only" id="dropdown-description" hidden> |
|
Press Enter to activate/deactivate dropdown |
|
</div> |
|
<ul class="Header-submenu js-desktop-submenu-hover" aria-label="submenu"> |
|
<li class="Header-submenuItem"> |
|
<div> |
|
<a href="/doc/effective_go"> |
|
Effective Go |
|
|
|
</a> |
|
</div> |
|
<p>Tips for writing clear, performant, and idiomatic Go code</p> |
|
</li> |
|
<li class="Header-submenuItem"> |
|
<div> |
|
<a href="/doc"> |
|
Go User Manual |
|
|
|
</a> |
|
</div> |
|
<p>A complete introduction to building software with Go</p> |
|
</li> |
|
<li class="Header-submenuItem"> |
|
<div> |
|
<a href="https://pkg.go.dev/std"> |
|
Standard library |
|
|
|
</a> |
|
</div> |
|
<p>Reference documentation for Go's standard library</p> |
|
</li> |
|
<li class="Header-submenuItem"> |
|
<div> |
|
<a href="/doc/devel/release"> |
|
Release Notes |
|
|
|
</a> |
|
</div> |
|
<p>Learn what's new in each Go release</p> |
|
</li> |
|
</ul> |
|
</li> |
|
<li class="Header-menuItem "> |
|
<a href="https://pkg.go.dev" aria-label=Packages aria-describedby="dropdown-description"> |
|
Packages |
|
</a> |
|
<div class="screen-reader-only" id="dropdown-description" hidden> |
|
Press Enter to activate/deactivate dropdown |
|
</div> |
|
</li> |
|
<li class="Header-menuItem "> |
|
<a href="#" class="js-desktop-menu-hover" aria-label=Community aria-describedby="dropdown-description"> |
|
Community <i class="material-icons" aria-hidden="true">arrow_drop_down</i> |
|
</a> |
|
<div class="screen-reader-only" id="dropdown-description" hidden> |
|
Press Enter to activate/deactivate dropdown |
|
</div> |
|
<ul class="Header-submenu js-desktop-submenu-hover" aria-label="submenu"> |
|
<li class="Header-submenuItem"> |
|
<div> |
|
<a href="/talks/"> |
|
Recorded Talks |
|
|
|
</a> |
|
</div> |
|
<p>Videos from prior events</p> |
|
</li> |
|
<li class="Header-submenuItem"> |
|
<div> |
|
<a href="https://www.meetup.com/pro/go"> |
|
Meetups |
|
<i class="material-icons">open_in_new</i> |
|
</a> |
|
</div> |
|
<p>Meet other local Go developers</p> |
|
</li> |
|
<li class="Header-submenuItem"> |
|
<div> |
|
<a href="/wiki/Conferences"> |
|
Conferences |
|
<i class="material-icons">open_in_new</i> |
|
</a> |
|
</div> |
|
<p>Learn and network with Go developers from around the world</p> |
|
</li> |
|
<li class="Header-submenuItem"> |
|
<div> |
|
<a href="/blog"> |
|
Go blog |
|
|
|
</a> |
|
</div> |
|
<p>The Go project's official blog.</p> |
|
</li> |
|
<li class="Header-submenuItem"> |
|
<div> |
|
<a href="/help"> |
|
Go project |
|
|
|
</a> |
|
</div> |
|
<p>Get help and stay informed from Go</p> |
|
</li> |
|
<li class="Header-submenuItem"> |
|
<div> |
|
Get connected |
|
</div> |
|
<p></p> |
|
<div class="Header-socialIcons"> |
|
|
|
<a class="Header-socialIcon" aria-label="Get connected with google-groups (Opens in new window)" href="https://groups.google.com/g/golang-nuts"><img src="/images/logos/social/google-groups.svg" /></a> |
|
<a class="Header-socialIcon" aria-label="Get connected with github (Opens in new window)" href="https://github.com/golang"><img src="/images/logos/social/github.svg" /></a> |
|
<a class="Header-socialIcon" aria-label="Get connected with twitter (Opens in new window)" href="https://twitter.com/golang"><img src="/images/logos/social/twitter.svg" /></a> |
|
<a class="Header-socialIcon" aria-label="Get connected with reddit (Opens in new window)" href="https://www.reddit.com/r/golang/"><img src="/images/logos/social/reddit.svg" /></a> |
|
<a class="Header-socialIcon" aria-label="Get connected with slack (Opens in new window)" href="https://invite.slack.golangbridge.org/"><img src="/images/logos/social/slack.svg" /></a> |
|
<a class="Header-socialIcon" aria-label="Get connected with stack-overflow (Opens in new window)" href="https://stackoverflow.com/tags/go"><img src="/images/logos/social/stack-overflow.svg" /></a> |
|
</div> |
|
</li> |
|
</ul> |
|
</li> |
|
</ul> |
|
<button class="Header-navOpen js-headerMenuButton Header-navOpen--white" aria-label="Open navigation."> |
|
</button> |
|
</div> |
|
</nav> |
|
|
|
</div> |
|
</header> |
|
<aside class="NavigationDrawer js-header"> |
|
<nav class="NavigationDrawer-nav"> |
|
<div class="NavigationDrawer-header"> |
|
<a href="/"> |
|
<img class="NavigationDrawer-logo" src="/images/go-logo-blue.svg" alt="Go."> |
|
</a> |
|
</div> |
|
<ul class="NavigationDrawer-list"> |
|
|
|
<li class="NavigationDrawer-listItem js-mobile-subnav-trigger NavigationDrawer-hasSubnav"> |
|
<a href="#"><span>Why Go</span> <i class="material-icons">navigate_next</i></a> |
|
|
|
<div class="NavigationDrawer NavigationDrawer-submenuItem"> |
|
<nav class="NavigationDrawer-nav"> |
|
<div class="NavigationDrawer-header"> |
|
<a href="#"><i class="material-icons">navigate_before</i>Why Go</a> |
|
</div> |
|
<ul class="NavigationDrawer-list"> |
|
<li class="NavigationDrawer-listItem"> |
|
<a href="/solutions/case-studies"> |
|
Case Studies |
|
|
|
</a> |
|
|
|
</li> |
|
<li class="NavigationDrawer-listItem"> |
|
<a href="/solutions/use-cases"> |
|
Use Cases |
|
|
|
</a> |
|
|
|
</li> |
|
<li class="NavigationDrawer-listItem"> |
|
<a href="/security/"> |
|
Security |
|
|
|
</a> |
|
|
|
</li> |
|
</ul> |
|
</div> |
|
</div> |
|
</li> |
|
|
|
|
|
|
|
<li class="NavigationDrawer-listItem "> |
|
<a href="/learn/">Learn</a> |
|
</li> |
|
|
|
|
|
<li class="NavigationDrawer-listItem js-mobile-subnav-trigger NavigationDrawer-hasSubnav"> |
|
<a href="#"><span>Docs</span> <i class="material-icons">navigate_next</i></a> |
|
|
|
<div class="NavigationDrawer NavigationDrawer-submenuItem"> |
|
<nav class="NavigationDrawer-nav"> |
|
<div class="NavigationDrawer-header"> |
|
<a href="#"><i class="material-icons">navigate_before</i>Docs</a> |
|
</div> |
|
<ul class="NavigationDrawer-list"> |
|
<li class="NavigationDrawer-listItem"> |
|
<a href="/doc/effective_go"> |
|
Effective Go |
|
|
|
</a> |
|
|
|
</li> |
|
<li class="NavigationDrawer-listItem"> |
|
<a href="/doc"> |
|
Go User Manual |
|
|
|
</a> |
|
|
|
</li> |
|
<li class="NavigationDrawer-listItem"> |
|
<a href="https://pkg.go.dev/std"> |
|
Standard library |
|
|
|
</a> |
|
|
|
</li> |
|
<li class="NavigationDrawer-listItem"> |
|
<a href="/doc/devel/release"> |
|
Release Notes |
|
|
|
</a> |
|
|
|
</li> |
|
</ul> |
|
</div> |
|
</div> |
|
</li> |
|
|
|
|
|
|
|
<li class="NavigationDrawer-listItem "> |
|
<a href="https://pkg.go.dev">Packages</a> |
|
</li> |
|
|
|
|
|
<li class="NavigationDrawer-listItem js-mobile-subnav-trigger NavigationDrawer-hasSubnav"> |
|
<a href="#"><span>Community</span> <i class="material-icons">navigate_next</i></a> |
|
|
|
<div class="NavigationDrawer NavigationDrawer-submenuItem"> |
|
<nav class="NavigationDrawer-nav"> |
|
<div class="NavigationDrawer-header"> |
|
<a href="#"><i class="material-icons">navigate_before</i>Community</a> |
|
</div> |
|
<ul class="NavigationDrawer-list"> |
|
<li class="NavigationDrawer-listItem"> |
|
<a href="/talks/"> |
|
Recorded Talks |
|
|
|
</a> |
|
|
|
</li> |
|
<li class="NavigationDrawer-listItem"> |
|
<a href="https://www.meetup.com/pro/go"> |
|
Meetups |
|
<i class="material-icons">open_in_new</i> |
|
</a> |
|
|
|
</li> |
|
<li class="NavigationDrawer-listItem"> |
|
<a href="/wiki/Conferences"> |
|
Conferences |
|
<i class="material-icons">open_in_new</i> |
|
</a> |
|
|
|
</li> |
|
<li class="NavigationDrawer-listItem"> |
|
<a href="/blog"> |
|
Go blog |
|
|
|
</a> |
|
|
|
</li> |
|
<li class="NavigationDrawer-listItem"> |
|
<a href="/help"> |
|
Go project |
|
|
|
</a> |
|
|
|
</li> |
|
<li class="NavigationDrawer-listItem"> |
|
<div>Get connected</div> |
|
<div class="Header-socialIcons"> |
|
|
|
<a class="Header-socialIcon" href="https://groups.google.com/g/golang-nuts"><img src="/images/logos/social/google-groups.svg" /></a> |
|
<a class="Header-socialIcon" href="https://github.com/golang"><img src="/images/logos/social/github.svg" /></a> |
|
<a class="Header-socialIcon" href="https://twitter.com/golang"><img src="/images/logos/social/twitter.svg" /></a> |
|
<a class="Header-socialIcon" href="https://www.reddit.com/r/golang/"><img src="/images/logos/social/reddit.svg" /></a> |
|
<a class="Header-socialIcon" href="https://invite.slack.golangbridge.org/"><img src="/images/logos/social/slack.svg" /></a> |
|
<a class="Header-socialIcon" href="https://stackoverflow.com/tags/go"><img src="/images/logos/social/stack-overflow.svg" /></a> |
|
</div> |
|
</li> |
|
</ul> |
|
</div> |
|
</div> |
|
</li> |
|
|
|
|
|
</ul> |
|
</nav> |
|
</aside> |
|
<div class="NavigationDrawer-scrim js-scrim" role="presentation"></div> |
|
<main class="SiteContent SiteContent--default" id="main-content"> |
|
|
|
<div id="blog"><div id="content"> |
|
<div id="content"> |
|
|
|
<div class="Article" data-slug="/blog/execution-traces-2024"> |
|
|
|
<h1 class="small"><a href="/blog/">The Go Blog</a></h1> |
|
|
|
|
|
<h1>More powerful Go execution traces</h1> |
|
|
|
<p class="author"> |
|
Michael Knyszek<br> |
|
14 March 2024 |
|
</p> |
|
|
|
<p>The <a href="/pkg/runtime/trace">runtime/trace</a> package contains a powerful tool for understanding and |
|
troubleshooting Go programs. |
|
The functionality within allows one to produce a trace of each goroutine’s execution over some |
|
time period. |
|
With the <a href="/pkg/cmd/trace"><code>go tool trace</code> command</a> (or the excellent open source |
|
<a href="https://gotraceui.dev/" rel="noreferrer" target="_blank">gotraceui tool</a>), one may then visualize and explore the data within these |
|
traces.</p> |
|
<p>The magic of a trace is that it can easily reveal things about a program that are hard to see in |
|
other ways. |
|
For example, a concurrency bottleneck where lots of goroutines block on the same channel might be |
|
quite difficult to see in a CPU profile, because there’s no execution to sample. |
|
But in an execution trace, the <em>lack</em> of execution will show up with amazing clarity, and the stack |
|
traces of blocked goroutines will quickly point at the culprit.</p> |
|
<div class="image"> |
|
<img src="execution-traces-2024/gotooltrace.png" alt=""> |
|
</div> |
|
<p>Go developers are even able to instrument their own programs with <a href="/pkg/runtime/trace#Task">tasks</a>, |
|
<a href="/pkg/runtime/trace#WithRegion">regions</a>, and <a href="/pkg/runtime/trace#Log">logs</a> that |
|
they can use to correlate their higher-level concerns with lower-level execution details.</p> |
|
<h2 id="issues">Issues</h2> |
|
<p>Unfortunately, the wealth of information in execution traces can often be out of reach. |
|
Four big issues with traces have historically gotten in the way.</p> |
|
<ul> |
|
<li>Traces had high overheads.</li> |
|
<li>Traces didn’t scale well, and could become too big to analyze.</li> |
|
<li>It was often unclear when to start tracing to capture a specific bad behavior.</li> |
|
<li>Only the most adventurous gophers could programmatically analyze traces, given the lack of a |
|
public package for parsing and interpreting execution traces.</li> |
|
</ul> |
|
<p>If you’ve used traces in the last few years, you’ve likely been frustrated by one or more of these |
|
problems. |
|
But we’re excited to share that over the last two Go releases we’ve made big progress in all four |
|
of these areas.</p> |
|
<h2 id="low-overhead-tracing">Low-overhead tracing</h2> |
|
<p>Prior to Go 1.21, the run-time overhead of tracing was somewhere between 10–20% CPU for many |
|
applications, which limits tracing to situational usage, rather than continuous usage like CPU |
|
profiling. |
|
It turned out that much of the cost of tracing came down to tracebacks. |
|
Many events produced by the runtime have stack traces attached, which are invaluable to actually |
|
identifying what goroutines were doing at key moments in their execution.</p> |
|
<p>Thanks to work by Felix Geisendörfer and Nick Ripley on optimizing the efficiency of tracebacks, |
|
the run-time CPU overhead of execution traces has been cut dramatically, down to 1–2% for many |
|
applications. |
|
You can read more about the work done here in <a href="https://blog.felixge.de/reducing-gos-execution-tracer-overhead-with-frame-pointer-unwinding/" rel="noreferrer" target="_blank">Felix’s great blog |
|
post</a> |
|
on the topic.</p> |
|
<h2 id="scalable-traces">Scalable traces</h2> |
|
<p>The trace format and its events were designed around relatively efficient emission, but required |
|
tooling to parse and keep around the state of the entirety of a trace. |
|
A few hundred MiB trace could require several GiB of RAM to analyze!</p> |
|
<p>This issue is unfortunately fundamental to how traces are generated. |
|
To keep run-time overheads low, all events are written to the equivalent of thread-local buffers. |
|
But this means events appear out of their true order, and the burden is placed on the trace |
|
tooling to figure out what really happened.</p> |
|
<p>The key insight to making traces scale while keeping overheads low was to occasionally split the |
|
trace being generated. |
|
Each split point would behave a bit like simultaneously disabling and reenabling tracing in one |
|
go. |
|
All the trace data so far would represent a complete and self-contained trace, while the new trace |
|
data would seamlessly pick up from where it left off.</p> |
|
<p>As you might imagine, fixing this required <a href="/issue/60773">rethinking and rewriting a lot of the foundation of |
|
the trace implementation</a> in the runtime. |
|
We’re happy to say that the work landed in Go 1.22 and is now generally available. |
|
<a href="/doc/go1.22#runtime/trace">A lot of nice improvements</a> came with the rewrite, including some |
|
improvements to the <a href="/doc/go1.22#trace"><code>go tool trace</code> command</a> as well. |
|
The gritty details are all in the <a href="https://github.com/golang/proposal/blob/master/design/60773-execution-tracer-overhaul.md" rel="noreferrer" target="_blank">design |
|
document</a>, |
|
if you’re curious.</p> |
|
<p>(Note: <code>go tool trace</code> still loads the full trace into memory, but <a href="/issue/65315">removing this |
|
limitation</a> for traces produced by Go 1.22+ programs is now feasible.)</p> |
|
<h2 id="flight-recording">Flight recording</h2> |
|
<p>Suppose you work on a web service and an RPC took a very long time. |
|
You couldn’t start tracing at the point you knew the RPC was already taking a while, because the |
|
root cause of the slow request already happened and wasn’t recorded.</p> |
|
<p>There’s a technique that can help with this called flight recording, which you may already be |
|
familiar with from other programming environments. |
|
The insight with flight recording is to have tracing on continuously and always keep the most |
|
recent trace data around, just in case. |
|
Then, once something interesting happens, the program can just write out whatever it has!</p> |
|
<p>Before traces could be split, this was pretty much a non-starter. |
|
But because continuous tracing is now viable thanks to low overheads, and the fact that the runtime |
|
can now split traces any time it needs, it turns out it was straightforward to implement flight |
|
recording.</p> |
|
<p>As a result, we’re happy to announce a flight recorder experiment, available in the |
|
<a href="/pkg/golang.org/x/exp/trace#FlightRecorder">golang.org/x/exp/trace package</a>.</p> |
|
<p>Please try it out! |
|
Below is an example that sets up flight recording to capture a long HTTP request to get you started.</p> |
|
<div class="code"> |
|
<pre> <span class="comment">// Set up the flight recorder.</span> |
|
fr := trace.NewFlightRecorder() |
|
fr.Start() |
|
|
|
<span class="comment">// Set up and run an HTTP server.</span> |
|
var once sync.Once |
|
http.HandleFunc("/my-endpoint", func(w http.ResponseWriter, r *http.Request) { |
|
start := time.Now() |
|
|
|
<span class="comment">// Do the work...</span> |
|
doWork(w, r) |
|
|
|
<span class="comment">// We saw a long request. Take a snapshot!</span> |
|
if time.Since(start) > 300*time.Millisecond { |
|
<span class="comment">// Do it only once for simplicity, but you can take more than one.</span> |
|
once.Do(func() { |
|
<span class="comment">// Grab the snapshot.</span> |
|
var b bytes.Buffer |
|
_, err = fr.WriteTo(&b) |
|
if err != nil { |
|
log.Print(err) |
|
return |
|
} |
|
<span class="comment">// Write it to a file.</span> |
|
if err := os.WriteFile("trace.out", b.Bytes(), 0o755); err != nil { |
|
log.Print(err) |
|
return |
|
} |
|
}) |
|
} |
|
}) |
|
log.Fatal(http.ListenAndServe(":8080", nil)) |
|
</pre> |
|
</div> |
|
<p>If you have any feedback, positive or negative, please share it to the <a href="/issue/63185">proposal |
|
issue</a>!</p> |
|
<h2 id="trace-reader-api">Trace reader API</h2> |
|
<p>Along with the trace implementation rewrite came an effort to clean up the other trace internals, |
|
like <code>go tool trace</code>. |
|
This spawned an attempt to create a trace reader API that was good enough to share and that could |
|
make traces more accessible.</p> |
|
<p>Just like the flight recorder, we’re happy to announce that we also have an experimental trace reader |
|
API that we’d like to share. |
|
It’s available in the <a href="/pkg/golang.org/x/exp/trace#Reader">same package as the flight recorder, |
|
golang.org/x/exp/trace</a>.</p> |
|
<p>We think it’s good enough to start building things on top of, so please try it out! |
|
Below is an example that measures the proportion of goroutine block events that blocked to wait on |
|
the network.</p> |
|
<div class="code"> |
|
<pre> <span class="comment">// Start reading from STDIN.</span> |
|
r, err := trace.NewReader(os.Stdin) |
|
if err != nil { |
|
log.Fatal(err) |
|
} |
|
|
|
var blocked int |
|
var blockedOnNetwork int |
|
for { |
|
<span class="comment">// Read the event.</span> |
|
ev, err := r.ReadEvent() |
|
if err == io.EOF { |
|
break |
|
} else if err != nil { |
|
log.Fatal(err) |
|
} |
|
|
|
<span class="comment">// Process it.</span> |
|
if ev.Kind() == trace.EventStateTransition { |
|
st := ev.StateTransition() |
|
if st.Resource.Kind == trace.ResourceGoroutine { |
|
from, to := st.Goroutine() |
|
|
|
<span class="comment">// Look for goroutines blocking, and count them.</span> |
|
if from.Executing() && to == trace.GoWaiting { |
|
blocked++ |
|
if strings.Contains(st.Reason, "network") { |
|
blockedOnNetwork++ |
|
} |
|
} |
|
} |
|
} |
|
} |
|
<span class="comment">// Print what we found.</span> |
|
p := 100 * float64(blockedOnNetwork) / float64(blocked) |
|
fmt.Printf("%2.3f%% instances of goroutines blocking were to block on the network\n", p) |
|
</pre> |
|
</div> |
|
<p>And just like the flight recorder, there’s a <a href="/issue/62627">proposal issue</a> that would |
|
be a great place to leave feedback!</p> |
|
<p>We’d like to quickly call out Dominik Honnef as someone who tried it out early, provided great |
|
feedback, and has contributed support for older trace versions to the API.</p> |
|
<h2 id="thank-you">Thank you!</h2> |
|
<p>This work was completed, in no small part, thanks to the help of the those in the <a href="/issue/57175">diagnostics |
|
working group</a>, started over a year ago as a collaboration between stakeholders from |
|
across the Go community, and open to the public.</p> |
|
<p>We’d like to take a moment to thank those community members who have attended the diagnostic |
|
meetings regularly over the last year: Felix Geisendörfer, Nick Ripley, Rhys Hiltner, Dominik |
|
Honnef, Bryan Boreham, thepudds.</p> |
|
<p>The discussions, feedback, and work you all put in have been instrumental to getting us to where we |
|
are today. |
|
Thank you!</p> |
|
|
|
</div> |
|
|
|
|
|
<div class="Article prevnext"> |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<p> |
|
|
|
|
|
<b>Next article: </b><a href="/blog/survey2024-h1-results">Go Developer Survey 2024 H1 Results</a><br> |
|
|
|
|
|
|
|
|
|
<b>Previous article: </b><a href="/blog/generic-slice-functions">Robust generic functions on slices</a><br> |
|
|
|
|
|
<b><a href="/blog/all">Blog Index</a></b> |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
</div> |
|
|
|
|
|
</div> |
|
</div> |
|
|
|
<script src="/js/jquery.js"></script> |
|
<script src="/js/playground.js"></script> |
|
<script src="/js/play.js"></script> |
|
<script src="/js/godocs.js"></script> |
|
|
|
</main> |
|
<footer class="Site-footer"> |
|
<div class="Footer"> |
|
<div class="Container"> |
|
<div class="Footer-links"> |
|
<div class="Footer-linkColumn"> |
|
<a href="/solutions/" class="Footer-link Footer-link--primary" aria-describedby="footer-description"> |
|
Why Go |
|
</a> |
|
<a href="/solutions/use-cases" class="Footer-link" aria-describedby="footer-description"> |
|
Use Cases |
|
</a> |
|
<a href="/solutions/case-studies" class="Footer-link" aria-describedby="footer-description"> |
|
Case Studies |
|
</a> |
|
</div> |
|
<div class="Footer-linkColumn"> |
|
<a href="/learn/" class="Footer-link Footer-link--primary" aria-describedby="footer-description"> |
|
Get Started |
|
</a> |
|
<a href="/play" class="Footer-link" aria-describedby="footer-description"> |
|
Playground |
|
</a> |
|
<a href="/tour/" class="Footer-link" aria-describedby="footer-description"> |
|
Tour |
|
</a> |
|
<a href="https://stackoverflow.com/questions/tagged/go?tab=Newest" class="Footer-link" aria-describedby="footer-description"> |
|
Stack Overflow |
|
</a> |
|
<a href="/help/" class="Footer-link" aria-describedby="footer-description"> |
|
Help |
|
</a> |
|
</div> |
|
<div class="Footer-linkColumn"> |
|
<a href="https://pkg.go.dev" class="Footer-link Footer-link--primary" aria-describedby="footer-description"> |
|
Packages |
|
</a> |
|
<a href="/pkg/" class="Footer-link" aria-describedby="footer-description"> |
|
Standard Library |
|
</a> |
|
<a href="https://pkg.go.dev/about" class="Footer-link" aria-describedby="footer-description"> |
|
About Go Packages |
|
</a> |
|
</div> |
|
<div class="Footer-linkColumn"> |
|
<a href="/project" class="Footer-link Footer-link--primary" aria-describedby="footer-description"> |
|
About |
|
</a> |
|
<a href="/dl/" class="Footer-link" aria-describedby="footer-description"> |
|
Download |
|
</a> |
|
<a href="/blog/" class="Footer-link" aria-describedby="footer-description"> |
|
Blog |
|
</a> |
|
<a href="https://github.com/golang/go/issues" class="Footer-link" aria-describedby="footer-description"> |
|
Issue Tracker |
|
</a> |
|
<a href="/doc/devel/release" class="Footer-link" aria-describedby="footer-description"> |
|
Release Notes |
|
</a> |
|
<a href="/brand" class="Footer-link" aria-describedby="footer-description"> |
|
Brand Guidelines |
|
</a> |
|
<a href="/conduct" class="Footer-link" aria-describedby="footer-description"> |
|
Code of Conduct |
|
</a> |
|
</div> |
|
<div class="Footer-linkColumn"> |
|
<a href="https://www.twitter.com/golang" class="Footer-link Footer-link--primary" aria-describedby="footer-description"> |
|
Connect |
|
</a> |
|
<a href="https://www.twitter.com/golang" class="Footer-link" aria-describedby="footer-description"> |
|
Twitter |
|
</a> |
|
<a href="https://github.com/golang" class="Footer-link" aria-describedby="footer-description"> |
|
GitHub |
|
</a> |
|
<a href="https://invite.slack.golangbridge.org/" class="Footer-link" aria-describedby="footer-description"> |
|
Slack |
|
</a> |
|
<a href="https://reddit.com/r/golang" class="Footer-link" aria-describedby="footer-description"> |
|
r/golang |
|
</a> |
|
<a href="https://www.meetup.com/pro/go" class="Footer-link" aria-describedby="footer-description"> |
|
Meetup |
|
</a> |
|
<a href="https://golangweekly.com/" class="Footer-link" aria-describedby="footer-description"> |
|
Golang Weekly |
|
</a> |
|
</div> |
|
</div> |
|
</div> |
|
</div> |
|
<div class="screen-reader-only" id="footer-description" hidden> |
|
Opens in new window. |
|
</div> |
|
<div class="Footer"> |
|
<div class="Container Container--fullBleed"> |
|
<div class="Footer-bottom"> |
|
<img class="Footer-gopher" src="/images/gophers/pilot-bust.svg" alt="The Go Gopher"> |
|
<ul class="Footer-listRow"> |
|
<li class="Footer-listItem"> |
|
<a href="/copyright" aria-describedby="footer-description">Copyright</a> |
|
</li> |
|
<li class="Footer-listItem"> |
|
<a href="/tos" aria-describedby="footer-description">Terms of Service</a> |
|
</li> |
|
<li class="Footer-listItem"> |
|
<a href="http://www.google.com/intl/en/policies/privacy/" aria-describedby="footer-description" |
|
target="_blank" |
|
rel="noopener"> |
|
Privacy Policy |
|
</a> |
|
</li> |
|
<li class="Footer-listItem"> |
|
<a |
|
href="/s/website-issue" aria-describedby="footer-description" |
|
target="_blank" |
|
rel="noopener" |
|
> |
|
Report an Issue |
|
</a> |
|
</li> |
|
<li class="Footer-listItem go-Footer-listItem"> |
|
<button class="go-Button go-Button--text go-Footer-toggleTheme js-toggleTheme" aria-label="Toggle theme"> |
|
<img |
|
data-value="auto" |
|
class="go-Icon go-Icon--inverted" |
|
height="24" |
|
width="24" |
|
src="/images/icons/brightness_6_gm_grey_24dp.svg" |
|
alt="System theme"> |
|
<img |
|
data-value="dark" |
|
class="go-Icon go-Icon--inverted" |
|
height="24" |
|
width="24" |
|
src="/images/icons/brightness_2_gm_grey_24dp.svg" |
|
alt="Dark theme"> |
|
<img |
|
data-value="light" |
|
class="go-Icon go-Icon--inverted" |
|
height="24" |
|
width="24" |
|
src="/images/icons/light_mode_gm_grey_24dp.svg" |
|
alt="Light theme"> |
|
</button> |
|
</li> |
|
</ul> |
|
<a class="Footer-googleLogo" target="_blank" href="https://google.com" rel="noopener"> |
|
<img class="Footer-googleLogoImg" src="/images/google-white.png" alt="Google logo"> |
|
</a> |
|
</div> |
|
</div> |
|
</div> |
|
<script src="/js/jquery.js"></script> |
|
<script src="/js/carousels.js"></script> |
|
<script src="/js/searchBox.js"></script> |
|
<script src="/js/misc.js"></script> |
|
<script src="/js/hats.js"></script> |
|
<script src="/js/playground.js"></script> |
|
<script src="/js/godocs.js"></script> |
|
<script async src="/js/copypaste.js"></script> |
|
</footer> |
|
<section class="Cookie-notice js-cookieNotice"> |
|
<div>go.dev uses cookies from Google to deliver and enhance the quality of its services and to |
|
analyze traffic. <a target=_blank href="https://policies.google.com/technologies/cookies">Learn more.</a></div> |
|
<div><button class="go-Button">Okay</button></div> |
|
</section> |
|
</body> |
|
</html> |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|