blob: 8f9289914ddada22c5e817f09b0effef6a3a83f0 [file] [log] [blame]
<!DOCTYPE html>
<html lang="en">
<head>
<link rel="preconnect" href="https://www.googletagmanager.com">
<script >(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src=
'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
})(window,document,'script','dataLayer','GTM-W8MVQXG');</script>
<meta charset="utf-8">
<meta name="description" content="Go is an open source programming language that makes it easy to build simple, reliable, and efficient software.">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="theme-color" content="#00add8">
<link rel="stylesheet" href="https://fonts.googleapis.com/css?family=Google+Sans:400,500,600|Work+Sans:400,500,600|Roboto:400,500,700|Open+Sans:Source+Code+Pro|Material+Icons">
<link rel="stylesheet" href="/css/styles.css">
<script>(function(w,d,s,l,i){w[l]=w[l]||[];w[l].push({'gtm.start':
new Date().getTime(),event:'gtm.js'});var f=d.getElementsByTagName(s)[0],
j=d.createElement(s),dl=l!='dataLayer'?'&l='+l:'';j.async=true;j.src=
'https://www.googletagmanager.com/gtm.js?id='+i+dl;f.parentNode.insertBefore(j,f);
})(window,document,'script','dataLayer','GTM-W8MVQXG');</script>
<script src="/js/site.js"></script>
<title>How Google&#39;s Core Data Solutions Team Uses Go - go.dev</title>
</head>
<body class="Site">
<noscript><iframe src="https://www.googletagmanager.com/ns.html?id=GTM-W8MVQXG"
height="0" width="0" style="display:none;visibility:hidden"></iframe></noscript>
<header class="Site-header js-siteHeader">
<div class="Banner">
<div class="Banner-inner">
<div class="Banner-message">Black Lives Matter</div>
<a class="Banner-action"
href="https://support.eji.org/give/153413/#!/donation/checkout"
target="_blank"
rel="noopener">
Support the Equal Justice Initiative
</a>
</div>
</div>
<div class="Header Header--dark">
<nav class="Header-nav">
<a href="https://go.dev/">
<img
class="js-headerLogo Header-logo"
src="/images/go-logo-white.svg"
alt="Go">
</a>
<div class="Header-rightContent">
<ul class="Header-menu">
<li class="Header-menuItem ">
<a href="/solutions/">Why Go</a>
</li>
<li class="Header-menuItem ">
<a href="/learn/">Getting Started</a>
</li>
<li class="Header-menuItem ">
<a href="https://pkg.go.dev">Discover Packages</a>
</li>
<li class="Header-menuItem ">
<a href="/about">About</a>
</li>
</ul>
<button class="Header-navOpen js-headerMenuButton Header-navOpen--white" aria-label="Open navigation.">
</button>
</div>
</nav>
</div>
</header>
<aside class="NavigationDrawer js-header">
<nav class="NavigationDrawer-nav">
<div class="NavigationDrawer-header">
<a href="https://go.dev/">
<img class="NavigationDrawer-logo" src="/images/go-logo-blue.svg" alt="Go.">
</a>
</div>
<ul class="NavigationDrawer-list">
<li class="NavigationDrawer-listItem NavigationDrawer-listItem--active">
<a href="/solutions/">Why Go</a>
</li>
<li class="NavigationDrawer-listItem ">
<a href="/learn/">Getting Started</a>
</li>
<li class="NavigationDrawer-listItem ">
<a href="https://pkg.go.dev">Discover Packages</a>
</li>
<li class="NavigationDrawer-listItem ">
<a href="/about">About</a>
</li>
</ul>
</nav>
</aside>
<div class="NavigationDrawer-scrim js-scrim" role="presentation"></div>
<main class="SiteContent SiteContent--default">
<div>
<div class="WhoUsesSubPage-hero--caseStudy">
<div class="WhoUsesSubPage-heroInner--caseStudy">
<div class="WhoUsesSubPage-heroContent--caseStudy">
<div class="WhoUsesSubPage-heroText--caseStudy">
<div class="BreadcrumbNav">
<ol class="BreadcrumbNav-inner">
<li class="BreadcrumbNav-li ">
<a class="BreadcrumbNav-link" href="/solutions/">
Why Go
</a>
</li>
<li class="BreadcrumbNav-li ">
<a class="BreadcrumbNav-link" href="/solutions/google/">
Google
</a>
</li>
<li class="BreadcrumbNav-li active">
<a class="BreadcrumbNav-link" href="/solutions/google/coredata">
Core Data
</a>
</li>
</ol>
</div>
<h1>How Google&#39;s Core Data Solutions Team Uses Go</h1>
<div class="Article-author">Prasanna Meda, Software Engineer, Core Data Solutions</div>
</div>
<div class="WhoUsesSubPage-heroImg">
<img src="/images/go_core_data_case_study.png" alt="Core Data">
</div>
</div>
</div>
</div>
<article class="Article Article--solutions">
<div class="CaseStudy-content">
<div class="CaseStudy-contentBody">
<p>Google&rsquo;s mission is “to organize the world&rsquo;s information and make it universally
accessible and useful.” One of the teams responsible for organizing that
information is Google’s Core Data Solutions team. The team, among other things,
maintains services to index web pages across the globe. These web indexing
services help support products like Google Search by keeping search results
updated and comprehensive, and they’re written in Go.</p>
<p>In 2015, to keep up with Google’s scale, our team needed to rewrite our indexing
stack from a single monolithic binary written in C++ to multiple components in a
microservices architecture. We decided to rewrite many indexing services in Go,
which we now use to power the majority of our architecture.</p>
<div class="BackgroundQuote">
<p class="BackgroundQuote-body">
“Go’s built-in concurrency is a natural fit because engineers on the team are
encouraged to use concurrency and parallel algorithms.”
</p>
<div class="BackgroundQuote-author">
<span>&mdash; Minjae Hwang</span>
,&nbsp;
<span class="BackgroundQuote-title">Software Engineer</span>
</div>
</div>
<p>When choosing a language, our team found that several of Go’s features made it
particularly suitable. For instance, Go’s built-in concurrency is a natural fit
because engineers on the team are encouraged to use concurrency and parallel
algorithms. Engineers have also found that “Go code is more natural,” allowing
them to spend their time focusing on business logic and analysis rather than on
managing memory and optimizing performance.</p>
<p>Writing code is much simpler when writing in Go, as it helps lessen cognitive
burden during the development process. For example, when working with C++,
sophisticated IDEs might, “show that the source code has no compile error when
there actually is one” whereas “in Go, [the code] will always compile when [the
IDE] says the code has no compile error,” said MinJae Hwang, a software engineer
on the Core Data Solutions team. Reducing small friction points along the
development process, such as shortening the cycle of fixing compile errors,
helped our team ship faster during the original rewrite, and has helped keep our
maintenance costs low.</p>
<p>“When I’m in C++ and I want to use more packages, I have to write pieces such as
headers. When I&rsquo;m writing in Go, <strong>built-in tools allow me to use packages more
easily. My development velocity is much faster,</strong>” Hwang also shared.</p>
<p>With simple language syntax and support of Go tools, several members of our team
find it much easier to write in Go code. We’ve also found that Go does a really
good job of static type checking and that certain Go fundamentals, such as the
godoc command, have helped the team build a more disciplined culture around
writing documentation.</p>
<div class="BackgroundQuote">
<p class="BackgroundQuote-body">
“&hellip;Google’s web indexing was re-architected within a year. More impressively,
most developers on the team were rewriting in Go while also learning it.”
</p>
<div class="BackgroundQuote-author">
<span>&mdash; Prasanna Meda</span>
,&nbsp;
<span class="BackgroundQuote-title">Software Engineer</span>
</div>
</div>
<p>Working on a product used so heavily around the world is no small task and our
team’s decision to use Go wasn’t a simple one, but doing so helped us move
faster. As a result, Google’s web indexing was re-architected within a year.
More impressively, most developers on the team were rewriting in Go while also
learning it.</p>
<p>In addition to the Core Data Solutions team, engineering teams across Google
have adopted Go in their development process. Read about how the
<a href="/solutions/google/chrome/">Chrome</a> and <a href="/solutions/google/firebase/">Firebase
Hosting</a> teams use Go to build fast, reliable, and
efficient software at scale.</p>
</div>
<div class="CaseStudy-contentAside">
<div class="CaseStudy-aboutBlock">
<img src="/images/logos/google.svg" class="CaseStudy-aboutBlockImg" alt="Core Data">
<h3 class="CaseStudy-aboutBlockTitle">
About Core Data
</h3>
<p class="CaseStudy-aboutBlockBody"><p>Google is a technology company whose mission is to organize the world’s
information and make it universally accessible and useful.</p>
<p>In this case study, Google’s Core Data Solutions team shares their journey
with Go, including their decision to rewrite web indexing services in Go,
taking advantage of Go’s built-in concurrency, and observing how Go helps to
improve the development process.</p>
</p>
</div>
</div>
</div>
</article>
</div>
</main>
<footer class="Site-footer">
<div class="Footer">
<div class="Container">
<div class="Footer-links">
<div class="Footer-linkColumn">
<a href="/solutions/" class="Footer-link Footer-link--primary">
Why Go
</a>
<a href="/solutions/#use-cases" class="Footer-link">
Use Cases
</a>
<a href="/solutions/#case-studies" class="Footer-link">
Case Studies
</a>
</div>
<div class="Footer-linkColumn">
<a href="/learn/" class="Footer-link Footer-link--primary">
Getting Started
</a>
<a href="https://play.golang.org" class="Footer-link">
Playground
</a>
<a href="https://tour.golang.org" class="Footer-link">
Tour
</a>
<a href="https://stackoverflow.com/questions/tagged/go?tab=Newest" class="Footer-link">
Stack Overflow
</a>
</div>
<div class="Footer-linkColumn">
<a href="https://pkg.go.dev" class="Footer-link Footer-link--primary">
Discover Packages
</a>
</div>
<div class="Footer-linkColumn">
<a href="/about" class="Footer-link Footer-link--primary">
About
</a>
<a href="https://golang.org/dl/" class="Footer-link">
Download
</a>
<a href="https://blog.golang.org" class="Footer-link">
Blog
</a>
<a href="https://github.com/golang/go/issues" class="Footer-link">
Issue Tracker
</a>
<a href="https://golang.org/doc/devel/release.html" class="Footer-link">
Release Notes
</a>
<a href="https://blog.golang.org/go-brand" class="Footer-link">
Brand Guidelines
</a>
<a href="https://golang.org/conduct" class="Footer-link">
Code of Conduct
</a>
</div>
<div class="Footer-linkColumn">
<a href="https://www.twitter.com/golang" class="Footer-link Footer-link--primary">
Connect
</a>
<a href="https://www.twitter.com/golang" class="Footer-link">
Twitter
</a>
<a href="https://github.com/golang" class="Footer-link">
GitHub
</a>
<a href="https://invite.slack.golangbridge.org/" class="Footer-link">
Slack
</a>
<a href="https://reddit.com/r/golang" class="Footer-link">
r/golang
</a>
<a href="https://www.meetup.com/pro/go" class="Footer-link">
Meetup
</a>
<a href="https://golangweekly.com/" class="Footer-link">
Golang Weekly
</a>
</div>
</div>
</div>
</div>
<div class="Footer">
<div class="Container Container--fullBleed">
<div class="Footer-bottom">
<img class="Footer-gopher" src="/images/gophers/pilot-bust.svg" alt="The Go Gopher">
<ul class="Footer-listRow">
<li class="Footer-listItem">
<a href="/copyright">Copyright</a>
</li>
<li class="Footer-listItem">
<a href="/tos">Terms of Service</a>
</li>
<li class="Footer-listItem">
<a href="http://www.google.com/intl/en/policies/privacy/"
target="_blank"
rel="noopener">
Privacy Policy
</a>
</li>
<li class="Footer-listItem">
<a
href="https://golang.org/s/discovery-feedback"
target="_blank"
rel="noopener"
>
Report an Issue
</a>
</li>
<li class="Footer-listItem">
<a
href="https://golang.org"
target="_blank"
rel="noopener"
>golang.org
</a>
</li>
</ul>
<a class="Footer-googleLogo" target="_blank" href="https://google.com" rel="noopener">
<img class="Footer-googleLogoImg" src="/images/google-white.png" alt="Google logo">
</a>
</div>
</div>
</div>
<script src="/js/carousels.js"></script>
<script src="/js/searchBox.js"></script>
<script src="/js/misc.js"></script>
<script src="/js/hats.js"></script>
</footer>
</body>
</html>