Go: From Godep To vgo, A Commentated History
"Those who don't know history are doomed to repeat it." ― Edmund Burke
There are many variations on this saying but the essential element is that it's important, and I would argue useful, to know the history of something. It can provide depth, understanding, and insight.
At the moment there are many debates going on about package management in Go. There are questions being asked, such as how should it work or whose ideas should we follow?
To give context to these ideas it's worth looking at the history of dependency management in Go. It's a story of people, discovery, differences, disconnects, and even a little drama.
In The Beginning...
In the beginning there was
go get
and the
GOPATH
.
The
GOPATH
is one workspace where all your source lives. It comes from the way Google does source control where most of their source is in a monorepo. The
GOPATH
can essentially point to a monorepo location.
go get
would pull, or update with the
-u
flag, from the tip of source control for a remote location. It was like
cp
but knew how to talk to remote source control systems.
There were no versions. It was built around how Google handles source and dependencies. Go is a Google owned and run project so a system built around their style is where this all started.
Godep
The first major community tool was
godep
. Early on it provided a way to snapshot the VCS revisions you were using in your
GOPATH
and then restore those to the
GOPATH
. This
provided a way for different applications to use different revisions of the same dependencies.
Godep did have some manual steps you had to do when switching between applications. For example, you needed to restore the revisions of dependencies for that application to the
GOPATH
. But, it would work alongside the Google workflow
so it worked.
There were some other tools lurking around about the same time but none of them gained the traction of godep.
The Vendor Directory
After awhile there were several tools trying to solve dependency management. A common theme for these projects was the desire for different applications to use different versions of the same dependency.
With that we got the
vendor
directory. Prior to this the Go toolchain would look in the
GOPATH
and
GOROOT
, where the standard library lives, for a package. With the
vendor
directory there was
a new discovery point relative to an imports current directory.
This next step was safely layered in. It worked for those at Google because they could use it or skip it and safely keep their existing workflows. It worked for the community because any dependency manager could put the dependencies in the
vendor
directories and things just worked with the toolchain.
Dependency Management Like Everyone Else
While godep was popular, some other tools came along that received a significant amount of usage. For example, there was a project I worked on named glide . The idea with Glide was to provide package management similar to how it worked in JavaScript, Python, Rust, PHP, and pretty much all of the languages with modern package managers.
Glide didn't stand alone. There were numerous package managers that had market share. Things were getting splintered.
From Many To One
How can we solve a problem of too many package managers? Form a committee, right?
A committee was formed to figure out package dependency management for Go . Andrew Gerrand, from the Go team at Google, was a member of this committee. There was a second advisory group, which I was a member of, that provided them with insight.
To support the committee a couple of other activities kicked off.
- A survey of the Go community about package management . This asked questions about Go, package management, and package managers. It even queried people on their experience with package management in other languages.
- Interviews with people at companies who used Go. The idea was to understand their needs if they were to build applications around Go.
The committee came to some conclusions on what we needed based on the information from the community and some debate. They wrote up what was needed in a specification .
dep was built to have one new tool that met the needs outlined in the spec. Some of us who worked on tools like godep and glide put our support behind this effort.
The community had come up with a solution that was a single point to collaborate on continuity solving the problem.
That solution worked with the existing
vendor
setup and could be layered in alongside the workflows
of people at Google.
The expectation was that dep, a community initiative with the support of the Go team, would become the standard manager.
But, this is not the end of the story.
Enter vgo
At the GopherCon in Denver in 2017 there was a contributor summit . It was a place where some people could get together to discuss Go the day prior to GopherCon.
At this summit there was time set aside for people to discuss dependency management and we did for some time. Around the conclusion of that time Russ Cox, the current lead of the Go team, came to the table. He made a comment that he could do better if he went off on his own and built something. That something was later announced as vgo. It is the thing he went off on his own, apart from the community, and created.
Despite community objections, vgo was recently accepted as the path forward.
