Document how to use dep with Docker #1094

stephenafamo · 2017-08-30T16:52:09Z

What does this do / why do we need it?

Documents how to use dep with Docker.

What should your reviewer look out for in this PR?

Changes to the FAQ. Whether the documentation properly explains how it can be used.

Do you need help or clarification on anything?

I followed mostly what was said by @ebati on this pull request.
I'd like to know if there is any other use case I should document.

Which issue(s) does this PR fix?

fixes #974.

ebati · 2017-08-30T17:48:53Z

dep ensure -vendor-only is not available in the released version 😞. It would be great to have a new release.

ebati · 2017-08-30T17:53:15Z

docs/FAQ.md

+
+    FROM golang:1.8 AS builder
+
+    RUN apt-get update && apt-get install -y unzip --no-install-recommends && \


New releases will be available as bare binary (not in zip) so there is no need to install unzip. sha256 sum will also be available with binary, so no need to hardcode hash below. Most probably downloaded binary will need execute permission chmod +x.

carolynvs

I can't wait to be able to link people to this FAQ! 💖

carolynvs · 2017-08-30T22:23:44Z

docs/FAQ.md

@@ -26,6 +26,7 @@ Summarize the question and quote the reply, linking back to the original comment
 * [How does `dep` handle symbolic links?](#how-does-dep-handle-symbolic-links)
 * [Does `dep` support relative imports?](#does-dep-support-relative-imports)
 * [How do I make `dep` resolve dependencies from my `GOPATH`?](#how-do-i-make-dep-resolve-dependencies-from-my-gopath)
+* [How do I use `dep` with Docker?](#how-do-i-make-dep-resolve-dependencies-from-my-gopath)


The link should point to #how-do-i-use-dep-with-docker.

carolynvs · 2017-08-30T22:33:07Z

docs/FAQ.md

+
+    COPY Gopkg.toml Gopkg.lock ./
+
+    RUN dep ensure -vendor-only


Let's add a comment that explains that at this point, vendor/ has been cached.

Added more comments to explain

carolynvs · 2017-08-30T22:36:43Z

docs/FAQ.md

+
+    RUN dep ensure -vendor-only
+    COPY . .
+    RUN CGO_ENABLED=0 GOOS=linux go build -ldflags "-s -w" -a -installsuffix cgo -o *** 


Let's add a comment with a link to more info on what black magic is being used, specifically ldflags, -a, and installstuffix cgo. This has a mix of recommendations floating around on the net and it's not clear what is strictly needed to run in docker, for example why did we omit the netgo workaround as well?

I'm not sure what the best reference is, but https://blog.docker.com/2016/09/docker-golang/, at least explains some of the concerns of running a go binary in a container.

I generally use these flags, but if a project needs cgo support this will not work.

CGO_ENABLE=0 disables cgo so this will automagically use netgo 😉.

Build options related to the line are as follows:

-a force rebuilding of packages that are already up-to-date. -installsuffix suffix a suffix to use in the name of the package installation directory, in order to keep output separate from default builds. If using the -race flag, the install suffix is automatically set to race or, if set explicitly, has _race appended to it. Likewise for the -msan flag. Using a -buildmode option that requires non-default compile flags has a similar effect. -ldflags 'flag list' arguments to pass on each go tool link invocation.

Since we are building go only version(CGO_ENABLED=0), we also need to build packages that we depend with cgo disabled, so -a (rebuild even if up to date) and to not mix with previously build version of the same package installsuffix cgo (use cgo suffix in installation directory).

Linker options are used to make binary small by striping unnecessary symbols.

-s Omit the symbol table and debug information. -w Omit the DWARF symbol table.

On the other hand, this FAQ might not be place to explain these options. It might be better to use bare go build and using alpine based image for both stages.

If we are going to include all those flags, quick comments or links to explain the magic are required IMO. If there is a simpler example that only demonstrates the dep aspect, then that would be preferable.

Decided to only use the -o flag, and I added a short comment why.

Add more comments to the sample docker file. Remove most flags in the docker build command to prevent confusion.

stephenafamo · 2017-08-31T18:11:16Z

Made some changes, please review again @carolynvs

carolynvs

Let's move this under "Best Practices" since the behavior section intended to explain why dep does this crazy things it does. Everything else is 💯.

Once you have that fixed, I will merge after v0.3.1 is released.

Move the section on how to use dep with docker from under the Behaviour section to under Best practices section

stephenafamo · 2017-08-31T22:36:17Z

Moved @carolynvs

ibrasho · 2017-09-01T07:59:42Z

docs/FAQ.md

+
+## How do I use `dep` with Docker?
+
+`dep ensure -vendor-only` creates the vendor folder from a valid `Gopkg.toml` and `Gopkg.lock` without checking for Go code.


Actually, dep ensure -vendor-only care about Gopkg.lock only. But it still requires Gopkg.toml to exists. 😁

Yeah that threw me off too for a sec but the FAQ is less about "how things are exactly implemented" and more high level, "you need both a toml and lock in order for this to work" type stuff. 😀

I vote for leaving it as-is for now since both files are required.

ibrasho

LGTM.

Should be merged as soon as we have a new release. 😁

sdboyer · 2017-09-10T15:10:11Z

@carolynvs i think your review condition has been satisfied - can you approve?

i'm hoping to merge the case sensitivity stuff tonight.

ebati · 2017-09-10T18:05:32Z

docs/FAQ.md

+
+Sample dockerfile:
+
+    FROM golang:1.8 AS builder


It is better to update base image to golang:1.9-alpine. (alpine since other stage use alpine and c library may cause error.)

I would prefer that the end image use scratch instead.

If you need to use Go in a container you should know how to use the correct image that satisfies your dependencies.

How to run static go binaries in Docker isn't the responsibility of the dep FAQ. The question that we want to answer is only the part about caching the results of dep ensure. If this is going to trigger a "static binary in a docker container" bikeshed, or if we are worried that we have to represent a perfect best practice example in our FAQ, then I vote we rip this out and only show an example up to running dep ensure.

I agree. Removing the parts after dep ensure is a great idea.

I agree too. Going to go ahead and do that.

carolynvs · 2017-09-11T04:21:51Z

@sdboyer This is good to go as soon as you cut a new release.

dpdornseifer · 2017-09-12T01:50:51Z

docs/FAQ.md

+
+    FROM golang:1.8 AS builder
+
+    RUN curl -fsSL -o /usr/local/bin/dep https://github.com/golang/dep/releases/download/vX.X.X/dep-linux-amd64 && chmod +x /usr/local/bin/dep


Hi, just came over the docker file issue. Might wanna add

RUN curl -fsSL -o /tmp/dep.zip \ https://github.com/golang/dep/releases/download/v0.3.0/dep-linux-amd64.zip \ && unzip /tmp/dep.zip -d /usr/local/bin \ && chmod +x /usr/local/bin/dep

This PR is waiting on our new release scheme, which won't use zipped binaries on our GitHub releases.

Removes everything after dep ensure -vendor-only This is to avoid oversimplication or ignoring best practices

carolynvs · 2017-09-19T16:48:54Z

v0.3.1 has been cut so this is ready go

Document how to use dep with Docker

2afc7f7

stephenafamo requested a review from sdboyer as a code owner August 30, 2017 16:52

googlebot added the cla: yes label Aug 30, 2017

ebati reviewed Aug 30, 2017

View reviewed changes

carolynvs suggested changes Aug 30, 2017

View reviewed changes

stephenafamo added 2 commits August 31, 2017 18:09

Merge branch 'master' of github.com:golang/dep

4a31058

Add more explanation to FAQ for docker use

b42b64d

Add more comments to the sample docker file. Remove most flags in the docker build command to prevent confusion.

carolynvs suggested changes Aug 31, 2017

View reviewed changes

Move docker FAQ under best practices

5202ee0

Move the section on how to use dep with docker from under the Behaviour section to under Best practices section

ibrasho reviewed Sep 1, 2017

View reviewed changes

ibrasho approved these changes Sep 9, 2017

View reviewed changes

ibrasho added the docs label Sep 9, 2017

ebati reviewed Sep 10, 2017

View reviewed changes

carolynvs approved these changes Sep 11, 2017

View reviewed changes

dpdornseifer reviewed Sep 12, 2017

View reviewed changes

stephenafamo added 3 commits September 12, 2017 17:16

Merge branch 'master' of github.com:golang/dep

52ebe51

Remove extras in sample dockerfile

394c3b6

Removes everything after dep ensure -vendor-only This is to avoid oversimplication or ignoring best practices

Merge branch 'master' of github.com:golang/dep

505c5c8

carolynvs merged commit 0076b93 into golang:master Sep 19, 2017

zknill pushed a commit to zknill/dep that referenced this pull request Oct 6, 2017

Document how to use dep with Docker (golang#1094)

33dd928

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Document how to use dep with Docker #1094

Document how to use dep with Docker #1094

stephenafamo commented Aug 30, 2017

ebati commented Aug 30, 2017

ebati Aug 30, 2017

carolynvs left a comment

carolynvs Aug 30, 2017

stephenafamo Aug 31, 2017

carolynvs Aug 30, 2017

stephenafamo Aug 31, 2017

carolynvs Aug 30, 2017

ebati Aug 31, 2017

carolynvs Aug 31, 2017

stephenafamo Aug 31, 2017

stephenafamo commented Aug 31, 2017

carolynvs left a comment •

edited

Loading

stephenafamo commented Aug 31, 2017

ibrasho Sep 1, 2017

carolynvs Sep 1, 2017

ibrasho left a comment

sdboyer commented Sep 10, 2017

ebati Sep 10, 2017

ibrasho Sep 11, 2017

carolynvs Sep 11, 2017

ibrasho Sep 11, 2017

stephenafamo Sep 12, 2017

carolynvs commented Sep 11, 2017

dpdornseifer Sep 12, 2017

carolynvs Sep 12, 2017

carolynvs commented Sep 19, 2017


		FROM golang:1.8 AS builder

		RUN apt-get update && apt-get install -y unzip --no-install-recommends && \


		## How do I use `dep` with Docker?

		`dep ensure -vendor-only` creates the vendor folder from a valid `Gopkg.toml` and `Gopkg.lock` without checking for Go code.


		FROM golang:1.8 AS builder

		RUN curl -fsSL -o /usr/local/bin/dep https://github.com/golang/dep/releases/download/vX.X.X/dep-linux-amd64 && chmod +x /usr/local/bin/dep

Document how to use dep with Docker #1094

Document how to use dep with Docker #1094

Conversation

stephenafamo commented Aug 30, 2017

What does this do / why do we need it?

What should your reviewer look out for in this PR?

Do you need help or clarification on anything?

Which issue(s) does this PR fix?

ebati commented Aug 30, 2017

Choose a reason for hiding this comment

carolynvs left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stephenafamo commented Aug 31, 2017

carolynvs left a comment • edited Loading

Choose a reason for hiding this comment

stephenafamo commented Aug 31, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ibrasho left a comment

Choose a reason for hiding this comment

sdboyer commented Sep 10, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

carolynvs commented Sep 11, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

carolynvs commented Sep 19, 2017

carolynvs left a comment •

edited

Loading