Minimal changes to enable a Go workspace

[ivanvc]

This pull request is a follow-up on #19314.

It introduces minimal changes to enable a workspace in the project. We can iterate on it later to remove the script nuances of looping over the modules to execute commands, as they can now be executed from the top level of the repository.

Part of #18409.

Please read https://github.com/etcd-io/etcd/blob/main/CONTRIBUTING.md#contribution-flow.

[k8s-ci-robot]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ivanvc
Once this PR has been reviewed and has the lgtm label, please assign ahrtr for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 68.95%. Comparing base (eb7607b) to head (eb81e5b).
Report is 17 commits behind head on main.

Additional details and impacted files

see 30 files with indirect coverage changes

@@            Coverage Diff             @@
##             main   #19423      +/-   ##
==========================================
+ Coverage   68.89%   68.95%   +0.05%     
==========================================
  Files         420      420              
  Lines       35753    35753              
==========================================
+ Hits        24632    24652      +20     
+ Misses       9698     9673      -25     
- Partials     1423     1428       +5     

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update eb7607b...eb81e5b. Read the comment docs.

[ivanvc]

/test all

[ivanvc]

/cc @ahrtr @serathius

[serathius]

Could we merge it independent to migration to workspaces as a separate PR? Having less changes in the main PR will increase confidence during review.

[ivanvc]

Great idea. Thanks, Marek 🙇. I'll break down this pull request even further, as most of the commits can coexist with the current codebase.

[ivanvc]

I opened #19463.

[serathius]

GOFLAGS=-mod=mod is this even a correct flag? Could we remove it regardless of workspace migration?

[ivanvc]

Yes, I think so. This should predate Go 1.1113.

[ivanvc]

Ah, never mind. Actually, I believe, due to the note in bom_pass, we may need to leave the -mod=mod flag in the tests. But you're right. There's no need for it to be here, as test_lib.sh doesn't define a GOFLAGS=-mod that we need to override. I'll open a pull request for this change.

[serathius]

When I run the two commands now I get different results, is that different when we enable workspaces?

$ go list -f '{{if not .Main}}{{if not .Indirect}}{{.Path}}{{end}}{{end}}' -m all | sort
github.com/anishathalye/porcupine
github.com/coreos/go-semver
github.com/golang/protobuf
github.com/google/go-cmp
github.com/grpc-ecosystem/go-grpc-middleware
github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus
github.com/grpc-ecosystem/grpc-gateway/v2
github.com/prometheus/client_golang
github.com/prometheus/client_model
github.com/prometheus/common
github.com/soheilhy/cmux
github.com/stretchr/testify
go.etcd.io/etcd/api/v3
go.etcd.io/etcd/client/pkg/v3
go.etcd.io/etcd/client/v2
go.etcd.io/etcd/client/v3
go.etcd.io/etcd/etcdctl/v3
go.etcd.io/etcd/etcdutl/v3
go.etcd.io/etcd/pkg/v3
go.etcd.io/etcd/server/v3
go.etcd.io/gofail
go.etcd.io/raft/v3
golang.org/x/crypto
golang.org/x/sync
golang.org/x/time
google.golang.org/grpc
google.golang.org/protobuf
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc
go.opentelemetry.io/otel
go.opentelemetry.io/otel/sdk
go.opentelemetry.io/otel/trace
go.opentelemetry.io/proto/otlp
go.uber.org/zap

$ go mod edit -json | jq -r '.Require[] | select(.Inderect == null) | .Path' | sort
github.com/anishathalye/porcupine
github.com/beorn7/perks
github.com/bgentry/speakeasy
github.com/cenkalti/backoff/v4
github.com/cespare/xxhash/v2
github.com/cheggaaa/pb/v3
github.com/coreos/go-semver
github.com/coreos/go-systemd/v22
github.com/creack/pty
github.com/davecgh/go-spew
github.com/dustin/go-humanize
github.com/fatih/color
github.com/gogo/protobuf
github.com/golang/groupcache
github.com/golang-jwt/jwt/v5
github.com/golang/protobuf
github.com/go-logr/logr
github.com/go-logr/stdr
github.com/google/btree
github.com/google/go-cmp
github.com/google/uuid
github.com/gorilla/websocket
github.com/grpc-ecosystem/go-grpc-middleware
github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus
github.com/grpc-ecosystem/go-grpc-middleware/v2
github.com/grpc-ecosystem/grpc-gateway/v2
github.com/inconshreveable/mousetrap
github.com/jonboulle/clockwork
github.com/klauspost/compress
github.com/mattn/go-colorable
github.com/mattn/go-isatty
github.com/mattn/go-runewidth
github.com/munnerz/goautoneg
github.com/olekukonko/tablewriter
github.com/pmezard/go-difflib
github.com/prometheus/client_golang
github.com/prometheus/client_model
github.com/prometheus/common
github.com/prometheus/procfs
github.com/rivo/uniseg
github.com/sirupsen/logrus
github.com/soheilhy/cmux
github.com/spf13/cobra
github.com/spf13/pflag
github.com/stretchr/testify
github.com/tmc/grpc-websocket-proxy
github.com/VividCortex/ewma
github.com/xiang90/probing
go.etcd.io/bbolt
go.etcd.io/etcd/api/v3
go.etcd.io/etcd/client/pkg/v3
go.etcd.io/etcd/client/v2
go.etcd.io/etcd/client/v3
go.etcd.io/etcd/etcdctl/v3
go.etcd.io/etcd/etcdutl/v3
go.etcd.io/etcd/pkg/v3
go.etcd.io/etcd/server/v3
go.etcd.io/gofail
go.etcd.io/raft/v3
golang.org/x/crypto
golang.org/x/net
golang.org/x/sync
golang.org/x/sys
golang.org/x/text
golang.org/x/time
google.golang.org/genproto/googleapis/api
google.golang.org/genproto/googleapis/rpc
google.golang.org/grpc
google.golang.org/protobuf
go.opentelemetry.io/auto/sdk
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc
go.opentelemetry.io/otel
go.opentelemetry.io/otel/exporters/otlp/otlptrace
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc
go.opentelemetry.io/otel/metric
go.opentelemetry.io/otel/sdk
go.opentelemetry.io/otel/trace
go.opentelemetry.io/proto/otlp
gopkg.in/natefinch/lumberjack.v2
gopkg.in/yaml.v3
go.uber.org/multierr
go.uber.org/zap
sigs.k8s.io/json
sigs.k8s.io/yaml

[ivanvc]

This observation led me down a rabbit hole. It took me a while to review why running go list—m all differs from the output of doing go mod edit—json.

It made me realize I had an issue with my jq expression. So, I revised the expression. The amended version is: .Require[] | select(.Indirect | not) | .Path. And I wrote the following to double-check that the output matches:

$ find . -name go.mod -exec /bin/bash -c 'echo evaluating $(dirname {}); cd $(dirname {}); diff <(go list -f "{{if not .Main}}{{if not .Indirect}}{{.Path}}{{end}}{{end}}" -m all | sort) <(go mod edit -json | jq -r ".Require[] | select(.Indirect | not) | .Path" | sort)' \;
evaluating ./client/pkg
evaluating ./client/v3
evaluating ./client/internal/v2
evaluating ./etcdctl
evaluating ./etcdutl
evaluating ./pkg
evaluating ./server
evaluating ./tests
evaluating ./tools/mod
evaluating ./tools/rw-heatmaps
evaluating ./tools/testgrid-analysis
evaluating ./api
evaluating .

So, there are no differences.

My rabbit hole, investigation, solution, and verification may have been an overkill 😅, as in the context of the original function, it filters the output only to keep go.etcd.io/etcd/*/v{2,3} modules (to do the version bump), which are always in the go.mod. But thanks for the good catch :) I'll update the code and open a pull request targeting this change, as it can coexist with the current code without the workspace.

[ahrtr]

I will review this PR after we officially release v3.6.0, thx