Unable to generate mocks in reflect mode

[nezorflame]

Actual behavior A clear and concise description of what the bug is.

My project uses Go modules, but due to the CI restrictions we also make use of the vendor folder.
So, our modules are synced by go mod tidy && go mod vendor command.

While using mockgen in a reflect mode, I'm unable to generate mocks. Can't pinpoint exactly when this became broken as this was working fine before, but it doesn't anymore.

The error message:

❯ go generate ./...

/Users/nezorflame/Go/src/github.com/nezorflame/mockgen-test - service - doc.go
prog.go:12:2: cannot find package "." in:
        /Users/nezorflame/Go/src/github.com/nezorflame/mockgen-test/vendor/github.com/golang/mock/mockgen/model
prog.go:12:2: cannot find package "." in:
        /Users/nezorflame/Go/src/github.com/nezorflame/mockgen-test/vendor/github.com/golang/mock/mockgen/model
prog.go:12:2: cannot find module providing package github.com/golang/mock/mockgen/model: working directory is not part of a module
prog.go:14:2: cannot find module providing package github.com/nezorflame/mockgen-test/internal/client: working directory is not part of a module
2020/10/22 19:41:04 Loading input failed: exit status 1
internal/service/doc.go:2: running "mockgen": exit status 1

Seems to be somewhat similar to the #423 but the proposed solution didn't work for me.

Expected behavior A clear and concise description of what you expected to
happen.

Mocks should've been generated fine.

To Reproduce Steps to reproduce the behavior

I've published a test project which can be used to reproduce the issue.

mockgen is called during the go:generate directive at internal/service/doc.go:

//go:generate echo $PWD - $GOPACKAGE - $GOFILE
//go:generate mockgen -package $GOPACKAGE -destination mocks_test.go github.com/nezorflame/mockgen-test/internal/client Producer

The interface to generate the mocks upon is pretty simple and is stored at internal/client/contract.go:

type Producer interface {
	Produce(ctx context.Context, objects []string) error
}

So, the steps are:

1. Download the project
2. (Optional) Generate the mocks without the vendor:
   a. Run go mod tidy && go mod vendor to clean up the dependencies
   b. Run go generate ./... once - it'll work fine, mocks_test.go file will be generated
   c. Delete mocks_test.go file to reproduce the error
3. Run go generate ./...
4. The error appears

It seems that the fact that vendor contains dependencies somehow breaks the ability of mockgen to generate the mocks, since the vendor-less mode works fine.

Additional Information

• gomock mode (reflect or source): reflect
• gomock version or git ref: v1.4.4
• golang version: v1.5.3

go env output:

GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/nezorflame/Library/Caches/go-build"
GOENV="/Users/nezorflame/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="<my_gopath>/pkg/mod"
GONOPROXY="my.company.com"
GONOSUMDB="my.company.com"
GOOS="darwin"
GOPATH="<my_gopath>"
GOPRIVATE="my.company.com"
GOPROXY="https://proxy.golang.org,https://goproxy.io,https://gocenter.io,direct"
GOROOT="<my_goroot>"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="<my_goroot>/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/4p/q8s4q9s50qbdl5vllqg5fs000000gp/T/go-build294373709=/tmp/go-build -gno-record-gcc-switches -fno-common"

[cvgw]

The error message is due to github.com/golang/mock/mockgen/model not being vendored golang/go#41936

I would assume the reason you started seeing it at an indeterminate time has to do with them slowly making modules the default behavior in new versions of golang.

How to fix this is a bit more tricky.

The golang program generated by reflection mode depends on github.com/golang/mock/mockgen/model, but in most cases nothing else would depend on this, hence it not being included in the vendor folder.

There are probably some hacky work arounds like forcing the import with an _ "github.com/golang/mock/mockgen/model", but I'm not sure what the long term fix is.

cc @codyoss

[cvgw]

As super hacky work around you can copy the directory github.com/golang/mock/mockgen/model into your vendor folder

[codyoss]

I think this is related to recent changes in Go tooling. Adding --build_flags=--mod=mod is one workaround. The other would be to include an empty import of that package so it gets pulled in import _ "github.com/golang/mock/mockgen/model".

[nezorflame]

Thanks!
For now I'll use the proposed workaround with the empty import (and a comment to this issue), but it'd be nice to see this resolved.
Anything I could do to help further?

[cvgw]

Thanks!
For now I'll use the proposed workaround with the empty import (and a comment to this issue), but it'd be nice to see this resolved.
Anything I could do to help further?

I think what needs to happen now is some discussion about how to conceptually solve this problem; any input would be welcome and appreciated.

The basic overview (IIUC @codyoss can correct me) is that during reflection mode we dynamically generate some golang source code for a new main program. This main program, the reflection program, is the heart of what makes reflection mode work.

The source code is written out by mockgen and then mockgen executes some shell commands to compile and run the reflection program.

Problem we are hitting is in the dependency management; the reflection program requires github.com/golang/mock/mockgen/model but your code does not.

When the go toolchain attempts to compile the reflection program it sees your vendor directory, but github.com/golang/mock/mockgen/model is not contained and so it fails.

So we essentially have two different build environments that require different dependencies.

[valters]

One of the problems is that the error message is so cryptic and unergonomic, it is impossible to figure out why this is happening. And the reason why (as comments above explain) is that that the mockgen/model directory never was vendored in the first place.

My workaround (strangely enough) was to remove the whole vendor directory, and this allows mockgen to work.