misc/cgo: testplugin fails on Darwin with `runtime: bad pointer in frame` on the latest tip

[YoshikiShibata]

Please answer these questions before submitting your issue. Thanks!

What did you do?

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

Try to rebuild the latest tip version with all.bash

What did you expect to see?

All tests would pass.

What did you see instead?

##### ../misc/cgo/testplugin
runtime: bad pointer in frame main.main at 0xc000301cf0: 0x8
fatal error: invalid pointer found on stack

runtime stack:
runtime.throw(0x750b94c, 0x1e)
	/Users/yoshiki/tools/go/src/runtime/panic.go:598 +0x72 fp=0x7ffeefbfe9a8 sp=0x7ffeefbfe978 pc=0x7486332
runtime.adjustpointers(0xc000301c90, 0x7ffeefbfeaa0, 0x7ffeefbfee50, 0x4170448, 0x4178360)
	/Users/yoshiki/tools/go/src/runtime/stack.go:592 +0x23c fp=0x7ffeefbfea18 sp=0x7ffeefbfe9a8 pc=0x74998dc
runtime.adjustframe(0x7ffeefbfed60, 0x7ffeefbfee50, 0x4178360)
	/Users/yoshiki/tools/go/src/runtime/stack.go:663 +0x325 fp=0x7ffeefbfead0 sp=0x7ffeefbfea18 pc=0x7499c25
runtime.gentraceback(0xffffffffffffffff, 0xffffffffffffffff, 0x0, 0xc000000180, 0x0, 0x0, 0x7fffffff, 0x750e220, 0x7ffeefbfee50, 0x0, ...)
	/Users/yoshiki/tools/go/src/runtime/traceback.go:310 +0x12a1 fp=0x7ffeefbfedc8 sp=0x7ffeefbfead0 pc=0x74a2111
runtime.copystack(0xc000000180, 0x4000, 0x7ffeefbfef01)
	/Users/yoshiki/tools/go/src/runtime/stack.go:891 +0x26c fp=0x7ffeefbfef80 sp=0x7ffeefbfedc8 pc=0x749a70c
runtime.newstack()
	/Users/yoshiki/tools/go/src/runtime/stack.go:1063 +0x315 fp=0x7ffeefbff118 sp=0x7ffeefbfef80 pc=0x749ab25
runtime.morestack()
	/Users/yoshiki/tools/go/src/runtime/asm_amd64.s:481 +0x8f fp=0x7ffeefbff120 sp=0x7ffeefbff118 pc=0x74a7baf

goroutine 1 [copystack]:
strings.NewReplacer(0x0, 0x0, 0x0, 0xc0000d4188)
	/Users/yoshiki/tools/go/src/strings/replace.go:23 +0x71b fp=0xc000301830 sp=0xc000301828 pc=0x74d3c2b
plugin2.init.0()
	/Users/yoshiki/tools/go/misc/cgo/testplugin/src/plugin2/plugin2.go:21 +0x32 fp=0xc000301860 sp=0xc000301830 pc=0x74d7482
plugin2.init()
	<autogenerated>:1 +0x60 fp=0xc000301870 sp=0xc000301860 pc=0x74d7740
plugin.open(0x40f3e83, 0x7, 0xc0000dc000, 0x200000, 0x200000)
	/Users/yoshiki/tools/go/src/plugin/plugin_dlopen.go:113 +0xac8 fp=0xc000301b28 sp=0xc000301870 pc=0x40b4478
plugin.Open(0x40f3e83, 0xa, 0x40f31d0, 0x1, 0xc0000ba3e8)
	/Users/yoshiki/tools/go/src/plugin/plugin.go:32 +0x35 fp=0xc000301b60 sp=0xc000301b28 pc=0x40b3575
main.main()
	/Users/yoshiki/tools/go/misc/cgo/testplugin/src/host/host.go:129 +0x918 fp=0xc000301f88 sp=0xc000301b60 pc=0x40b5d98
runtime.main()
	/Users/yoshiki/tools/go/src/runtime/proc.go:198 +0x207 fp=0xc000301fe0 sp=0xc000301f88 pc=0x402d5d7
runtime.goexit()
	/Users/yoshiki/tools/go/src/runtime/asm_amd64.s:1385 +0x1 fp=0xc000301fe8 sp=0xc000301fe0 pc=0x4053771

goroutine 2 [force gc (idle)]:
runtime.gopark(0x40fb368, 0x4186d20, 0x40f48cb, 0xf, 0x40fb214, 0x1)
	/Users/yoshiki/tools/go/src/runtime/proc.go:291 +0x105 fp=0xc000034768 sp=0xc000034748 pc=0x402da15
runtime.goparkunlock(0x4186d20, 0x40f48cb, 0xf, 0x14, 0x1)
	/Users/yoshiki/tools/go/src/runtime/proc.go:297 +0x5e fp=0xc0000347a8 sp=0xc000034768 pc=0x402dade
runtime.forcegchelper()
	/Users/yoshiki/tools/go/src/runtime/proc.go:248 +0xca fp=0xc0000347e0 sp=0xc0000347a8 pc=0x402d86a
runtime.goexit()
	/Users/yoshiki/tools/go/src/runtime/asm_amd64.s:1385 +0x1 fp=0xc0000347e8 sp=0xc0000347e0 pc=0x4053771
created by runtime.init.4
	/Users/yoshiki/tools/go/src/runtime/proc.go:237 +0x35

goroutine 3 [GC sweep wait]:
runtime.gopark(0x40fb368, 0x4186e00, 0x40f445a, 0xd, 0x401fc14, 0x1)
	/Users/yoshiki/tools/go/src/runtime/proc.go:291 +0x105 fp=0xc000034f60 sp=0xc000034f40 pc=0x402da15
runtime.goparkunlock(0x4186e00, 0x40f445a, 0xd, 0x14, 0x1)
	/Users/yoshiki/tools/go/src/runtime/proc.go:297 +0x5e fp=0xc000034fa0 sp=0xc000034f60 pc=0x402dade
runtime.bgsweep(0xc000064000)
	/Users/yoshiki/tools/go/src/runtime/mgcsweep.go:52 +0xa2 fp=0xc000034fd8 sp=0xc000034fa0 pc=0x401fcd2
runtime.goexit()
	/Users/yoshiki/tools/go/src/runtime/asm_amd64.s:1385 +0x1 fp=0xc000034fe0 sp=0xc000034fd8 pc=0x4053771
created by runtime.gcenable
	/Users/yoshiki/tools/go/src/runtime/mgc.go:216 +0x58

goroutine 18 [finalizer wait]:
runtime.gopark(0x40fb368, 0x41a31b8, 0x40f46d6, 0xe, 0x14, 0x1)
	/Users/yoshiki/tools/go/src/runtime/proc.go:291 +0x105 fp=0xc000030718 sp=0xc0000306f8 pc=0x402da15
runtime.goparkunlock(0x41a31b8, 0x40f46d6, 0xe, 0x14, 0x1)
	/Users/yoshiki/tools/go/src/runtime/proc.go:297 +0x5e fp=0xc000030758 sp=0xc000030718 pc=0x402dade
runtime.runfinq()
	/Users/yoshiki/tools/go/src/runtime/mfinal.go:175 +0xac fp=0xc0000307e0 sp=0xc000030758 pc=0x4016fec
runtime.goexit()
	/Users/yoshiki/tools/go/src/runtime/asm_amd64.s:1385 +0x1 fp=0xc0000307e8 sp=0xc0000307e0 pc=0x4053771
created by runtime.createfing
	/Users/yoshiki/tools/go/src/runtime/mfinal.go:156 +0x61

Does this issue reproduce with the latest release (go1.10.1)?

N/A

System details

go version devel +da02dcda02 Mon Apr 2 23:39:16 2018 +0000 darwin/amd64
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/yoshiki/Library/Caches/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/yoshiki/gocode"
GORACE=""
GOROOT="/Users/yoshiki/tools/go"
GOTMPDIR=""
GOTOOLDIR="/Users/yoshiki/tools/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/rk/f21ym6ks1ydfj0rf1kqt1ctc0000gn/T/go-build235793060=/tmp/go-build -gno-record-gcc-switches -fno-common"
GOROOT/bin/go version: go version devel +da02dcda02 Mon Apr 2 23:39:16 2018 +0000 darwin/amd64
GOROOT/bin/go tool compile -V: compile version devel +da02dcda02 Mon Apr 2 23:39:16 2018 +0000
uname -v: Darwin Kernel Version 17.4.0: Sun Dec 17 09:19:54 PST 2017; root:xnu-4570.41.2~1/RELEASE_X86_64
ProductName:	Mac OS X
ProductVersion:	10.13.3
BuildVersion:	17D102
lldb --version: lldb-902.0.79.2
  Swift-4.1

[odeke-em]

Thank you @YoshikiShibata for reporting this issue!

/cc @TocarIP @bradfitz, perhaps this just added CL https://go-review.googlesource.com/#/c/97255/ aka commit fd22542
which was the last to touch strings.Replace?

[bradfitz]

Duplicate of #23369 it seems? But actually hurting us now.

/cc @aclements @randall77 @crawshaw @hirochachacha @bcmills

[bradfitz]

I really don't want to roll back https://go-review.googlesource.com/#/c/97255/ (commit fd22542). It's innocent. It just tickled an existing bug.

But re-disabling plugin support on Darwin would also be sad.

[bradfitz]

/cc @mdempsky too

[cherrymui]

The throwing part of the stack trace (from morestack to throw) has the PCs of runtime functions at 0x74xxxxx, whereas for goroutine 1 runtime.goexit and runtime.main have PCs at 0x40xxxxx. For one module, I think the linker will put all runtime functions together, adjacent in PC. So it seems that morestack from another module's runtime is called. In the second runtime, runtime.framepointer_enabled is not set by the linker, so the stack scan goes off from there.

If I understand correctly, we suppose to always use the same runtime, right? This seems broken on darwin/amd64.

[aclements]

We probably shouldn't even allow loading a plugin where the value of framepointer_enabled doesn't match that of the main binary. Even if we're using one runtime, this would make it unable to unwind frames created by the plugin.

[aclements]

Apparently this is a todo. In plugin/plugin_dlopen.go:

	// TODO(crawshaw): look for plugin note, confirm it is a Go plugin
	// and it was built with the correct toolchain.

(where "correct toolchain" should include the experiments string)

[cherrymui]

In this case, the plugin is indeed built with the "correct toolchain", and frame pointer is indeed enabled. But the linker doesn't set runtime.framepointer_enabled for runtimes other than the main runtime, presumably because it is not supposed to be used. Always setting it does seem to make the test pass, but I guess it's just hiding the real bug.

[aclements]

Okay, that makes more sense. (We should still check that it's compatible, but you're right that that's not the problem here.)