proposal: spec: reduce error handling boilerplate using ?

[ianlancetaylor]

Proposal Details

Background

As discussed in the introduction to an earlier, declined, proposal, Go programs have a lot of error checking code. In surveys error handling is listed as the biggest specific challenge people face using Go today.

There have been many proposals to address this, summarized in a meta-proposal.

This is yet another such proposal. This proposal has some similarities to onerr return, add "or err: statement" after function calls for error handling, and use ? simplify handling of multiple-return-values. It is in some ways a reworking of simplify error handling with || err suffix. There are probably a number of other proposals that fed into this one even if I can't remember them now.

The goal of this proposal is to introduce a new syntax that reduces the amount of code required to check errors in the normal case, without obscuring flow of control.

New syntax

This section is an informal description of the proposal, with examples. A more precise description appears below.

I propose permitting statements of the form

	r, err := SomeFunction()
	if err != nil {
		return fmt.Errorf("something failed: %v", err)
	}

to be written as

	r := SomeFunction() ? {
		return fmt.Errorf("something failed: %v", err)
	}

The ? absorbs the error result of the function. It introduces a new block, which is executed if the error result is not nil. Within the new block, the identifier err refers to the absorbed error result.

Similarly, statements of the form

	if err := SomeFunction2(); err != nil {
		return fmt.Errorf("something else failed: %v", err)
	}

may be written as

	SomeFunction2() ? {
		return fmt.Errorf("something else failed: %v", err)
	}

Further, I propose that the block following the ? is optional. If the block is omitted, it acts as though there were a block that simply returns the error from the function. For example, code like

	if err := SomeFunction2(); err != nil {
		return err
	}

may in many cases be written as

	SomeFunction2() ?

Formal proposal

This section presents the formal proposal.

An assignment or expression statement may be followed by a question mark (?). The question mark is a new syntactic element, the first permitted use of ? in Go outside of string and character constants. The ? causes conditional execution similar to an if statement. A ? at the end of a line causes a semicolon to be automatically inserted after it.

A ? uses a value as described below, referred to here as the qvalue.

For a ? after an assignment statement, the qvalue is the last of the values produced by the right hand side of the assignment. The number of variables on the left hand side of the assignment must be one less than the number of values produced by the right hand side (the right hand side values may come from a function call as usual). It is not valid to use a ? if there is only one value on the right hand side of the assignment.

For a ? after an expression statement the qvalue is the last of the values of the expression. It is not valid to use a ? after an expression statement that has no values.

The qvalue must be of interface type and must implement the predeclared type error; that is, it must have the method Error() string. In most cases it will simply be of type error.

A ? is optionally followed by a block. The block may be omitted if the statement using ? appears in the body of a function, and the enclosing function has at least one result, and the qvalue is assignable to the last result (this means that the type of the last result must implement the predeclared type error, and will often simply be error).

Execution of the ? depends on the qvalue. If the qvalue is nil, execution proceeds as normal, skipping over the block if there is one.

If the ? is not followed by a block, and the qvalue is not nil, then the function returns immediately. The qvalue is assigned to the final result. If the other results (if any) are named, they retain their current values. If they are not named, they are set to the zero value of their type. The results are then returned. Deferred functions are executed as usual.

If the ? is followed by a block, and the qvalue is not nil, then the block is executed. Within the block a new variable err is implicitly declared, possibly shadowing other variables named err. The value and type of this err variable will be those of the qvalue.

That completes the proposal.

Examples

func Run() error {
	Start() ? // returns error from Start if not nil
	Wait() ?  // returns error from Wait if not nil
	return nil
}

func CopyFile(src, dst string) error {
	r := os.Open(src) ? {
		return fmt.Errorf("copy %s %s: %v", src, dst, err)
	}
	defer r.Close()

	w := os.Create(dst) ? {
		return fmt.Errorf("copy %s %s: %v", src, dst, err)
	}

	io.Copy(w, r) ? {
		w.Close()
		os.Remove(dst)
		return fmt.Errorf("copy %s %s: %v", src, dst, err)
	}

	w.Close() ? {
		os.Remove(dst)
		return fmt.Errorf("copy %s %s: %v", src, dst, err)
	}
}

func MustOpen(n string) *os.File {
	f := os.Open(n) ? {
		panic(err)
	}
	return f
}

func TestFileData(t *testing.T) {
	f := os.Open("testfile") ? {
		t.Fatal(err)
	}
	...
}

func CreateIfNotExist(name string) error {
	f, err := os.OpenFile(name, os.O_EXCL|os.O_WRONLY, 0o666)
	if errors.Is(err, fs.ErrExist) {
		return nil
	}
	err ? // returns err if it is not nil
	// write to f ...
}

Discussion

This new syntax is partly inspired by Rust's question mark operator, though Rust permits ? to appear in the middle of an expression and does not support the optional block. Also, I am suggesting that gofmt will enforce a space before the ?, which doesn't seem to be how Rust is normally written.

Absorbing the error returned by a function, and optionally returning automatically if the error is not nil, is similar to the earlier try proposal. However, it differs in that

• ? is an explicit syntactic element, not a call to a predeclared function, and
• ? may only appear at the end of the statement, not in the middle of an expression.

Declaring the err variable

As discussed above, when a block follows the ? it implicitly declares a new err variable. There are no other cases in Go where we implicitly declare a new variable in a scope. Despite that fact, I believe this is the right compromise to maintain readability while reducing boilerplate.

A common suggestion among early readers of this proposal is to declare the variable explicitly, for example by writing

	r := SomeFunction() ? err {
		return fmt.Errorf("something failed: %v", err)
	}

In practice, though, the variable would essentially always be simply err. This would just become additional boilerplate. Since the main goal of this proposal is to reduce boilerplate, I believe that we should try our best to do just that, and introduce err in the scope rather than requiring people to declare it explicitly.

If the implicit declaration of err seems too problematic, another approach would be to introduce a new predeclared name. The name err would not be appropriate here, as that would be too often shadowed in existing code. However, a name like errval or erv would work. Within a ? optional block, this name would evaluate to the qvalue. Outside of a ? optional block, referring to the name would be a compilation error. This would have some similarities to the predeclared name iota, which is only valid within a const declaration.

A third approach would be for errval or erv to be a predeclared function that returns the qvalue.

Supporting other types

As discussed above the qvalue must be an interface type that implements error. It would be possible to support other interface types. However, the ? operator, and especially the implicitly declared err variable, is specifically for error handling. Supporting other types confuses that focus. Using ? with non-error types would also be confusing for the reader. Keeping a focus on just handling errors seems best.

It would also be possible to support non-interface types that implement error, such as the standard library type *os.SyscallError. However, returning a value of that type from a function that returns error would mean that the function always returns a non-nil error value, as discussed in the FAQ. Using different rules for ? would make an already-confusing case even more confusing.

Effects on standard library

I applied a simple rewriter to the standard library to introduce uses of ? where feasible. Here are some examples of new code:

archive/tar/common.go:

		h.Gname = iface.Gname() ?
		h.Uname = iface.Uname() ?

archive/tar/writer_test.go:

	// Test that we can get a long name back out of the archive.
	reader := NewReader(&buf)
	hdr = reader.Next() ? {
		t.Fatal(err)
	}
	if hdr.Name != longName {
		...

archive/zip/reader.go:

	var buf [directoryHeaderLen]byte
	io.ReadFull(r, buf[:]) ?

archive/zip/reader.go:

		p, err := findDirectory64End(r, directoryEndOffset)
		if err == nil && p >= 0 {
			directoryEndOffset = p
			err = readDirectory64End(r, p, d)
		}
		err ?

archive/zip/reader_test.go:

	b := hex.DecodeString(s) ? {
		panic(err)
	}

cmd/cgo/godefs.go:

func gofmt(n interface{}) string {
	gofmtBuf.Reset()
	printer.Fprint(&gofmtBuf, fset, n) ? {
		return "<" + err.Error() + ">"
	}
	return gofmtBuf.String()
}

cmd/cgo/out.go:

		fexp := creat(*exportHeader)
		fgcch := os.Open(*objDir + "_cgo_export.h") ? {
			fatalf("%s", err)
		}
		defer fgcch.Close()
		io.Copy(fexp, fgcch) ? {
			fatalf("%s", err)
		}
		fexp.Close() ? {
			fatalf("%s", err)
		}

os/exec/exec.go:

func (c *Cmd) Run() error {
	c.Start() ?
	return c.Wait()
}

The conversion tool found 544,294 statements in the standard library. It was able to convert 8820 of them to use ?. In all, 1.6% of all statements were changed. 1380 statements, or 0.25% of the total, were changed to use a ? with no optional block.

In other words, adopting this change across the ecosystem would touch an enormous number of lines of existing Go code. Of course, changing existing code could happen over time, or be skipped entirely, as current code would continue to work just fine.

Pros and cons

Pros

Advantage 1: Rewriting

	r, err := SomeFunction()
	if err != nil {
		return fmt.Errorf("something failed: %v", err)
	}

to

	r := SomeFunction() ? {
		return fmt.Errorf("something failed: %v", err)
	}

reduces the error handling boilerplate from 9 tokens to 5, 24 non-whitespace characters to 12, and 3 boilerplate lines to 2.

Rewriting

	r, err := SomeFunction()
	if err != nil {
		return err
	}

to

	r := SomeFunction() ?

reduces boilerplate from 9 tokens to 1, 24 non-whitespace characters to 1, and 3 boilerplate lines to 0.

Advantage 2: This change turns the main code flow into a straight line, with no intrusive if err != nil statements and no obscuring if v, err = F() { … } statements. All error handling either disappears or is indented into a block.

Advantage 3: That said, when a block is used the } remains on a line by itself, unindented, as a signal that something is happening. (I'm also listing this as a disadvantage, below.)

Advantage 4: Unlike the try proposal and some other error handling proposals, there is no hidden control flow. The control flow is called out by an explicit ? operator that can't be in the middle of an expression, though admittedly the operator is small and perhaps easy to miss at the end of the line. I hope the blank before it will make it more visible.

Advantage 5: To some extent this reduces a couple of common error handling patterns to just one, as there is no need to decide between

	var (
		v   string
		err error
	)
	if v, err = F(); err != nil {
		...
	}

and

	v, err := F()
	if err != nil {
		...
	}

Instead people can consistently write

	v := F() ? {
		...
	}

Cons

Disadvantage 1: This is unlike existing languages, which may make it harder for novices to understand. As noted above it is similar to the Rust ? operator, but still different. However, it may not be too bad: Todd Kulesza did a user study and discovered that people unfamiliar with the syntax were able to see that the code had to do with error handling.

Disadvantage 2: The shadowing of any existing err variable may be confusing. Here is an example from the standard library where the ? operator can not be easily used:

fmt/scan.go:

	for n = 1; !utf8.FullRune(r.buf[:n]); n++ {
		r.buf[n], err = r.readByte()
		if err != nil {
			if err == io.EOF {
				err = nil // must change outer err
				break
			}
			return
		}
	}
	// code that later returns err

In this example the assignment err = nil has to change the err variable that exists outside of the for loop. Using the ? operator would introduce a new err variable shadowing the outer one. (In this example using the ? operator would cause a compiler error, because the assignment err = nil would set a variable that is never used.)

Disadvantage 3: When using a block, the } remains on a line itself, taking up space as pure boilerplate. (I'm also listing this as an advantage, above.)

Disadvantage 4: No other block in Go is optional. The semicolon insertion rule, and the fact that a block is permitted where a statement is permitted, means that inserting or removing a newline can convert one valid Go program into another. As far as I know, that is not true today.

For example, these two functions would both be valid and have different meanings, although the only difference is whitespace.

func F1() error {
	err := G1()
	log.Print(err)
	G2() ?
	{
		log.Print(err)
	}
	return nil
}

func F2() error {
	err := G1()
	log.Print(err)
	G2() ? {
		log.Print(err)
	}
	return nil
}

Disadvantage 5: For an expression statement that just calls a function that returns an error, it's easy to accidentally forget the ? and write F() rather than F() ?. Of course it's already easy to forget to check the error result, but once people become accustomed to this proposal it may be easy to overlook the missing ? when reading code.

Disadvantage 6: This proposal has no support for chaining function calls, as in F().G().H(), where F and G also have an error result.

Disadvantage 7: This proposal makes it easier to simply return an error than to annotate the error, by using a plain ? with no block. This may encourage programmers to skip error annotations even when they are desirable.

Disadvantage 8: We really only get one chance to change error handling syntax in Go. We aren't going to make a second change that touches 1.5% of the lines of existing Go code. Is this proposal the best that we can do?

Disadvantage 9: We don't actually have to make any changes to error handling. Although it is a common complaint about Go, it's clear that Go is usable today. Perhaps no change is better than this change. Perhaps no change is better than any change.

Transition

If we adopt this proposal, we should provide tools that can be used to automatically rewrite existing Go code into the new syntax. Not everyone will want to run such a tool, but many people will. Using such a tool will encourage Go code to continue to look the same in different projects, rather than taking different approaches. This tool can't be gofmt, as correct handling requires type checking which gofmt does not do. It could be an updated version of go fix. See also modernizers.

We will have to update the go/ast package to support the use of ?, and we will have to update all packages that use go/ast to support the new syntax. That is a lot of packages.

We will also have to update the introductory documentation and the tour. And, of course, existing Go books will be out of date and will need updating by their authors. The change to the language and compiler is the easiest part of the work.

Possible extensions

These are some possible extensions to the above proposal. These are not part of this proposal, but suggest ways that the language could be developed if this proposal seems useful.

Permit return …, err

This proposal works well with proposal #21182 which permits return …, err to return all zero values other than a final error value. For example,

// OpenOrCreate opens a file, creating it if necessary, and returns the
// file's modification time.
func OpenOrCreate(s string) (*os.File, time.Time, error) {
	f := os.OpenFile(s, os.O_CREATE|os.O_RDWR, 0o666) ?
	fi := f.Stat() ? {
		f.Close()
		return ..., err // instead of "return nil, time.Time{}, err"
	}
	return f, fi.ModTime(), nil
}

That said, if we know that err != nil, then writing err ? is the same as writing return …, err (ignoring the details of named result parameters). We could use err ? in the above example. It would require some analysis to see how often return …, err would be useful when we don't know whether or not err is nil.

Permit using ? with no block outside of a function

We could permit using ? with no block outside of a function, by having it implicitly call panic(err). For example:

// fileContents is a package-scope variable that will
// be set to the contents of the file.
// If os.ReadFile fails, the program will panic
// at initialization time.
var fileContents = os.ReadFile("data") ?

I don't think this comes up often enough to be worth doing. Note that this proposal permits

var fileContents = os.ReadFile("data") ? {
	panic(fmt.Sprintf("could not read data file: %v", err))
}

Permit using ? with no block in a test

We could permit using ? with no block in a test function, by having it implicitly call t.Fatal(err). For example:

func TestSomething(t *testing.T) {
	f := os.ReadFile("data") ?
}

I'm not fond of this because it means that we have to somehow recognize test functions in the language. This proposal does already permit

func TestSomething(t *testing.T) {
	f := os.ReadFile("data") ? {
		t.Fatal(err)
	}
}

or, for that matter,

func TestSomething(t *testing.T) {
	testSomething(t) ? {
		t.Error(err)
	}
}

func testSomething(t *testing.T) error {
	f := os.ReadFile("data") ?
	... // other statements that use ? to handle errors
}

Let gofmt retain single line

We could let gofmt retain a ? block on a single line, as in

	r := os.Open(src) ? { return fmt.Errorf("copy %s %s: %v", src, dst, erv) }

This would reduce the error handling boilerplate by 2 newlines, and keep all error handling indented.

I am not in favor of this myself but I know that people will suggest it.

@gabyhelp's overview of this issue: #71203 (comment)

[jimmyfrasche]

If, instead of a block, it took an optional value of type func(error) error that would allow reusing handlers and with #21498 then inline handlers would just be

stmt ? err => {
  return err
}

which is still quite short

[kortschak]

Would OtherFunction(SomeFunction() ?) be accepted where func SomeFunction() (T, error) and func OtherFunction(v T)? Also for _, v := range (SomeFunction() ?) { … } where func SomeFunction() ([]T, error). (both in a returning context)

[dominikh]

If, instead of a block, it took an optional value of type func(error) error that would allow reusing handlers and with #21498 then inline handlers would just be

stmt ? err => {
  return err
}

which is still quite short

An enticing example would be

func TestSomething(t *testing.T) {
	f := os.ReadFile("data") ? t.Fatal
}

but that would have to accept other signatures in addition to func(error) error, probably using some rule of assignability

[ianlancetaylor]

@kortschak

Would OtherFunction(SomeFunction() ?) be accepted where func SomeFunction() (T, error) and func OtherFunction(v T)? Also for _, v := range (SomeFunction() ?) { … } where func SomeFunction() ([]T, error). (both in a returning context)

No. The ? is only permitted at the end of an assignment statement or an expression statement.

[kortschak]

@ianlancetaylor Is there a rationale for that that I missed?

[Merovius]

Personally, I don't feel like the first version (with the block) pays for itself. The visual difference between x, err := f(); if err != nil { … } and x := f() ? { … } just seems pretty small to me, especially with all the indentation. The second version (without the block) on the other hand, is a significant saving and I see the value in that. So I would only add the second version, if it where up to me.

Also, a nit: I find the formatting with the extra space between the rvalue and the ? (i.e. x := f() ? vs. x := f()?) pretty ugly, though I understand that it's probably intentional to make the ? stand out more.

[ianlancetaylor]

@jimmyfrasche

stmt ? err => {
  return err
}

I think that's a little confusing because that return statement is presumably not returning from the short function literal, it's returning from the function that is calling that function literal.

[ianlancetaylor]

Is there a rationale for that that I missed?

No, I didn't write anything about that topic. The rationale is that I don't think a return should occur in the middle of an expression. I think that is harder to understand, especially as we heard toward cases like A(B() ?, C() ?). I think that was one of the difficulties with the try proposal.

[kortschak]

I think that's fair for the calls, but what about the range case (or thinking about it now, switch with .(type); there is no ambiguity there as it's essentially just sugar for the assignment and then loop.

[gabyhelp]

Related Issues

• proposal: Go 2 : add support for conditional statements #25582 (closed)
• proposal: Go 2: Use `?<variable>` simplify handling of multiple-return-values #33074 (closed)
• proposal: spec: conditional return with return ?err? #70170 (closed)
• proposal: Go 2: use ? during variable assignment as implicit goto statement #64953 (closed)
• proposal: Go 2: simplify error handling with ? and ... #33152 (closed)
• proposal: Go 2: error handling with functions and an error return? #27567 (closed)
• proposal: Go 2: Error Operator #65184 (closed)
• proposal: Go 2: Error-Handling Paradigm with ?err Grammar Sugar #60779 (closed)
• proposal: Go 2: `? :` for returning on function errors #65579 (closed)
• proposal: Go 2: operator to cause early function return on error #32601 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[gophun]

So I would only add the second version, if it where up to me.

Introducing a special syntax solely for the undecorated (in many cases discouraged) error return might make it appear disproportionately more appealing than the preferred, decorated method of handling errors.