proposal: spec: Compile-time recognition of functions that cannot return normally, using `!return`

[apparentlymart]

Programming Experience

Go experience: Experienced
Other languages experience: Rust, C, Python, JavaScript

Related Idea

A similar idea was previously proposed as #69591.

Compared to that proposal, this one proposes "non-returning" as a separate characteristic of a function signature independent of its return type, to limit the impact of the change on existing tools, and on package reflect.

This also proposes a different syntax that uses a combination of a punctuation character followed by an existing keyword, rather than introducing a new predeclared identifier.

This proposal does not directly affect error handling, but I am proposing it today primarily for its interaction with #71528, which is an error-handling proposal. In particular, I am interested to find out whether the idea in #71528 might potentially constitute sufficient new information (relative to the discussion in #69591, which was already declined) to reconsider the value in explicitly tracking functions that do not return.

Although I've opened these as separate issues because they are technically independent of one another, in practice I think that this proposal only has potential value if something like #71528 were also accepted. Without that, this proposal is effectively the same as #69591, which was already declined.

Proposal

I propose allowing a new syntax form in the "Result" production of a function or function type declaration, which declares that a function is guaranteed not to return normally. For example:

package os

func Exit(code int) !return {
    // ...
}

!return is pronounced "not return".

When a function is marked in this way, it is prohibited from including any return statements and it must end with a non-return terminating statement.

An unconditional call to a function marked in this way is a new kind of terminating statement.

Because such a function is a terminating statement, the compiler no longer requires an in-practice-unreachable terminating statement after a call to any of these functions to convince the compiler that the end of the function is unreachable. For example:

func raiseNegative() !return {
    panic(errors.New("negative"))
}

func example(v int) int {
    switch {
    case v < 0:
        raiseNegative()
        // Without this proposal, a dynamically-unreachable return or panic would be required here.
    default:
        return v
    }
}

If #71528 were also accepted, and t.FailNow and all of its unconditional callers were modified to be !return, the two proposals would combine to permit test code like the following:

func TestSomething(t *testing.T) {
    got := MightFail() ? err {
        t.Fatalf("unexpected error: %s", err)
        // without _this_ proposal, but under the rules of #71528,
        // a dynamically-unreachable return or panic would be
        // required here.
    }

    // (remainder of the test, presumably using "got")
}

---

For most type-checking purposes, and for the purposes of all existing APIs for either static analysis or reflection over function types, a !return function is treated as a function whose signature includes no results. !return is an entirely new, separate property of a function type, exposed via new, separate reflection API features. In particular, a function type with !return is assignable to another function type that is identical except for not having !return, although the opposite is not true.

Most existing reflection or static analysis code can ignore this new property and treat such a function as if it returns normally with no results, and thus this change is backward-compatible with existing tools which can treat !return functions just like normal functions that have no results.

The !return property of a function signature affects only what is allowed inside the function's body (as described above) and the function's recognition as a terminating statement when called from elsewhere. Tools that use the "terminating statement" concept for control flow analysis should be updated to recognize !return functions as terminating statements. For example, a tool which detects and warns about unreachable code would begin reporting that any code dominated by a call to a !return function as being unreachable.

Many existing library functions whose documented behavior is consistent with the new !return property have that behavior due to delegating to a lower-level function that has that behavior. For example, testing.T.FailNow delegates to runtime.Goexit. Therefore existing library functions can be gradually improved with !return markings by working outwards from the lowest-level functions. The lowest-level functions will, if implemented in Go themselves, need to include a dynamically-unreachable call to panic as the final non-library-based terminating statement that therefore makes the entire call chain valid as !return. Those which are implemented in assembly language and only declared in Go can have their declaration updated to be !return as long as the assembly implementation guarantees that indeed the function will not return.

Language Spec Changes

Under Function Types, insert the new production NotReturn and redefine Result as follows, leaving all of the other productions unchanged:

NotReturn = "!" "return" .
Result    = Parameters | Type | NotReturn .

Also in that section, add:

A function whose signature includes !return is guaranteed not to return normally once called. All other functions may or may not return normally.

Under Terminating Statements, point 2 changes from "A call to the built-in function panic", to "A call to any !return function".

Under Handling panics, the signature of panic changes to the following, thereby making panic itself a !return function so that it qualifies under the change from the previous paragraph:

func panic(interface{}) !return

Under Function declarations, change the paragraph beginning "If the function's signature declares result parameters", to start instead with "If the function's signature declares result parameters, or !return", and add a new paragraph after its example:

If the function's signature includes !return in the result position, the function's body must not include any return statements, and a return statement is not allowed as its terminating statement.

Informal Change

Most functions return control back to their caller after completing execution, optionally returning one or more values. Some functions may behave in more unusual ways, such as by terminating the entire program. A function marked as !return is special in that it is guaranteed not to return control, and so any statements after it in the program will not be executed.

panic is a !return function that is built into the language. The standard library also includes some !return functions. You can write !return functions in your own program if you wish, although most programs use only the ones from the standard library.

If you write your own !return function then the compiler will return an error unless it can prove that the function does not return. A typical way to ensure that your function does not return is to unconditionally call another, lower-level function that is also marked as !return.

Is this change backward compatible?

Since !return is defined as a tighter constraint on a function's behavior, it is always backward-compatible to add !return to any existing function that returns no results, as long as that function already has no normal return path due to characteristics of its existing implementation.

A function that does not return due to calling another function that does not return cannot be marked as !return until all of its non-returning callees have already been marked as !return, unless the author introduces dynamically-unreachable panic calls or return statements to satisfy the compiler that the function definitely cannot return.

!return is defined as a new, separate property of a function type, as opposed to a new return type, to avoid making breaking changes to package reflect, the static analysis packages, and the tools that use them. Any existing code that is not aware of !return will find what appears to be just a normal function that returns no results. Only tools that actually need to rely on the !return property need be updated to make use of new API features for reporting that.

EDIT: Refer to #71553 (comment) and #71553 (comment) for some further discussion on a backward-compatibility gap that the original proposal text did not identify, and some informal ideas on how it could be addressed by some modifications to the proposal.

Orthogonality: How does this change interact or overlap with existing features?

This change extends the definition of "terminating statement" to include calls to certain user-defined functions. It therefore increases the number of statements that are valid as the last statement in a normal function.

It gives the compiler awareness of a property of a function that is currently not represented in the type system and captured only in human-readable documentation, allowing the compiler to perform more precise analysis and in particular to avoid the need for a redundant, dynamically-unreachable terminating statement after a call to a function that is guaranteed by its documentation to never return.

Although this is only a proposed rather than existing feature, it also addresses one of the ergonomic gaps in #71528, again by allowing the compiler to perform more precise analysis of the control flow within an exceptional block and thus avoid a redundant, dynamically-unreachable terminating statement.

Since I've defined !return as a property of a function type, it is technically possible to declare a variable that can only have !return functions assigned to it. However, it's unclear to me whether that's actually useful in practice.

Would this change make Go easier or harder to learn, and why?

Since there are various functions in the standard library that are already effectively !return per their documentation, most Go programmers can continue to rely on the documentation to learn about this characteristic and so would not need to directly learn about !return in order to successfully write most Go code, unless they want to provide a function that has that characteristic itself.

Several other programming languages with similar characteristics to Go have a feature similar to this. C23 and C++11 both represent this using a function attribute, although those languages do not actually prevent such a function from returning: it's simply undefined behavior to return. Rust includes the "never" type, spelled !, which represents this characteristic as a type that has no values and thus cannot be constructed to return it.

As with those other languages, I expect this feature would be used primarily by standard library features that interact with the runtime or the operating system, and so most Go application or library developers would not learn about it at all.

Some new Go developers might find it helpful for their tools to indicate when a !return function makes a later statement effectively unreachable, thereby avoiding them having to refer to the function's documentation to learn that. However, I expect that benefit is marginal.

Cost Description

This change introduces yet another property to track about a function type, and a new assignability rule for function types that differ only in !return. Function types are already quite complicated.

The runtime type representation used by reflect would need to use another bit of abi.Type.TFlag to represent the presence of !return. This flag applies only to function types, which is not true for any other bit of that field that's currently allocated, and that's potentially confusing. (Other implementations are possible, but this one avoids growing the overall size of the runtime type representation.)

A function marked as !return will often require at least one of its callees to also be marked as !return, and so although there is no requirement to immediately add !return to all existing non-returning functions in the ecosystem there would still likely be pressure from higher-level library authors on lower-level library authors to add !return annotations, so that the higher-level authors can avoid introducing redundant dynamically-unreachable terminating statements to satisfy the compiler that a function is indeed !return. In turn, users of those higher-level libraries may pressure the library authors to add !return to existing functions so that they can be used ergonomically as the terminating statement of a function or in an exception block.

Changes to Go ToolChain

Any tool that attempts to detect and report unreachable code should be changed to reflect the new definition of "terminating statements", which in turn means becoming able to check whether all callees are !return. This includes the unreachable check in go vet.

All tools that include a Go source code parser must learn to treat !return as valid in the function signature result position, rather than reporting it as a syntax error.

Performance Costs

The main additional cost would be felt at compile-time, in the code which verifies whether a function ends with a terminating statement, and in the new code which verifies that a !return function definitely does not return.

!return has no significant runtime performance impact, since all of the checks related to it should complete at compile time. At runtime, a !return function is exactly equivalent to a function that returns no results, except that the control flow instructions in the generated code will definitely not include something equivalent to ret.

[gabyhelp]

Related Issues

• proposal: spec: add `nothing` type for functions that don't return #69591 (closed)
• proposal: Go 2: return error on assignment to _ #65345 (closed)
• proposal: spec: conditional return with return ?err? #70170 (closed)
• spec: go not as smart about log.Panicf() as it is about panic() #10037 (closed)
• Proposal: Panics returned as errors if the function has an error result type #32871 (closed)
• proposal: spec: conditional return with "return if" #70147 (closed)
• Getting 'missing return at end of function' after panic / in dead code #13094 (closed)
• proposal: Go 2: unmuddle syntax of if, conditional return, and several other removals #28245 (closed)
• proposal: Go 2: provide symbol (e.g. #) to abort/panic on non-zero return value #22122 (closed)

Related Discussions

• log.Panicf() not as good as panic(); or, the case for NoReturn

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[seankhliao]

I honestly don't think those changes anything from #69591. You save writing a return, at the cost of a language change and decreased readability as you have to check every function signature to understand control flow, instead of just learning the few standard ones.

[apparentlymart]

Hi @seankhliao,

Over in #71528 I'm hearing that it's desirable for readers to just "know" that functions like t.Fatalf prevent execution of subsequent statements, if the alternative would be to need to write return or panic(...) to convince the compiler that control cannot reach the end of an exception block. So it's surprising to hear over here that it's actually better to have a redundant unreachable terminating statement so that the reader doesn't need to memorize which functions prevent execution of the next statement.

I know that you in particular didn't argue that, and so I'm not meaning to say that this is contradictory feedback -- everyone is allowed to prioritize different things, after all -- but I do find this tension quite interesting.

Perhaps the trick is that people have different preferences and expectations for test code vs main code when it comes to what is "explicit enough" and what is "readable"? 🤔

[seankhliao]

see also #37193

testing's heavy use of runtime.Goexit compared to regular code already enables many helper functions that are more compact than most proposed error handling syntaxes. Over indexing on testing code is probably not a great idea.
testing also encourages keep going, which would be incompatible with #71528

[tmthrgd]

I don't see how this is backwards compatible. Consider:

package example

func Exit() !return {
        panic("exit")
}

package example2

var _ func() = example.Exit

If example.Exit had been originally declared without !return, then adding it breaks this toy program.

[apparentlymart]

Hi @tmthrgd,

Per this proposal, func() !return is assignable to func (), because it's still a function that doesn't return any results.

The opposite is not true, but there cannot be any existing program containing a variable of type func() !return because there is not yet any syntax to express that.

[tmthrgd]

package example

func Exit() !return {
        panic("exit")
}

package example2

var exit = example.Exit

func init() {
         exit = func() {}
}

That's not backward compatible.

[apparentlymart]

I assume the point you are making is that without some sort of exception to type inference, the compiler would infer example2.exit to have type func () !return after this change, but today it would be inferred as func (). In which case, fair enough: that would be incompatible.

It seems that to make this case work (without a weird type inference exception) it would be necessary for !return to be a property of specific concrete functions rather than of function types, as it the case in some other languages. The "terminating statement" rule would apply only to direct function calls, and would never apply to an indirect call through a function pointer.

It would not be possible to create a function pointer that can only have !return functions assigned to it, but I don't think that's crucial for the goals this proposal was intended to meet, which are primarily related to specific standard library functions such as those in package testing that indirectly call runtime.Goexit.

I'm not going to update the proposal to reflect that since I expect this proposal to be declined for other reasons anyway, but I'm just noting the above in case someone finds this discussion in future and wishes to embark on the dubious errand of making yet another "no return" proposal. 😀

[apparentlymart]

r.e. @seankhliao's further remarks about testing:

The other proposal #37193 is compatible with "keep going", but would require explicitly including a fallthrough statement in the error-handling block, and I expect some would consider that a deal-breaker.

ThingUnderTest() ? err {
    t.Errorf("unexpected error: %s")
    fallthrough
}

In my experience "keep going" tends to apply more to non-error-related outcomes like got != want than to "unexpected error" situations, but of course it is sometimes possible to keep going when an error occurs.

Subtests using t.Run also provide another variant of "keep going" that is more directly compatible with #37193, because it can halt one specific subtest (by returning from the subtest's function) but allow other subtests to proceed.

I do broadly agree with you though that the error-handling proposal seems less relevant to test code than to main code. Others in the discussion seem to disagree however, and that is what prompted me to start this proposal: I have the sense that for some there is no value in an error-handling proposal that is not useful and ergonomic in test code.