Closed
Description
I built my application using clang-cl with optimizations enabled and the /hotpatch flag (to try out Live++). It compiled and linked successfully, but when I try to run it, the very first memory allocation hits an int 3 instruction. It seems all the memory allocation functions (default malloc, default new/new[], custom malloc/new, etc) are just a nop followed by several int 3s. It seems to have assumed it's unreachable code or something?
For example, a somewhat reduced test case (and here's a Godbolt link):
typedef unsigned long long size_t;
extern "C" {
void* mi_new(size_t size);
}
void *mi_new_test(size_t count)
{
return mi_new(count);
}
void *builtin_malloc_test(size_t count)
{
return __builtin_malloc(count);
}When buit with -O0 -target x86_64-pc-windows-msvc19.38.33133 -fms-hotpatch, it yields fairly reasonable (if unoptimized) code:
"?mi_new_test@@YAPEAX_K@Z": # @"?mi_new_test@@YAPEAX_K@Z"
sub rsp, 40
mov qword ptr [rsp + 32], rcx
mov rcx, qword ptr [rsp + 32]
call mi_new
nop
add rsp, 40
ret
"?builtin_malloc_test@@YAPEAX_K@Z": # @"?builtin_malloc_test@@YAPEAX_K@Z"
sub rsp, 40
mov qword ptr [rsp + 32], rcx
mov rcx, qword ptr [rsp + 32]
call malloc
nop
add rsp, 40
ret
When -O2 (or indeed any -O flag higher than 0) is used, it emits this:
"?mi_new_test@@YAPEAX_K@Z": # @"?mi_new_test@@YAPEAX_K@Z"
xchg ax, ax
"?builtin_malloc_test@@YAPEAX_K@Z": # @"?builtin_malloc_test@@YAPEAX_K@Z"
xchg ax, ax
What happened here?