gcc ASAN doesn't stop on purported runtime error

I would like address sanitizer to abort when it catches something. I thought it was designed to do that by default but it does not seem to be working for me. I also tried ASAN_OPTIONS=halt_on_error=1 which had no effect. Here are the details:

In a project I work on we use address sanitizer and it gave off this warning/error for weeks without anyone realizing it:

runtime error: null pointer passed as argument xx, which is declared to never be null

Despite being called a runtime error it does not stop the program or cause a bad exit code. Here is a simple program to demonstrate it:

/*
gcc -fsanitize=address,undefined \
    -Wformat \
    -Werror=format-security \
    -Werror=array-bounds \
    -g -o xasan xasan.c
*/

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(int argc, char *argv[]) {
  fprintf(stderr, "before\n");
  memcpy(NULL, argc > 1 ? "" : NULL, argc > 1 ? 1 : 0);
  fprintf(stderr, "after\n");
  return 0;
}

The argc trick is just so the gcc doesn't optimize out the memcpy call, basically in our code it ends up as memcpy(dst, NULL, 0) which causes the runtime error/warning.

I would expect that 'after' would not be output after the runtime error but it is and the program exit code is 0. Is this a bug? The manual says it's supposed to stop.