Some background:I'm writing a bare-metal C++ app/OS for the Raspberry Pi 4B (in 64-bit mode, so booting kernel8.elf off of an SD card) and I've been running into strange crashes/hangs (where logging to the screenbuffer just stops with no explanation) while doing pretty normal C++ tasks such as:
- Constructing an object in main() (though making the same object a global seemed to work)
- Compiling at -O0, while -O2 and -O3 tend to work
- Calling member functions
My first instinct was to look for UB in my program, and I'm pretty certain that isn't the problem. The ctor in question just calls some inline ASM to read the PI's model number and set the MMIO base address accordingly.
I'm unable to post my sourcecode publicly, but I think (and hope) that this is an issue with how I'm compiling/linking my executable.
I'd prefer to compile/link against the C++20 stdlib, since I want to use std::arrays and optionals throughout this project.
Here's the makefile I'm using
ARCH=cortex-a72TRIPLE=aarch64-none-elfXDIR:=/PATH/TO/GCC/$(TRIPLE)GCC_VERSION=11.2.1XBINDIR:=$(XDIR)/binXLIBDIR1:=$(XDIR)/libXLIBDIR2:=$(XDIR)/lib/gcc/$(TRIPLE)/$(GCC_VERSION)XLIBDIR3:=$(XDIR)/$(TRIPLE)/libAR:=$(XBINDIR)/$(TRIPLE)-arASM:=$(XBINDIR)/$(TRIPLE)-gccCC:=$(XBINDIR)/$(TRIPLE)-gccCXX:=$(XBINDIR)/$(TRIPLE)-g++LD:=$(XBINDIR)/$(TRIPLE)-ldOBJCOPY:=$(XBINDIR)/$(TRIPLE)-objcopyRANLIB:=$(XBINDIR)/$(TRIPLE)-ranlibSIZE:=$(XBINDIR)/$(TRIPLE)-sizeSTRIP:=$(XBINDIR)/$(TRIPLE)-strip# COMPILE OPTIONSWARNINGS=-Wall -Wextra -Wpedantic OPTS=-O3CFLAGS:=-g $(OPTS) -pipe -flto=auto -static-pie -fsigned-char $(WARNINGS) -mcpu=$(ARCH)\ -static -ffreestanding -nostartfilesCXXFLAGS:=$(CFLAGS) -std=c++20 -fno-exceptions -fno-unwind-tables -fno-rttiLDFLAGS:= -Wl,-nmagic\ -Wl,-Tlinker.ld\ -L.\ -L$(XLIBDIR1)\ -L$(XLIBDIR2)\ -L$(XLIBDIR3)\ -lc\ -lgcc\ -lstdc++\ -Wl,-gc-sectionsRM=rm -f# Source files and include dirsSOURCES := $(wildcard src/*.cc) $(wildcard src/*.c) $(wildcard src/*.S) $(wildcard test/*cc)# Create .o and .d files for every .cc and .S (hand-written assembly) fileOBJECTS := $(patsubst %.c, %.o, $(patsubst %.S, %.o, $(patsubst %.cc,%.o,$(SOURCES))))DEPENDS := $(patsubst %.c, %.d, $(patsubst %.S, %.d, $(patsubst %.cc,%.d,$(SOURCES))))INC=-Iinclude# .PHONY means these rules get executed even if# files of those names exist..PHONY: all clean# The first rule is the default, ie. "make",# "make all" and "make kernel8.elf" mean the sameall: kernel8.elfclean: $(RM) $(OBJECTS) $(DEPENDS) kernel8.elf kernel8.img# Linking the executable from the object fileskernel8.elf kernel8.img &: $(OBJECTS) linker.ld $(CXX) $(INC) $(CXXFLAGS) $(filter-out %.ld, $^) -o $@ $(LDFLAGS) $(OBJCOPY) $@ -O binary kernel8.img-include $(DEPENDS)QEMUCMD=qemu-system-aarch64 -M raspi3b\ -kernel kernel8.img\ -display none\ -serial null\ -serial stdio\ -semihosting\ -d unimpdebug: CFLAGS += -DDEBUG -Ogdebug: kernel8.img $(QEMUCMD) -s -Srun: kernel8.img $(QEMUCMD)%.o: %.cc Makefile $(CXX) $(INC) $(CXXFLAGS) -MMD -MP -c $< -o $@ $(LDFLAGS)%.o: %.c Makefile $(CXX) $(INC) $(CFLAGS) -MMD -MP -c $< -o $@ $(LDFLAGS)Here's linker.ld:
ENTRY(_start)SECTIONS { . = 0x80000; .text : { KEEP(*(.text.boot)) *(.text .text.* .gnu.linkonce.t*) } .rodata : { *(.rodata .rodata.* .gnu.linkonce.r*) } PROVIDE(_data = .); .data : { *(.data .data.* .gnu.linkonce.d*) } .bss (NOLOAD) : { . = ALIGN(16); __bss_start = .; __bss_start__ = __bss_start; *(.bss .bss.*) *(COMMON) __bss_end = .; } __bss_size = SIZEOF(.bss); . = ALIGN(4096); .init_array : { __init_array_start = .; KEEP (*(.init_array*)) __init_array_end = .; } __end = .; /* needed for certain newlib routines that (potentially) call _sbrk */ end = __bss_end; __end__ = end; __dso_handle = 0;}And finally, here's the .S file that I define _start in:
//https://www.rpi4os.com/part1-bootstrapping/.section ".text.boot" // Make sure the linker puts this at the start of the kernel image.global _start // Execution starts here.global exit_start: // Check processor ID is zero (executing on main core), else hang mrs x1, mpidr_el1 and x1, x1, #3 // We're not on the main core, so hang in an infinite loop cbnz x1, exit // We're on the main core! // initialize SP ldr x0, =_start mov sp, x0 // init global objects and BSS before handing control over to C++ bl init // Jump to our main() routine in C++ (make sure it doesn't return) bl main // if main returns, just spinexit: b exitinit just calls std::fill to zero .bss, then iterates through the ctors in init_array.
For my compiler, I'm using the arm GNU toolchain on an x86-64 linux host, which I downloaded from here.
I'm hoping there's something that's just obviously wrong with my setup, since I've tried all the usual debugging steps from C++-land, to no avail. The exact same code works when called inline, but hangs/crashes when executed by a method.Thanks!
