Cannot modify data segment register. When tried General Protection Error is thrown

I have been trying to create an ISR handler following this tutorial by James Molloy but I got stuck. Whenever I throw a software interrupt, general purpose registers and the data segment register is pushed onto the stack with the variables automatically pushed by the CPU. Then the data segment is changed to the value of 0x10 (Kernel Data Segment Descriptor) so the privilege levels are changed. Then after the handler returns those values are poped. But whenever the value in ds is changed a GPE is thrown with the error code 0x2544 and after a few seconds the VM restarts. (linker and compiler i386-elf-gcc , assembler nasm)

I tried placing hlt instructions in between instructions to locate which instruction was throwing the GPE. After that I was able to find out that the the `mov ds,ax' instruction. I tried various things like removing the stack which was initialized by the bootstrap code to deleting the privilege changing parts of the code. The only way I can return from the common stub is to remove the parts of my code which change the privilege levels but as I want to move towards user mode I still want them to stay.

Here is my common stub:

isr_common_stub:
    pusha                    ; Pushes edi,esi,ebp,esp,ebx,edx,ecx,eax
    xor eax,eax
    mov ax, ds               ; Lower 16-bits of eax = ds.
    push eax                 ; save the data segment descriptor

    mov ax, 0x10  ; load the kernel data segment descriptor
    mov ds, ax
    mov es, ax
    mov fs, ax
    mov gs, ax

    call isr_handler

    xor eax,eax
    pop eax
    mov ds, ax ; This is the instruction everything fails;
    mov es, ax
    mov fs, ax
    mov gs, ax
    popa
    iret

My ISR handler macros:

extern isr_handler

%macro ISR_NOERRCODE 1
  global isr%1        ; %1 accesses the first parameter.
  isr%1:
    cli
    push byte 0
    push %1
    jmp isr_common_stub
%endmacro

%macro ISR_ERRCODE 1
  global isr%1
  isr%1:
    cli
    push byte %1
    jmp isr_common_stub
%endmacro
ISR_NOERRCODE 0
ISR_NOERRCODE 1
ISR_NOERRCODE 2
ISR_NOERRCODE 3
...

My C handler which results in "Received interrupt: 0xD err. code 0x2544"

#include <stdio.h>
#include <isr.h>
#include <tty.h>

void isr_handler(registers_t regs) {
    printf("ds: %x \n" ,regs.ds);
    printf("Received interrupt: %x with err. code: %x \n", regs.int_no, regs.err_code);
}

And my main function:

void kmain(struct multiboot *mboot_ptr) {
    descinit(); // Sets up IDT and GDT
    ttyinit(TTY0); // Sets up the VGA Framebuffer
    asm volatile ("int $0x1"); // Triggers a software interrupt
    printf("Wow"); // After that its supposed to print this
}

As you can see the code was supposed to output,

ds: 0x10
Received interrupt: 0x1 with err. code: 0

but results in,

...
ds: 0x10
Received interrupt: 0xD with err. code: 0x2544

ds: 0x10
Received interrupt: 0xD with err. code: 0x2544
...

Which goes on until the VM restarts itself.

What am I doing wrong?