Yes, I leave my joystick plugged all the time too.

But it is still strange since the the changing sensor values only affect one joystick,
the other is super stable all the time.

I forget to tell you that before I can change the HALL sensors to S495A,
I also used Allegro A1302,
and with those, I also get the the same changing sensor values on that same joystick.