Probable wrong wiring, may inverting columns and rows - if matrix is used.

Old* joystick HAT's is just 4 micro-switch around an pole.

* VKB and VPC are using ALPS RKJXM (5 way) as HAT.