I made great progress today. Allthe buttons respond, all the leds work, i got the 2.2" tft working (until i get my 3.95" screen) and it can send keys to the pc. currently, it is just a basic keypad without addional functionality.
I'm a bit disappointed with the key response though. With these tact switches i have to press pretty firm to make contact. I may have to go back to the drawing board and find some proper backlit switches maybe cherry MX (although they are pretty expensive).