This commit includes a little restructuring necessary for keeping wayland objects properly.
It doesn't fix broken modifier functionality yet.