I have the same problem. However, there are a bunch of things I can't do with the controller, but I can use it as a primary control. Then I can still click on things (inventory, game speed/pause, skills). It's the only thing I've seen that completely goes away.