Ive been having this problem too. I assumed i was doing somthing wrong.
It is still playable but slightly annoying. Close to the edge of each side screen the view is quite distorted.
Is there any way of minimalising this?
Changing the FOV in the game, effectively "zooming in" the image (ie in FPS the view moves towards your gun tip). Its not that effective a solution.
The issue really is that you arent meant to be looking at the side screens. Imagine wearing a VR helmet, you dont spend your time spinning your eyes to look at the left/right of the front mounted LCD they use. The sides are there for immersion only (in first and 3rd person, overhead games are different), they exist only to aid your peripheral vision.