With my setup

I5 , 6600K (clocked at 4.4ghz)
GTX 970 4GB 1560mhz

I experienced the same issues,
These are a few steps that really helped me out:

1: Clean drivers, download the latest drivers from Nvidia, and perform a clean install.
2: After the fresh install, make sure to restart and clean old nvidia files
3: Once done, open the Nvidia Configuration panel, check all the settings and make sure that under "3D" settings, you check "Let the application decide", Nvidia standard has this setting on
"Nvidia settings", if you prefer this, you could edit the Global settings in the tab next to it... Make sure to not use the "Slider" to 'emphasize quality', as it is known to cause performance issues.
4: Check game config , if needed verify the installation

Once everything is clean, and setup
verify the health of your graphic card, there are several tools out there that can be used: It's which you prefer.

After checking this, the report will hopefully let you know if it is a software, or hardware problem.
If it seems to be healthy, and has no major issues that you need to address first, try running the same game
and check for differences.

My GTX 970 isn't really a FPS beast, but this is what I've done today and I can see major improvements in stability now: I Hope it helps you too !
If you have any questions, I'll try my best to help.