There are people on here with way more knowledge than me, but here we go:
You could use your 2 screens with the 560ti. Depends on the resolution of your screens - the 560ti has either 1 GB ram or 1,25 Gb ram. You could download the free process explorer here
https://technet.microsoft.com/en-us/sys ... 96653.aspx , enable GPU info and take a look how much of your GPU ram is consumed. And download gpu-z to watch the GPU usage of your current workflow
https://www.techpowerup.com/gpuz/ to see if it makes sense for you.
the hookup of your screens to the 560ti is easy. use a pci-e slot in your motherboard that doesn't affect the airflow of the 980ti (or switch positions, depends on your motherboard), connect the 2 6-pin power connectors and then connect your monitors (just pull the cables from the 980ti and plug them into the 560ti. Obviously do it while the computer is switched off and remove the power cable.
Yes, there are priority settings in Octane - that means, you can enable / disable video cards, set priorities on them (useful, if you have to work while rendering. - For example: if you have 2 video cards in your setup and you have to render in C4D, but still have to work in after effects, you could enable both GPUs, but set the priority on your display card (which is used for rendering, too) to low or medium. Or disable the display card for Octane and only render on the second one. and so on.)