aloh,
Choosing the GPU is first based on the use, for example, 3D modeling, simulation, computing, rendering, gaming, dtabase, and etc ., then selected according to the the specifications including: GPU type, memory capacity, clock speed, and power use, match the PSU capability to the computer, the cooling capacity, and the various choices to the budget. As mentioned it's possible to have a GPU that can not be fully utilized. I have a 2007 Dell Precision 390 Xeon X3230 4C@2.66GHz /375@ PSU and I tried a Quadro K4200 in it- listed as needing a 450W PSU minimum and the K4200 did better than expected, but was about 25% under the performance in a z420 /E5-1660 v2 system (6-cores @ 4.2GHz).
Here's a chart that is useful with regards to NVIDIA Quadro and GTX as it is includes listing according to the minimum power supply:
https://www.studio1productions.com/Articles/NVidia-GPU-Chart.htm
Compare the relative benchmark performance with this searchable chart:
https://www.videocardbenchmark.net/gpu_list.php
In general, the newer the GPU, the more effieicent it for power, espcially from 7XX GTX and newer, and Quadro M-series and newer. As the z600 has a 625W PSU, you can see that some very fast GPU's are possible.
Used GTX that I think have a very good cost /performance ratio are the GTX 770 ( budget), 970, 980Ti (< at one time the No.1 performer), and 1070 Ti (I'm trying one of these these last few days), and for Quadros: K4200, P1000 and P2000 (which I use in z620_2). If you look at the searchable GPU chart, you see that the Quadro P2000 has an average 3D of 7815 and the listing immediately below is the Titan Z at 7754. Of course the Titan Z is king of CUDA with 5760 cores, is a 384-bit card, and has 12GB, but the memory is DDR5 not GDDR5 or GDDR5X, and clock speeds are not fantastic. If I narrowed it down for best cost /performance, I'd say GTX 1070 Ti (500W PSU) and Quadro P2000 (600W PSU). Either of those two could be moved along to a system two generations newer- z440/640 for example.
What kind of applications are you using?
BambiBoomZ