Nelson,
Brian got back to us all about the PCIe M.2 SSD that he has up and running in his Z600 workstations, which should work fine as a bootable drive in the Z400, Z600, Z800 workstations (and , I believe, in both the Version 1 and Version 2 types of each). He explains more and shows it in the boot drive section of BIOS HERE.
Regarding your question.... the fastest your 2.5" form factor SSD can run in these ZX00 workstations is limited by their being SATA Gen II workstations just like the xw6400/8400, xw6600/8600, and the xw4600, even if you use a SATA Gen III SSD. So, cut that 6GB/s speed in half to 3GB/s speed, and compare it to the estimated speed of 10GB/s if you go for the M.2 PCIe SSD Brian has up and running on his.
It comes down to balance..... if all your components are fast I believe you will perceive a significant change. If you have a slow processor and slow memory and slow video card not so much. For intensive drive I/O chores then there will surely be a big difference.