We describe an experimental system for parallel distributed processing of numerical algorithms. Computations are performed on GPUs, and communication is facilitated using FPGA-adapters and self-developed interconnects. As case study we have implemented a conjugate gradient solver for the Poisson problem in three dimensions. The work focuses on machine architecture and distributed processing with special emphasis on alleviating communication bottlenecks. Also we present the implementation of a matrix-free preconditioner for the Poisson problem which does not add communication overhead.