The development of low-loss reconfigurable integrated optical devices enables further research into technologies including photonic signal processing, analogue quantum computing, and optical neural networks. Here, we introduce digital patterning of coupled waveguide arrays as a platform capable of implementing unitary matrix operations. Determining the required device geometry for a specific optical output is computationally challenging and requires a robust and versatile inverse design protocol. In this work we present an approach using high speed neural network surrogate based gradient optimization, capable of predicting patterns of refractive index perturbations based on switching of the ultra-low loss chalcogenide phase change material, antimony tri-selinide (Sb 2 Se 3 ). Results for a 3×3 silicon waveguide array are presented, demonstrating control of both amplitude and phase for each transmission matrix element. Network performance is studied using neural network optimization tools such as dataset augmentation and supplementation with random noise, resulting in an average fidelity of 0.94 for unitary matrix targets. Our results show that coupled waveguide arrays with perturbation patterns offer new routes for achieving programmable integrated photonics with a reduced footprint compared to conventional interferometer-mesh technology.