Orientation distributions estimated from diffusion MRI have multiple peaks per voxel. Tractography algorithms must choose among them arbitrarily, leading to errors. We propose a novel approach to making this choice in a manner informed by the data. We use post mortem optical and MRI data to train a convolutional neural network that can recognize voxel-wise connection patterns directly from diffusion data, circumventing the conventional paradigm of an orientation distribution. We introduce TRACARIS (TRACt Architectures Recovered from Imaging Signals), a tractography algorithm that uses these network-predicted, local connection patterns. We present preliminary validation results from a post mortem human brain sample.