The relationship between functional and structural connectivity strength in the brain remains uncertain. We compared high-field resting-state fMRI, diffusion-based tractography and neuronal tracer data to robustly characterize the rat connectome. Our study revealed that strong structural connectivity is not required for strong functional connectivity. We found distinct structure-function relationships at different hierarchical levels in the rat brain: functional connectivity strength correlated moderately with diffusion-based structural connectivity strength, but did not significantly correlate with neuronal tracer-based structural connectivity strength. Hereby we demonstrate the importance of examining or appraising connectivity at different hierarchical levels for reliable assessment of neural network organization.