Diff mode arranges for hunks to be treated as compiler error messages by <del>@kbd{C-x `}</del> <ins>@kbd{M-g M-n}</ins> and other commands that handle error messages (@pxref{Compilation Mode}).  Thus, you can use the compilation-mode commands to visit the corresponding source locations.