Finite $p$-groups whose non-abelian subgroups have the same center