Holgorsen refuses to talk conference realignment