A Labour mayor has announced he will not stand for re-election and said his four-year tenure had taken a "very heavy toll".