I've found it helpful to use relative numbering in normal mode and switch to absolute numbering in insert mode:
set relativenumber nonumber
autocmd InsertEnter * :set norelativenumber number
autocmd InsertLeave * :set relativenumber nonumber
IIRC this tip was from the Practical Vim book