Try to set syntax off:
Vim tries to keep your work safe and doesn't assume you can type several thousand characters per second. Read :help swap-file for some details on the buffering. The solution to your problem is this:
Turn off vim's swapfile while you are pasting text:
Turning off swap is not safe for normal operations! Immediately after the paste:
See :help swapfile for more details.
Folding could be the problem:
set foldenable " can slow Vim down with some plugins set foldlevelstart=99 " can slow Vim down with some plugins set foldmethod=syntax " can slow Vim down with some plugins
According to vim's help (:help foldmethod), the syntax setting caused vim to use syntax highlighting to determine how to automatically fold my code.
Other things to check/toggle are syntax, filetype, wrap and line length (some plugins can be slow with very long lines).
Running Vim without your current settings is a good starting point:
# vim -u NONE