Error message "Mage registry key already exists" after extension installation

After installing a Magento 1 extension or Magento 2 extension you get the following error message, e.g. if you try to login the Magento admin backend:

Mage registry key <...> already exists

The solution is in most cases very easy:

Please confirm that you followed all installation steps that are mentioned in the README file of the module. Especially assure that the Magento cache is cleared, e.g. by emptying the var/cache/ directory in a Magento default installation and that the Magento compilation process has been triggered (if enabled).