When importing a certificate and private key in Windows (e.g. from a PFX file), you are given the option to mark the key as exportable. If this is not ticked, it is not possible to export the private key at a later date.

The below instructions provide a method of extracting the private key into a PFX file.

On the server with the private key

Follow the below instructions, where A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0 is the thumbprint of the certificate.

  1. Launch an elevated PowerShell prompt
  2. Run the following commands to identify the file where the private key is stored:
    PS C:\Windows\system32> $a = Get-Item Cert:\LocalMachine\My\A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0A0
    PS C:\Windows\system32> $a.PrivateKey.CspKeyContainerInfo.UniqueKeyContainerName
    50ed65430216d17c6e6efff6819c923b_92a9b8e0-fce8-4402-8b90-66196ad1d0d1
    
  3. Use PsExec to spawn a command prompt in the SYSTEM context
    C:> PsExec64.exe -s -i cmd
    
  4. Extract the private key data.
    C:> copy "C:\Users\All Users\Microsoft\Crypto\RSA\MachineKeys\50ed65430216d17c6e6efff6819c923b_92a9b8e0-fce8-4402-8b90-66196ad1d0d1" c:\
    

On a non-production computer

  1. On a non-production computer import the public certificate part (.cer/.crt). Copy the file from above to C:\ .

  2. From a PowerShell prompt, find the GUID that represents this computer.

    PS C:\Windows\system32> Get-ItemPropertyValue Registry::HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Cryptography\ -Name MachineGuid
    2801936f-1239-4daa-89e5-f78df0ae0f2a
    
  3. Rename the file you placed in C:\ so that the value after the underscore (_) matches the MachineGuid value. In our example, the file will now be named 50ed65430216d17c6e6efff6819c923b_2801936f-1239-4daa-89e5-f78df0ae0f2a

  4. From a PowerShell prompt, move the file to the certificate store on this machine

    PS C:\Windows\system32> Move-Item "C:\50ed65430216d17c6e6efff6819c923b_92a9b8e0-fce8-4402-8b90-66196ad1d0d1" "C:\Users\All Users\Microsoft\Crypto\RSA\MachineKeys\50ed65430216d17c6e6efff6819c923b_2801936f-1239-4daa-89e5-f78df0ae0f2a"
    
  5. From an elevated cmd prompt, run the following, where AA73… is the serial-number of the certificate.

    C:> certutil -repairstore my AA73A8D8B69122DB7A861257400E52E4C14E39E5
    
  6. Check the local machine certificate store you will notice a padlock icon against the certificate, indicating the private key is available. You can now export the certificate with the private key.

This method has been repeatable for me, but may not work for you if the certificate private key is stored off-disk, such as a SmartCard.