Posted on 41 Comments on ESX 6.5 and 6.7 on HPE G6/G7 server PSOD fix

ESX 6.5 and 6.7 on HPE G6/G7 server PSOD fix

Background

HP’s done it again. They’ve managed to break their custom ESX ISO on the G6 and G7 servers. I suspect it’s the same for G8.

If like me you found yourself perplexed by the PSOD (pink screen of death) after upgrading from ESX 6.0 to 6.5, keep on reading for the fix (skip to the bottom for the download link to an already fixed ISO).

The issue

It seems the “hpe-smx-provider” driver version 6.5.0 from the ESX 6.5 ISO is causing the PSOD.

The fix and the drawbacks

(Simply) Replace the driver with it’s older counterpart (version 6.0.0) and reinstall (see below for the procedure or check the “download link” section at the very end for an already fixed ISO).  Unfortunately if you decide to create a custom ISO with the previous version of “hpe-smx-provider”, you will no longer be able to upgrade to any future ESX version and will need to do a full installation every time (thank you HP!). That being said, here is the procedure to customize the ISO:

Customize your own HPE ESX ISO

  1. Install PowerCLI from here.  At the time of this writing, the latest version was 6.5.0R1.
  2. Download both the ESX 6.5 and 6.0 offline bundles and save them to a convenient place (ex: C:\HP) — download from this link. To make it easier, rename them to something like HPE_ESX6.5.zip and HPE_ESX6.0.zip.
  3. Open up PowerCLI and do the following:Add the 6.5 bundle 
    cd C:\HP
Add-EsxSoftwareDepot -DepotUrl HPE_ESX6.5.zip

    # check that the profile was loaded and the vendor is HPE

    Get-EsxImageProfile

    # clone the profile (feel free to specify whatever you like for the “vendor” when prompted)

    New-EsxImageProfile -CloneProfile HPE-ESXi-6.5.0* -Name "HPE-ESX-6.5-CUST"

    # double check the profile was indeed cloned

    Get-EsxImageProfile

    # Remove the “hpe-smx-provider” driver from the clone

    Remove-EsxSoftwarePackage HPE-ESX-6.5-CUST hpe-smx-provider

    # Add the 6.0 bundle

    Add-EsxSoftwareDepot -DepotUrl HPE_ESX6.0.zip

    # check the profile

    Get-EsxImageProfile

    # check driver versions from both bundles

    Get-EsxSoftwarePackage | findstr smx

    # Add hpe-smx-provider version 6.00 to the custom profile

    add-esxsoftwarepackage -imageprofile HPE-ESX-6.5-CUST -softwarepackage "hpe-smx-provider 600.03.11.00.9-2768847"

    # Export the profile to an ISO

    Export-EsxImageProfile -ImageProfile HPE-ESX-6.5-CUST -ExportToIso -filepath "HPE-ESX-6.5-CUST.iso"

The results

Download links

The following images have hpe-smx-provider version 600.03.11.00.9-2768847. Nothing else was changed. The 6.5.0d (build 5310538) is running on an HP Proliant DL380 G7 since March 2018 and was also tested on a DL380 G6 for a couple of months. For convenience, I’ve also built a 6.7.0 ISO and at the request of a reader, 6.7.0 Update1 from April of 2019:

VMware-ESXi-6.5.0-5310538-HPE-650.10.1.5.20-Oct2017_CUSTOM.iso
VMware-ESXi-6.7.0-9484548-HPE-Gen9plus-670.10.3.5.6-Sep2018_CUSTOM.iso
VMware-ESXi-6.7.0-Update1-11675023-HPE-Gen9plus-670.U1.10.4.0.19-Apr2019_CUSTOM.iso

Enjoy!

41 thoughts on “ESX 6.5 and 6.7 on HPE G6/G7 server PSOD fix”

  2. Hi
    thank you so much for your information and the custom ISO
    i installed esxi custom iso 6.7 on the HP G8 successfully and without any error or crash, But i have a question, what about esxi update?
    i was trying to update esxi 6.7 to the latest version with command “esxcli”, i got following error:

    [dependencyerror] vib qlc_bootbank_qedrntv_3.9.18.0-1oem.670.0.0.7535516 requires qedentv_ver = x.9.18.0, but the requirement cannot be satisfied within the imageprofile. please refer to the log file for more details.

    i was resolved the problem and updated esxi by this solution:
    http://vcdx56.com/2017/09/update-esxi-fails-with-dependencyerror-requirement-cannot-be-satisfied-within-the-imageprofile/

    so, i wanted to ask you this solution is it ok or not?

    thanks again

    Reply

    1. The profile is a custom one, due to the “hpe-smx-provider” package downgrade and that is why you get that error message. Because of that, no upgrades can be made in the traditional way. Your only option is to create a custom ISO every time you want to upgrade. HP really screwed us over this time around.

      Reply

  3. Any Chance to have a customized ISO for the latest May2019 Release of ESX 6.7 ?

    Reply

    1. Hi,

      I will create one in the following days and reply again to your comment.

      Reply

  4. Hi,

    I have a question; i updated to the April version Custom ISO (6.7), but i do not have the HTML5 update of the web client.

    Why is that?

    Thanks

    Kind Regards,
    Danny

    Reply

    1. It’s probably not included in the original ISO. The only change the custom image has is the 1x driver.

      Reply

      1. Do you know if it is included in one of the other images?

        I know that the original 6.5 has the HTML 5 web client, right?

        Kind Regards,
        Danny

        Reply

        1. No idea if it’s included on the other images or not since I can’t run 6.7 on my DL380 G7 due to unsupported CPU.

          Yes, the HTML 5 client is included in the 6.5 image.

          Reply

  5. Thanks for preparing the ISO. I’ve tried to update from 6.5 to 6.7U1 on a HP DL 380 G7 yesterday. It worked in the end but I ran into some problems. At first i tried to update, so I mounted the ISO as a virtual DVD-Drive using ILO and was able to boot from it. The install aborted at first telling me there was a caching issue. I made sure settings for Swap and Scratch were set properly to the local datastore (ESXI itself runs from a SD-Card) and tried again. This time the installation went up to 78% but then stopped with a generic failure I don’t even remember. At first I didn’t really worry about it, I had a configuration backup (created using the “vim-cmd hostsvc/firmware/backup_config” command), so I did a clean install and tried to restore using the “vim-cmd hostsvc/firmware/restore_config” command.
    This also failed and finally made me worry. A common error seems to be a different UUID which wasn’t the case as this was the same machine. I tried a couple of times and finally gave up.
    In the end I found out (related blogpost: https://768kb.wordpress.com/2014/06/26/esxi-config-backup-restore/), actually its not so hard to do a manual restore without any ESXi cli-tool, Power CLI or vSphere functionality. It’s enough, at least it was for me, to extract the local.tgz file from the state.tgz in the backup file to the root of your ESXI host. After I did this and replaced the files with the ones from the backup my host came back up with the old configuration and everything worked again. In the end this took me about 3 hrs, far from the 30 mins I hoped for…

    Reply

  6. Any Chance to have a customized ISO for the HP ESXi 6.7 U2 ?

    Best Regards

    CS

    Reply

  7. Hi !!!
    Any Chance to have a customized ISO for VMware-ESXi-6.7.0-Update2-13006603-HPE-Gen9plus-670.U2.10.4.1.8-Apr2019.iso ?

    thanks in advance

    CS

    Reply

  8. I’m currently unable to download the 6.7U2 offline bundle from the official VMWare website. I’ll give it a try later on and let you know.

    PS: You can also customize the ISO yourself by following the tutorial.

    Reply

  9. Question:

    I’m trying to do this for the latest ESXi 6.7 U2 (also from April 2019) from the HPE link you provide and it doesn’t find hpe-smx-provider:

    WARNING: Package hpe-smx-provider could not be found in image profile HPE-ESX-6.7-CUST, not removing.

    But if I execute Get-EsxSoftwarePackage | findstr smx it finds “smx-provider”:

    smx-provider 670.03.15.00.3-7535516 HPE 08/10/2018 11…

    Is it a rename or this version already is fixed? What to do?

    Reply

    1. If you downloaded the ISO from the official website, it might be that HP renamed the package. In that case, simply replace that with the 6.0 version.

      Reply

  10. Hi,
    Can I upgrade esxi 6.0 to 6.7 using VCSA Upgrad, or have to be trough cli and mounted image??

    Reply

  11. I tried your “VMware-ESXi-6.7.0-Update1-11675023-HPE-Gen9plus-670.U1.10.4.0.19-Apr2019_CUSTOM.iso” in HP Proliant DL360 G7. It was pretty OK, but some of the system sensors (mainly System Chassis 1 UID Ligh and System Chassis 2 Sys. Health LED) were unknown. Then the whole Storage hw monitoring (including HPE Smart Array P410i) seemed to be unknown. Maybe some more packages than just hpe-smx-provider should be replaced? Or is that just mission impossible to do 100% esxi 6.7U1 for G7?

    Reply

  12. i’m trying to get the 10GB nic to work with esxi 6.0 or 6.5 ot 6.7
    does this custom iso fix it the 10gb nic?

    Reply

  13. hi
    successfully installed 6.7 update 3 hpe image without any modification on ml350 g6.
    compared with 6.7: lost some hardware sensors (fans, power meter) and there is a warning alert about “system board 10 memory”. downgrading smx didn’t solve issues.
    best regards

    Reply

  14. Hie, can you do the same for ESXI 7.0, I have an HP Proliant G6 Server working fine with your ESXI 6.7 image running brilliantly and looking forward to test 7.0 soon. Thanks in advance.

    Reply

      1. Wondering if you have this for 7.0 yet? Your other images have work very well for me! Thanks!

        Reply

    2. did you manage to solve this problem? i faced the same error and nothing helps.

      Reply

    3. Install iLO version 1.88 and BIOS 2018.02.22.
      With newer versions I was getting the same error

      Reply

  16. Recently came across your blog, I’m going to take a stab at building/testing a custom image from your tutorial for a fresh install to 7.0U1 on my DL360 G8. I applaud you for being very accommodating to your commenters and building custom images. I am currently running 6.7U3 upgraded via CLI from the original 6.5U3 HPE image. It has been stable for .

    Reply

  17. Hello,

    If I want to install esxi 6.5 update 3 on G6. I should download esxi 6.5 u3 and esxi 6 and change as you mentioned on esxi 6.5 u3 iso? Is that right? I think the lastest esxi version is base for any hp servers. Right?

    Reply

  18. Hi,
    I have a hp dl980 g7 and hp ml350p g8 which is installed ESXi 6.0 version and working properly with no concern now.
    I need to upgrade ESXi 6.0 to 6.5u3 and I created a custom iso which is mentioned above and because of the PSOD issue I didn’t upgrade the ESXi yet, and I made a customized iso and changed the hpe-smx-provider 6.5 with 6.0.
    So if my host working properly with ESXi 6.0 now (did not upgrade to 6.5 because of PSOD known issue), can I upgrade esxi 6.0 to 6.5 with customed iso or do I have to fresh install anyway?

    Reply

  19. Trying to make a custom bundle for 6.7U3:

    PowerCLI C:\users\ciprr008\Downloads> Remove-EsxSoftwarePackage HPE-ESX.6.7-CUST hpe-smx-provider
    WARNING: Package hpe-smx-provider could not be found in image profile HPE-ESX.6.7-CUST, not removing.

    So is the hpe-smx-provider no longer there?

    Reply

    2. Hi,
      for me it works but i got an error on this step too.
      In my case it wasd a “copy and paste ” Problem :D.
      When i paste the code in the cli shell it removes a Space Key betwen “Remove-EsxSoftwarePackage HPE-ESX-6.7-CUSThpe-smx-provider” it must be “Remove-EsxSoftwarePackage HPE-ESX-6.7-CUS Thpe-smx-provider” afrer setting the “space” Key it works for me.

      At the end i got another error on creating the iso File “Export-EsxImageProfile : Could not find a trusted signer.” and to fix that i addet the Parameter “-NoSignatureCheck” at the end. And now i have a 6.7 U3 ISO
      But haven not tested the installation at that moment…

      Thanks for this Article it helps me a lot!

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *