Relocate defect HDD Sectors of an iMac 27" Fusion Drive

I was facing a problem in one of my VMWare Fusion VMs I used for many years on an encrypted OSX High Sierra Fusion Drive (1TB HDD, 120GB SDD) on my 27" iMac. It led to VM crashes and HDD I/O error were reported in the OSX logs when I tried to reinstall a program in the VM.

I used SMARTReporter to perform a SMART long test, which failed with a SMART Value 197 (Current_Pending_Sector) count of 40, which means the drive internal SMART logic detected 40 defect sectors.

Note: You can test the SMART status of your drives either by using the OSX SMARTReporter tool (see App Store), or by installing the smartmontools using HomeBrew and performing "smartctl -a /dev/disk1" (disk1 is the HDD in my case. Use "diskutil list" to get your drive nodes).

A few people think the drive has some "magic" to relocate defect sectors automatically. WRONG. Modern SATA HDDs relocate defect sectors to spare sectors only on WRITE to such a defect sector.

So my question was: How can I detect and forcibly write to such a defect sector, so the HDD relocates the sector to a good one?

Using OSX High Sierra tools, I didn't find any possibility to write the sectors.
Therefore I booted Ubuntu from a prepared USB Stick and repaired the defect sectors there.
 
Prerequisites: Install HomeBrew if you do not have it installed in your OSX already.

Note:
The next steps can also be performed if your drive has defect sectors and you run any other OS (Linux / Windows, etc). If you are on Linux, you can skip Steps 1-2.

Warning!
You are writing dummy data to your HDD, so expect data loss. You perform the next steps on your own risk.
MAKE A BACKUP OF YOUR DRIVE TO ANOTHER DRIVE (TimeMachine, CarbonCopyCloner, etc.) before you start these steps. YOU ARE WARNED.

It may be wise to send your Mac to repair, or to replace the defective HDD with a new one instead of performing the next steps.

1. Install Ubuntu on an USB Stick

See https://tutorials.ubuntu.com/tutorial/tutorial-create-a-usb-stick-on-macos how to create such a bootable USB Stick

2. Boot Ubuntu from the USB Stick

For this, restart your Mac, press the Option (alt) Key until the boot tone occurs and select the "Efi" Drive. In the Ubuntu boot loader, start the Live version.
Note: If you use Magic Mouse / wireless keyboard, they are not automatically connected to Ubuntu. Best is you connect an USB keyboard and pair the Bluetooth Keyboard / Mouse to Ubuntu (See Preferences > Bluetooth)

3. Install needed tools in Ubuntu

We need some tools to fix the hdd. Open a terminal and enter:

    sudo bash
    apt install smartmontools sg3-utils

4. Start SMART long selftest

    #> smartctl -t long /dev/sda

5. Check SMART test progress/errors

    #> smartctl -a /dev/sda

If it prints something like this, it detected sector errors:

# 1 Extended offline Completed: read failure 90% 25836 1370708040

There we see the defect sector around LBA 1370708040.

Note:
   This mustn't be the exact error location, therefore we check this sector and the others behind it.

6. Try to read the defect sector

    #> hdparm --read-sector 1370708040 /dev/sda

Alternatively, use the sg-utils to read the sector:

       #> sg_verify --lba=1370708040 /dev/sda

If this reads something like "bad/missing sense data" or errors like this, the sector is defect.
So it is time to write the sector, so the HDD relocates the bad sector to a good one (Data at that sector is definitely lost)

4. Write bad sector (Relocate sector)

If 6. lead to sense errors, we try to write the defect sector, so it gets relocated:

    #> hdparm --yes-i-know-what-i-am-doing --write-sector 1370708040 /dev/sda

5. Check sectors behind the defect one

It is a good idea to check the sectors behind the defect one for errors as well, as there is a good chance they are defect as well.

    #> hdparm --read-sector 1370708041 /dev/sda


Repeat steps 5 and 4 on incremented sector numbers as long as there are no more unreadable sectors.
When you reach a zone where the sectors are ok again, repeat step 1 until smart checked the whole disk.

6. Boot into OSX and perform Disk repair

Reboot into OSX, start the Disk Utility and check the drive for errors.
If OSX reports unrepairable errors, you can try to fix it by performing an fsck in single-user mode. At this stage, you are out of scope of this document.

Good luck.

Import GitHub Enterprise into VMware vCenter 6.5

I wanted to try GitHub Enterprise 2.8.7 in my vCenter 6.5 env, but the OVF import always cancelled with an error message that the ProductInfo is not allowed in the envelope.

It seems the GitHub OVF template was created with a fairly old ovftool.

Fix:

1. Unpack the OVF (it's a ZIP file)
2. Edit the .ovf file and move the "ProductSection" XML Element to the <VirtualSystem> node. See this Gist.
3. Afterwards, re-compute the SHA1 fingerprint of the .ovf file and update the .md file with the new fingerprint
4 Re-package the files into a new .ovf zip archive.

After this change, it worked here to import he GH Enterprise OVF.