Replacing drive on a DE6600 with drive from a E2660 - possible?

[agapotron]

We have a DE6600 unit that we have been using as a JBOD for a while. It has been working just fine, with the head unit running ZFS. We started seeing errors on one of the drives, so we figured we would replace the drive with a drive from one of our E2660 units (we have plenty of those drives lying around).  But it seems that there's something that prevents us from using those drives as replacement. When we use a drive (that has been erased and initialized), it recognizes the drive with no problems, but eventually it starts giving errors like these:

[351024.356756] sd 1:0:124:0: [sday] tag#9457 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE

[351024.356766] sd 1:0:124:0: [sday] tag#9457 Sense Key : Aborted Command [current] [descriptor]

[351024.356771] sd 1:0:124:0: [sday] tag#9457 Add. Sense: Logical block reference tag check failed

[351024.356775] sd 1:0:124:0: [sday] tag#9457 CDB: Read(32)

[351024.356780] sd 1:0:124:0: [sday] tag#9457 CDB[00]: 7f 00 00 00 00 00 00 18 00 09 20 00 00 00 00 00

[351024.356784] sd 1:0:124:0: [sday] tag#9457 CDB[10]: 00 00 02 00 00 00 02 00 00 00 00 00 00 00 00 08

[351024.356787] print_req_error: 8 callbacks suppressed

[351024.356790] print_req_error: protection error, dev sday, sector 512 flags 0

At first, I thought maybe we just had another bad drive, but I have used at least 4 different known good drives as replacements, and they all do the same thing. So, is there a magic incantation to make an old drive from a E2660 work on a DE6600? The drives seem to be exactly the same brand, version number, firmware revision, etc. 

I don't have much faith that I will get an answer on this board, but at this point I'm pretty much out of other options. Thanks for any helpful suggestions...

[NetApp_RZ · Accepted solution]

Hello,

Given this line below, it indicates the drive you are using is throwing DA / T10PI errors due to the reference tag being mismatched.

[351024.356771] sd 1:0:124:0: [sday] tag#9457 Add. Sense: Logical block reference tag check failed

Not sure what exact drive model and firmware you have but if the drive is truly a DA enabled drive, it is low level formatted at 520 bytes per sector to include the extra space for the DA / T10PI information.

Odds are real good that your host only formatted/cleaned the standard 512 bytes per sector thus leaving the 8 bytes of T10/PI information untouched which is now no longer is in agreement with the data on disk so that when it is read, we have a mismatch.

Certainly will be some odd behavior when trying to use a DA enabled drive in a non DA enabled / aware application.
Whether the drive is Seagate or Hitachi, the vendors use special procedures to format the disks with 520 bytes per sector from the factory to assure the 8 bytes of T10/PI information are properly initialized as well.

Essentially a low level format.

A quick overview of PI is below from Seagate.
https://www.seagate.com/files/staticfiles/docs/pdf/whitepaper/safeguarding-data-from-corruption-technology-paper-tp621us.pdf

[agapotron]

Thanks for the information about the low level format. In my case, being on a linux system, I was able to do the process using the sg_format command. In particular, I compared the result of running the command on the replacement drive vs. running the command on an existing drive:

  sg_format -v /dev/sday

  sg_format -v /dev/sdaz

The only difference was with the protection level, as the existing disks reported the following:

  Protection: prot_en=1, p_type=1, p_i_exponent=0 [type 2 protection]

So, after reading the manual page for sg_format, I ended up running the following command, which *erases and reformats the whole drive*, but solved my problem. It took almost a full day to complete the reformatting:

 sg_format --format --fmtpinfo=3 --pfu=0 /dev/sday