Mailing List Archive

Used Replacement disk not recognized
Hey everybody,

I just had the pleasure to try to replace a faulty disk in a filer which is out-of-service with a "new" one from after market.
Of course, this didn't work out as easy as it should, so maybe some could help me get this disk going.

Here are the facts:
FAS2040
NetApp Release 8.1.4P8 7-Mode

The disk at 0d.01.13 was broken, so I pulled it a few days ago - now that the replacment disk has arrived, I tried to plug it in and this is what happens:

Thu Oct 22 14:30:10 CEST [riddler:scsi.cmd.notReadyCondition:notice]: Disk device 0d.01.13: Device returns not yet ready: CDB 0x1b: Sense Data SCSI:not ready - Drive spinning up (0x2 - 0x4 0x1 0x28)(2).

The interesting thing, I can see the disk by using `storage show disk`:

riddler> storage show disk
DISK SHELF BAY SERIAL VENDOR MODEL REV
--------------------- --------- ---------------- -------- ---------- ----
0d.01.13 1 13 J838WSRL NETAPP X302_HJUPI NA04

But I cannot see it in disk show.

riddler> disk show 0d.01.13
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------

Sysconfig -v tells me, that there is a newer firmware on the "new" disk:

01.13: NETAPP X302_HJUPI01TSSM NA04 847.5GB (1953525168 512B/sect)
01.14: NETAPP X302_WVULC01TSSA NA03 847.5GB (1953525168 512B/sect)

So I first tried to downgrade firmware using:

disk_fw_update -R 0d.01.13

but that did not really work out, no error message, but just no change on the firmware of the disk - also nothing in the logs.
So I read somewhere that this might be due to a bad/incompatible label on the disk and found out that the only way to get rid of the label is to boot into maintenance mode (thus having a downtime of parts of the HA-pair :-/) and recreate the label there.

So I booted into maintenance mode and have run the following command:

*> label makespare 0d.01.13
*********************************************************
Label editing is highly dangerous and should be attempted
ONLY by NetApp Customer Support Engineers.
Misuse can cause ALL data to be permanently lost.
*********************************************************
'Label save': Error while trying to save Label 1 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 1.
'Label save': Error while trying to save Label 2 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 2.
The current set of labels on the disks have been saved. You can
use the 'label restore' command to revert to these labels, the
'label keep' command to erase these labels, or 'label save' to
preserve the current set of labels.
*********************************************************
*** Run WAFL recovery tools (wafliron or wafliron_oc) after this. ***

Disk 0d.01.13 does not exist.

The last line suggests, that the disk still does not get recognized properly - and additionally, it (for whatever reason) claims to not be able to write the label 1 on disk 0d.03.22, although this is there and online:

riddler> disk show 0d.03.22
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------
0d.03.22 riddler (142248200) Pool0 N002XRML riddler (142248200)

I've stopped my expiriments at this point and am very thankful for fresh ideas :)

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Re: Used Replacement disk not recognized [ In reply to ]
This is not due to a bad label. You would have seen a message when taking
ownership of the disk about a bad label if that were the case. The disk
would also show up in the failed disk pool (aggr status -f) with a bad or
invalid label tag on it (I can't remember exactly what it says). The bad
label only comes in to play if a disk was in a higher version of Data
ONTAP, and was not marked as a spare, zeroed, and then ownership removed
prior to inserting it in your system. On your version of ONTAP the label
version is 11. So it would have had to have been in a system running 8.2
or higher.

Regarding 0d.03.22, is that disk a spare? What does your "sysconfig -r"
output look like?

On Thu, Oct 22, 2015 at 10:06 AM, Alexander Griesser <
AGriesser@anexia-it.com> wrote:

> Hey everybody,
>
>
>
> I just had the pleasure to try to replace a faulty disk in a filer which
> is out-of-service with a „new“ one from after market.
>
> Of course, this didn’t work out as easy as it should, so maybe some could
> help me get this disk going.
>
>
>
> Here are the facts:
>
> FAS2040
>
> NetApp Release 8.1.4P8 7-Mode
>
>
>
> The disk at 0d.01.13 was broken, so I pulled it a few days ago – now that
> the replacment disk has arrived, I tried to plug it in and this is what
> happens:
>
>
>
> Thu Oct 22 14:30:10 CEST [riddler:scsi.cmd.notReadyCondition:notice]: Disk
> device 0d.01.13: Device returns not yet ready: CDB 0x1b: Sense Data
> SCSI:not ready - Drive spinning up (0x2 - 0x4 0x1 0x28)(2).
>
>
>
> The interesting thing, I can see the disk by using `storage show disk`:
>
>
>
> riddler> storage show disk
>
> DISK SHELF BAY SERIAL VENDOR MODEL REV
>
> --------------------- --------- ---------------- -------- ---------- ----
>
> 0d.01.13 1 13 J838WSRL NETAPP X302_HJUPI NA04
>
>
>
> But I cannot see it in disk show.
>
>
>
> riddler> disk show 0d.01.13
>
> DISK OWNER POOL SERIAL NUMBER HOME
>
> ------------ ------------- ----- -------------
> -------------
>
>
>
> Sysconfig –v tells me, that there is a newer firmware on the „new“ disk:
>
>
>
> 01.13: NETAPP X302_HJUPI01TSSM NA04 847.5GB (1953525168
> 512B/sect)
>
> 01.14: NETAPP X302_WVULC01TSSA NA03 847.5GB (1953525168
> 512B/sect)
>
>
>
> So I first tried to downgrade firmware using:
>
>
>
> disk_fw_update –R 0d.01.13
>
>
>
> but that did not really work out, no error message, but just no change on
> the firmware of the disk – also nothing in the logs.
>
> So I read somewhere that this might be due to a bad/incompatible label on
> the disk and found out that the only way to get rid of the label is to boot
> into maintenance mode (thus having a downtime of parts of the HA-pair :-/)
> and recreate the label there.
>
>
>
> So I booted into maintenance mode and have run the following command:
>
>
>
> *> label makespare 0d.01.13
>
> *********************************************************
>
> Label editing is highly dangerous and should be attempted
>
> ONLY by NetApp Customer Support Engineers.
>
> Misuse can cause ALL data to be permanently lost.
>
> *********************************************************
>
> 'Label save': Error while trying to save Label 1 on disk 0d.03.22: error
> 15.
>
> Label save: 1 of 35 disks could not save Label 1.
>
> 'Label save': Error while trying to save Label 2 on disk 0d.03.22: error
> 15.
>
> Label save: 1 of 35 disks could not save Label 2.
>
> The current set of labels on the disks have been saved. You can
>
> use the 'label restore' command to revert to these labels, the
>
> 'label keep' command to erase these labels, or 'label save' to
>
> preserve the current set of labels.
>
> *********************************************************
>
> *** Run WAFL recovery tools (wafliron or wafliron_oc) after this.
> ***
>
>
>
> Disk 0d.01.13 does not exist.
>
>
>
> The last line suggests, that the disk still does not get recognized
> properly – and additionally, it (for whatever reason) claims to not be able
> to write the label 1 on disk 0d.03.22, although this is there and online:
>
>
>
> riddler> disk show 0d.03.22
>
> DISK OWNER POOL SERIAL NUMBER HOME
>
> ------------ ------------- ----- -------------
> -------------
>
> 0d.03.22 riddler (142248200) Pool0 N002XRML
> riddler (142248200)
>
>
>
> I’ve stopped my expiriments at this point and am very thankful for fresh
> ideas J
>
>
>
> Best,
>
>
>
> *Alexander Griesser*
>
> Head of Systems Operations
>
>
>
> ANEXIA Internetdienstleistungs GmbH
>
>
>
> E-Mail: AGriesser@anexia-it.com
>
> Web: http://www.anexia-it.com
>
>
>
> Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
>
> Geschäftsführer: Alexander Windbichler
>
> Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT
> U63216601
>
>
>
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> http://www.teaparty.net/mailman/listinfo/toasters
>
>
Re: Used Replacement disk not recognized [ In reply to ]
One thing to look at...
Do you have the latest disk qualification package?
Doubt that's it though. It looks like the disk is not properly spinning up... You replaced a bad drive with a not so good drive 😢
--tmac

Sent from Mobile Outlook




On Thu, Oct 22, 2015 at 7:10 AM -0700, "Alexander Griesser" <AGriesser@anexia-it.com> wrote:


















Hey everybody,


 


I just had the pleasure to try to replace a faulty disk in a filer which is out-of-service with a „new“ one from after market.


Of course, this didn’t work out as easy as it should, so maybe some could help me get this disk going.


 


Here are the facts:


FAS2040


NetApp Release 8.1.4P8 7-Mode


 


The disk at 0d.01.13 was broken, so I pulled it a few days ago – now that the replacment disk has arrived, I tried to plug it in and this is what happens:


 


Thu Oct 22 14:30:10 CEST [riddler:scsi.cmd.notReadyCondition:notice]: Disk device 0d.01.13: Device returns not yet ready: CDB 0x1b: Sense Data SCSI:not ready - Drive spinning up (0x2 - 0x4 0x1 0x28)(2).


 


The interesting thing, I can see the disk by using `storage show disk`:


 


riddler> storage show disk


DISK                  SHELF BAY SERIAL           VENDOR   MODEL      REV


--------------------- --------- ---------------- -------- ---------- ----


0d.01.13                1   13  J838WSRL         NETAPP   X302_HJUPI NA04


 


But I cannot see it in disk show.


 


riddler> disk show 0d.01.13


  DISK       OWNER                      POOL   SERIAL NUMBER         HOME


------------ -------------              -----  -------------         -------------


 


Sysconfig –v tells me, that there is a newer firmware on the „new“ disk:


 


                01.13: NETAPP   X302_HJUPI01TSSM NA04 847.5GB (1953525168 512B/sect)


                01.14: NETAPP   X302_WVULC01TSSA NA03 847.5GB (1953525168 512B/sect)


 


So I first tried to downgrade firmware using:


 


disk_fw_update –R 0d.01.13


 


but that did not really work out, no error message, but just no change on the firmware of the disk – also nothing in the logs.


So I read somewhere that this might be due to a bad/incompatible label on the disk and found out that the only way to get rid of the label is to boot into maintenance mode (thus having a downtime of parts of the HA-pair :-/) and recreate
the label there.


 


So I booted into maintenance mode and have run the following command:


 


*> label makespare 0d.01.13


*********************************************************


Label editing is highly dangerous and should be attempted


ONLY by NetApp Customer Support Engineers.


Misuse can cause ALL data to be permanently lost.


*********************************************************


'Label save': Error while trying to save Label 1 on disk 0d.03.22: error 15.


Label save: 1 of 35 disks could not save Label 1.


'Label save': Error while trying to save Label 2 on disk 0d.03.22: error 15.


Label save: 1 of 35 disks could not save Label 2.


The current set of labels on the disks have been saved. You can


use the 'label restore' command to revert to these labels, the


'label keep' command to erase these labels, or 'label save' to


preserve the current set of labels.


*********************************************************


        *** Run WAFL recovery tools (wafliron or wafliron_oc) after this. ***


 


Disk 0d.01.13 does not exist.


 


The last line suggests, that the disk still does not get recognized properly – and additionally, it (for whatever reason) claims to not be able to write the label 1 on disk 0d.03.22, although this is there and online:


 


riddler> disk show 0d.03.22


  DISK       OWNER                      POOL   SERIAL NUMBER         HOME


------------ -------------              -----  -------------         -------------


0d.03.22     riddler   (142248200)    Pool0  N002XRML              riddler   (142248200)


 


I’ve stopped my expiriments at this point and am very thankful for fresh ideas
J


 


Best,


 


Alexander Griesser


Head of Systems Operations


 


ANEXIA Internetdienstleistungs GmbH


 


E-Mail:
AGriesser@anexia-it.com



Web:
http://www.anexia-it.com



 


Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt


Geschäftsführer: Alexander Windbichler


Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601


 
AW: Used Replacement disk not recognized [ In reply to ]
Well, in fact, I ordered two replacement drives and both show the exact same symptom…
So I’m either the unluckiest guy on the world, or this is unrelated to the actual problem ☺

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: Tim McCarthy [mailto:tmacmd@gmail.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:38
An: toasters@teaparty.net; Alexander Griesser <AGriesser@anexia-it.com>
Betreff: Re: Used Replacement disk not recognized

One thing to look at...

Do you have the latest disk qualification package?

Doubt that's it though. It looks like the disk is not properly spinning up... You replaced a bad drive with a not so good drive 😢

--tmac
Sent from Mobile Outlook<http://aka.ms/Ox5hz3>



On Thu, Oct 22, 2015 at 7:10 AM -0700, "Alexander Griesser" <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>> wrote:
Hey everybody,

I just had the pleasure to try to replace a faulty disk in a filer which is out-of-service with a „new“ one from after market.
Of course, this didn’t work out as easy as it should, so maybe some could help me get this disk going.

Here are the facts:
FAS2040
NetApp Release 8.1.4P8 7-Mode

The disk at 0d.01.13 was broken, so I pulled it a few days ago – now that the replacment disk has arrived, I tried to plug it in and this is what happens:

Thu Oct 22 14:30:10 CEST [riddler:scsi.cmd.notReadyCondition:notice]: Disk device 0d.01.13: Device returns not yet ready: CDB 0x1b: Sense Data SCSI:not ready - Drive spinning up (0x2 - 0x4 0x1 0x28)(2).

The interesting thing, I can see the disk by using `storage show disk`:

riddler> storage show disk
DISK SHELF BAY SERIAL VENDOR MODEL REV
--------------------- --------- ---------------- -------- ---------- ----
0d.01.13 1 13 J838WSRL NETAPP X302_HJUPI NA04

But I cannot see it in disk show.

riddler> disk show 0d.01.13
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------

Sysconfig –v tells me, that there is a newer firmware on the „new“ disk:

01.13: NETAPP X302_HJUPI01TSSM NA04 847.5GB (1953525168 512B/sect)
01.14: NETAPP X302_WVULC01TSSA NA03 847.5GB (1953525168 512B/sect)

So I first tried to downgrade firmware using:

disk_fw_update –R 0d.01.13

but that did not really work out, no error message, but just no change on the firmware of the disk – also nothing in the logs.
So I read somewhere that this might be due to a bad/incompatible label on the disk and found out that the only way to get rid of the label is to boot into maintenance mode (thus having a downtime of parts of the HA-pair :-/) and recreate the label there.

So I booted into maintenance mode and have run the following command:

*> label makespare 0d.01.13
*********************************************************
Label editing is highly dangerous and should be attempted
ONLY by NetApp Customer Support Engineers.
Misuse can cause ALL data to be permanently lost.
*********************************************************
'Label save': Error while trying to save Label 1 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 1.
'Label save': Error while trying to save Label 2 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 2.
The current set of labels on the disks have been saved. You can
use the 'label restore' command to revert to these labels, the
'label keep' command to erase these labels, or 'label save' to
preserve the current set of labels.
*********************************************************
*** Run WAFL recovery tools (wafliron or wafliron_oc) after this. ***

Disk 0d.01.13 does not exist.

The last line suggests, that the disk still does not get recognized properly – and additionally, it (for whatever reason) claims to not be able to write the label 1 on disk 0d.03.22, although this is there and online:

riddler> disk show 0d.03.22
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------
0d.03.22 riddler (142248200) Pool0 N002XRML riddler (142248200)

I’ve stopped my expiriments at this point and am very thankful for fresh ideas ☺

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
RE: Used Replacement disk not recognized [ In reply to ]
Firmware is not older or newer – those two disks are from different manufacturers so they have different firmware; both are using the latest versions publically available.

KB https://kb.netapp.com/support/index?page=content&id=2010827&locale=en_US&access=s implies that Drive spinning up may be seen with X302 and is otherwise harmless, although there are so many different variants of X302 it is hard to guess whether it applies in your case. I would suggest open the case.

Last command you pasted suggests that everything is OK now; or what is remaining problem?

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Alexander Griesser
Sent: Thursday, October 22, 2015 5:06 PM
To: toasters@teaparty.net
Subject: Used Replacement disk not recognized

Hey everybody,

I just had the pleasure to try to replace a faulty disk in a filer which is out-of-service with a „new“ one from after market.
Of course, this didn’t work out as easy as it should, so maybe some could help me get this disk going.

Here are the facts:
FAS2040
NetApp Release 8.1.4P8 7-Mode

The disk at 0d.01.13 was broken, so I pulled it a few days ago – now that the replacment disk has arrived, I tried to plug it in and this is what happens:

Thu Oct 22 14:30:10 CEST [riddler:scsi.cmd.notReadyCondition:notice]: Disk device 0d.01.13: Device returns not yet ready: CDB 0x1b: Sense Data SCSI:not ready - Drive spinning up (0x2 - 0x4 0x1 0x28)(2).

The interesting thing, I can see the disk by using `storage show disk`:

riddler> storage show disk
DISK SHELF BAY SERIAL VENDOR MODEL REV
--------------------- --------- ---------------- -------- ---------- ----
0d.01.13 1 13 J838WSRL NETAPP X302_HJUPI NA04

But I cannot see it in disk show.

riddler> disk show 0d.01.13
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------

Sysconfig –v tells me, that there is a newer firmware on the „new“ disk:

01.13: NETAPP X302_HJUPI01TSSM NA04 847.5GB (1953525168 512B/sect)
01.14: NETAPP X302_WVULC01TSSA NA03 847.5GB (1953525168 512B/sect)

So I first tried to downgrade firmware using:

disk_fw_update –R 0d.01.13

but that did not really work out, no error message, but just no change on the firmware of the disk – also nothing in the logs.
So I read somewhere that this might be due to a bad/incompatible label on the disk and found out that the only way to get rid of the label is to boot into maintenance mode (thus having a downtime of parts of the HA-pair :-/) and recreate the label there.

So I booted into maintenance mode and have run the following command:

*> label makespare 0d.01.13
*********************************************************
Label editing is highly dangerous and should be attempted
ONLY by NetApp Customer Support Engineers.
Misuse can cause ALL data to be permanently lost.
*********************************************************
'Label save': Error while trying to save Label 1 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 1.
'Label save': Error while trying to save Label 2 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 2.
The current set of labels on the disks have been saved. You can
use the 'label restore' command to revert to these labels, the
'label keep' command to erase these labels, or 'label save' to
preserve the current set of labels.
*********************************************************
*** Run WAFL recovery tools (wafliron or wafliron_oc) after this. ***

Disk 0d.01.13 does not exist.

The last line suggests, that the disk still does not get recognized properly – and additionally, it (for whatever reason) claims to not be able to write the label 1 on disk 0d.03.22, although this is there and online:

riddler> disk show 0d.03.22
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------
0d.03.22 riddler (142248200) Pool0 N002XRML riddler (142248200)

I’ve stopped my expiriments at this point and am very thankful for fresh ideas :)

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
AW: Used Replacement disk not recognized [ In reply to ]
Oh, yes – you’re right.

0d.03.22 is a spare disk – so now the label message makes sense!
Sysconfig –r does not show the disk 0d.01.13 either and aggr status –f is also clean:

riddler> aggr status -f

Broken disks (empty)

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: Brad Thompson [mailto:brad.thompson877@gmail.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:36
An: Alexander Griesser <AGriesser@anexia-it.com>
Cc: toasters@teaparty.net
Betreff: Re: Used Replacement disk not recognized

This is not due to a bad label. You would have seen a message when taking ownership of the disk about a bad label if that were the case. The disk would also show up in the failed disk pool (aggr status -f) with a bad or invalid label tag on it (I can't remember exactly what it says). The bad label only comes in to play if a disk was in a higher version of Data ONTAP, and was not marked as a spare, zeroed, and then ownership removed prior to inserting it in your system. On your version of ONTAP the label version is 11. So it would have had to have been in a system running 8.2 or higher.
Regarding 0d.03.22, is that disk a spare? What does your "sysconfig -r" output look like?

On Thu, Oct 22, 2015 at 10:06 AM, Alexander Griesser <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>> wrote:
Hey everybody,

I just had the pleasure to try to replace a faulty disk in a filer which is out-of-service with a „new“ one from after market.
Of course, this didn’t work out as easy as it should, so maybe some could help me get this disk going.

Here are the facts:
FAS2040
NetApp Release 8.1.4P8 7-Mode

The disk at 0d.01.13 was broken, so I pulled it a few days ago – now that the replacment disk has arrived, I tried to plug it in and this is what happens:

Thu Oct 22 14:30:10 CEST [riddler:scsi.cmd.notReadyCondition:notice]: Disk device 0d.01.13: Device returns not yet ready: CDB 0x1b: Sense Data SCSI:not ready - Drive spinning up (0x2 - 0x4 0x1 0x28)(2).

The interesting thing, I can see the disk by using `storage show disk`:

riddler> storage show disk
DISK SHELF BAY SERIAL VENDOR MODEL REV
--------------------- --------- ---------------- -------- ---------- ----
0d.01.13 1 13 J838WSRL NETAPP X302_HJUPI NA04

But I cannot see it in disk show.

riddler> disk show 0d.01.13
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------

Sysconfig –v tells me, that there is a newer firmware on the „new“ disk:

01.13: NETAPP X302_HJUPI01TSSM NA04 847.5GB (1953525168 512B/sect)
01.14: NETAPP X302_WVULC01TSSA NA03 847.5GB (1953525168 512B/sect)

So I first tried to downgrade firmware using:

disk_fw_update –R 0d.01.13

but that did not really work out, no error message, but just no change on the firmware of the disk – also nothing in the logs.
So I read somewhere that this might be due to a bad/incompatible label on the disk and found out that the only way to get rid of the label is to boot into maintenance mode (thus having a downtime of parts of the HA-pair :-/) and recreate the label there.

So I booted into maintenance mode and have run the following command:

*> label makespare 0d.01.13
*********************************************************
Label editing is highly dangerous and should be attempted
ONLY by NetApp Customer Support Engineers.
Misuse can cause ALL data to be permanently lost.
*********************************************************
'Label save': Error while trying to save Label 1 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 1.
'Label save': Error while trying to save Label 2 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 2.
The current set of labels on the disks have been saved. You can
use the 'label restore' command to revert to these labels, the
'label keep' command to erase these labels, or 'label save' to
preserve the current set of labels.
*********************************************************
*** Run WAFL recovery tools (wafliron or wafliron_oc) after this. ***

Disk 0d.01.13 does not exist.

The last line suggests, that the disk still does not get recognized properly – and additionally, it (for whatever reason) claims to not be able to write the label 1 on disk 0d.03.22, although this is there and online:

riddler> disk show 0d.03.22
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------
0d.03.22 riddler (142248200) Pool0 N002XRML riddler (142248200)

I’ve stopped my expiriments at this point and am very thankful for fresh ideas ☺

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601


_______________________________________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
http://www.teaparty.net/mailman/listinfo/toasters
AW: Used Replacement disk not recognized [ In reply to ]
Hi Andrei,

the last command states "Disk 0d.01.13 does not exist", I don't think that this is cool...
The problem is that even after the reboot, the disk does not show up anywhere except for sysconfig or storage show disk.

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:41
An: Alexander Griesser <AGriesser@anexia-it.com>; toasters@teaparty.net
Betreff: RE: Used Replacement disk not recognized

Firmware is not older or newer - those two disks are from different manufacturers so they have different firmware; both are using the latest versions publically available.

KB https://kb.netapp.com/support/index?page=content&id=2010827&locale=en_US&access=s implies that Drive spinning up may be seen with X302 and is otherwise harmless, although there are so many different variants of X302 it is hard to guess whether it applies in your case. I would suggest open the case.

Last command you pasted suggests that everything is OK now; or what is remaining problem?

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: toasters-bounces@teaparty.net<mailto:toasters-bounces@teaparty.net> [mailto:toasters-bounces@teaparty.net] On Behalf Of Alexander Griesser
Sent: Thursday, October 22, 2015 5:06 PM
To: toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: Used Replacement disk not recognized

Hey everybody,

I just had the pleasure to try to replace a faulty disk in a filer which is out-of-service with a "new" one from after market.
Of course, this didn't work out as easy as it should, so maybe some could help me get this disk going.

Here are the facts:
FAS2040
NetApp Release 8.1.4P8 7-Mode

The disk at 0d.01.13 was broken, so I pulled it a few days ago - now that the replacment disk has arrived, I tried to plug it in and this is what happens:

Thu Oct 22 14:30:10 CEST [riddler:scsi.cmd.notReadyCondition:notice]: Disk device 0d.01.13: Device returns not yet ready: CDB 0x1b: Sense Data SCSI:not ready - Drive spinning up (0x2 - 0x4 0x1 0x28)(2).

The interesting thing, I can see the disk by using `storage show disk`:

riddler> storage show disk
DISK SHELF BAY SERIAL VENDOR MODEL REV
--------------------- --------- ---------------- -------- ---------- ----
0d.01.13 1 13 J838WSRL NETAPP X302_HJUPI NA04

But I cannot see it in disk show.

riddler> disk show 0d.01.13
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------

Sysconfig -v tells me, that there is a newer firmware on the "new" disk:

01.13: NETAPP X302_HJUPI01TSSM NA04 847.5GB (1953525168 512B/sect)
01.14: NETAPP X302_WVULC01TSSA NA03 847.5GB (1953525168 512B/sect)

So I first tried to downgrade firmware using:

disk_fw_update -R 0d.01.13

but that did not really work out, no error message, but just no change on the firmware of the disk - also nothing in the logs.
So I read somewhere that this might be due to a bad/incompatible label on the disk and found out that the only way to get rid of the label is to boot into maintenance mode (thus having a downtime of parts of the HA-pair :-/) and recreate the label there.

So I booted into maintenance mode and have run the following command:

*> label makespare 0d.01.13
*********************************************************
Label editing is highly dangerous and should be attempted
ONLY by NetApp Customer Support Engineers.
Misuse can cause ALL data to be permanently lost.
*********************************************************
'Label save': Error while trying to save Label 1 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 1.
'Label save': Error while trying to save Label 2 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 2.
The current set of labels on the disks have been saved. You can
use the 'label restore' command to revert to these labels, the
'label keep' command to erase these labels, or 'label save' to
preserve the current set of labels.
*********************************************************
*** Run WAFL recovery tools (wafliron or wafliron_oc) after this. ***

Disk 0d.01.13 does not exist.

The last line suggests, that the disk still does not get recognized properly - and additionally, it (for whatever reason) claims to not be able to write the label 1 on disk 0d.03.22, although this is there and online:

riddler> disk show 0d.03.22
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------
0d.03.22 riddler (142248200) Pool0 N002XRML riddler (142248200)

I've stopped my expiriments at this point and am very thankful for fresh ideas :)

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Re: AW: Used Replacement disk not recognized [ In reply to ]
Uh oh.
That *could* be the indicator of a bad slot 
If you have them....
Issue the command to remove a working spare 
Insert one of your replacement disks into that slot and put the spare you removed into the suspect slot 
If you see the same problems on the same slot, you likely have a bad slot on the shelf
Occasionally, this could be caused by a disk error in an adjacent slot... Although I have not seen that since the DS14 shelves.
Worth a look.
There are commands to query the disk statistics ( sas or fc counters)
Do not have them handy. Maybe someone else can advise there.
Sent from Mobile Outlook

_____________________________
From: Alexander Griesser <agriesser@anexia-it.com>
Sent: Thursday, October 22, 2015 10:40 AM
Subject: AW: Used Replacement disk not recognized
To: Tim McCarthy <tmacmd@gmail.com>, <toasters@teaparty.net>




Well, in fact, I ordered two replacement drives and both show the exact same symptom…

So I’m either the unluckiest guy on the world, or this is unrelated to the actual problem J

 

Best,

 

Alexander Griesser

Head of Systems Operations

 

ANEXIA Internetdienstleistungs GmbH

 

E-Mail: AGriesser@anexia-it.com

Web: http://www.anexia-it.com

 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt

Geschäftsführer: Alexander Windbichler

Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

 

Von: Tim McCarthy [mailto:tmacmd@gmail.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:38
An: toasters@teaparty.net; Alexander Griesser <AGriesser@anexia-it.com>
Betreff: Re: Used Replacement disk not recognized

 

One thing to look at...

 

Do you have the latest disk qualification package?

 

Doubt that's it though. It looks like the disk is not properly spinning up... You replaced a bad drive with a not so good drive 😢

 

--tmac

Sent from Mobile Outlook

 





On Thu, Oct 22, 2015 at 7:10 AM -0700, "Alexander Griesser" <AGriesser@anexia-it.com> wrote:

Hey everybody,

 

I just had the pleasure to try to replace a faulty disk in a filer which is out-of-service with a „new“ one from after market.

Of course, this didn’t work out as easy as it should, so maybe some could help me get this disk going.

 

Here are the facts:

FAS2040

NetApp Release 8.1.4P8 7-Mode

 

The disk at 0d.01.13 was broken, so I pulled it a few days ago – now that the replacment disk has arrived, I tried to plug it in and this is what happens:

 

Thu Oct 22 14:30:10 CEST [riddler:scsi.cmd.notReadyCondition:notice]: Disk device 0d.01.13: Device returns not yet ready: CDB 0x1b: Sense Data SCSI:not ready - Drive spinning up (0x2 - 0x4 0x1 0x28)(2).

 

The interesting thing, I can see the disk by using `storage show disk`:

 

riddler> storage show disk

DISK                  SHELF BAY SERIAL           VENDOR   MODEL      REV

--------------------- --------- ---------------- -------- ---------- ----

0d.01.13                1   13  J838WSRL         NETAPP   X302_HJUPI NA04

 

But I cannot see it in disk show.

 

riddler> disk show 0d.01.13

  DISK       OWNER                      POOL   SERIAL NUMBER         HOME

------------ -------------              -----  -------------         -------------

 

Sysconfig –v tells me, that there is a newer firmware on the „new“ disk:

 

                01.13: NETAPP   X302_HJUPI01TSSM NA04 847.5GB (1953525168 512B/sect)

                01.14: NETAPP   X302_WVULC01TSSA NA03 847.5GB (1953525168 512B/sect)

 

So I first tried to downgrade firmware using:

 

disk_fw_update –R 0d.01.13

 

but that did not really work out, no error message, but just no change on the firmware of the disk – also nothing in the logs.

So I read somewhere that this might be due to a bad/incompatible label on the disk and found out that the only way to get rid of the label is to boot into maintenance mode (thus having a downtime of parts of the HA-pair :-/) and recreate the label there.

 

So I booted into maintenance mode and have run the following command:

 

*> label makespare 0d.01.13

*********************************************************

Label editing is highly dangerous and should be attempted

ONLY by NetApp Customer Support Engineers.

Misuse can cause ALL data to be permanently lost.

*********************************************************

'Label save': Error while trying to save Label 1 on disk 0d.03.22: error 15.

Label save: 1 of 35 disks could not save Label 1.

'Label save': Error while trying to save Label 2 on disk 0d.03.22: error 15.

Label save: 1 of 35 disks could not save Label 2.

The current set of labels on the disks have been saved. You can

use the 'label restore' command to revert to these labels, the

'label keep' command to erase these labels, or 'label save' to

preserve the current set of labels.

*********************************************************

        *** Run WAFL recovery tools (wafliron or wafliron_oc) after this. ***

 

Disk 0d.01.13 does not exist.

 

The last line suggests, that the disk still does not get recognized properly – and additionally, it (for whatever reason) claims to not be able to write the label 1 on disk 0d.03.22, although this is there and online:

 

riddler> disk show 0d.03.22

  DISK       OWNER                      POOL   SERIAL NUMBER         HOME

------------ -------------              -----  -------------         -------------

0d.03.22     riddler   (142248200)    Pool0  N002XRML              riddler   (142248200)

 

I’ve stopped my expiriments at this point and am very thankful for fresh ideas J

 

Best,

 

Alexander Griesser

Head of Systems Operations

 

ANEXIA Internetdienstleistungs GmbH

 

E-Mail: AGriesser@anexia-it.com

Web: http://www.anexia-it.com

 

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt

Geschäftsführer: Alexander Windbichler

Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

 
RE: Used Replacement disk not recognized [ In reply to ]
Does it appear in “disk show -v”?

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: Alexander Griesser [mailto:AGriesser@anexia-it.com]
Sent: Thursday, October 22, 2015 5:44 PM
To: Borzenkov, Andrei; toasters@teaparty.net
Subject: AW: Used Replacement disk not recognized

Hi Andrei,

the last command states „Disk 0d.01.13 does not exist“, I don’t think that this is cool…
The problem is that even after the reboot, the disk does not show up anywhere except for sysconfig or storage show disk.

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com> [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:41
An: Alexander Griesser <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: RE: Used Replacement disk not recognized

Firmware is not older or newer – those two disks are from different manufacturers so they have different firmware; both are using the latest versions publically available.

KB https://kb.netapp.com/support/index?page=content&id=2010827&locale=en_US&access=s implies that Drive spinning up may be seen with X302 and is otherwise harmless, although there are so many different variants of X302 it is hard to guess whether it applies in your case. I would suggest open the case.

Last command you pasted suggests that everything is OK now; or what is remaining problem?

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: toasters-bounces@teaparty.net<mailto:toasters-bounces@teaparty.net> [mailto:toasters-bounces@teaparty.net] On Behalf Of Alexander Griesser
Sent: Thursday, October 22, 2015 5:06 PM
To: toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: Used Replacement disk not recognized

Hey everybody,

I just had the pleasure to try to replace a faulty disk in a filer which is out-of-service with a „new“ one from after market.
Of course, this didn’t work out as easy as it should, so maybe some could help me get this disk going.

Here are the facts:
FAS2040
NetApp Release 8.1.4P8 7-Mode

The disk at 0d.01.13 was broken, so I pulled it a few days ago – now that the replacment disk has arrived, I tried to plug it in and this is what happens:

Thu Oct 22 14:30:10 CEST [riddler:scsi.cmd.notReadyCondition:notice]: Disk device 0d.01.13: Device returns not yet ready: CDB 0x1b: Sense Data SCSI:not ready - Drive spinning up (0x2 - 0x4 0x1 0x28)(2).

The interesting thing, I can see the disk by using `storage show disk`:

riddler> storage show disk
DISK SHELF BAY SERIAL VENDOR MODEL REV
--------------------- --------- ---------------- -------- ---------- ----
0d.01.13 1 13 J838WSRL NETAPP X302_HJUPI NA04

But I cannot see it in disk show.

riddler> disk show 0d.01.13
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------

Sysconfig –v tells me, that there is a newer firmware on the „new“ disk:

01.13: NETAPP X302_HJUPI01TSSM NA04 847.5GB (1953525168 512B/sect)
01.14: NETAPP X302_WVULC01TSSA NA03 847.5GB (1953525168 512B/sect)

So I first tried to downgrade firmware using:

disk_fw_update –R 0d.01.13

but that did not really work out, no error message, but just no change on the firmware of the disk – also nothing in the logs.
So I read somewhere that this might be due to a bad/incompatible label on the disk and found out that the only way to get rid of the label is to boot into maintenance mode (thus having a downtime of parts of the HA-pair :-/) and recreate the label there.

So I booted into maintenance mode and have run the following command:

*> label makespare 0d.01.13
*********************************************************
Label editing is highly dangerous and should be attempted
ONLY by NetApp Customer Support Engineers.
Misuse can cause ALL data to be permanently lost.
*********************************************************
'Label save': Error while trying to save Label 1 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 1.
'Label save': Error while trying to save Label 2 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 2.
The current set of labels on the disks have been saved. You can
use the 'label restore' command to revert to these labels, the
'label keep' command to erase these labels, or 'label save' to
preserve the current set of labels.
*********************************************************
*** Run WAFL recovery tools (wafliron or wafliron_oc) after this. ***

Disk 0d.01.13 does not exist.

The last line suggests, that the disk still does not get recognized properly – and additionally, it (for whatever reason) claims to not be able to write the label 1 on disk 0d.03.22, although this is there and online:

riddler> disk show 0d.03.22
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------
0d.03.22 riddler (142248200) Pool0 N002XRML riddler (142248200)

I’ve stopped my expiriments at this point and am very thankful for fresh ideas :)

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
AW: Used Replacement disk not recognized [ In reply to ]
Bingo:

riddler> disk show -v
DISK OWNER POOL SERIAL NUMBER HOME CHKSUM
------------ ------------- ----- ------------- ------------- ------
0d.01.13 w-nas-04 (142255463) Pool0 J838WSRL w-nas-04 (142255463) Block

"w-nas-04" is none of my filers - so I need to remove the ownership here now, I guess.

riddler*> disk remove_ownership 0d.01.13 -f
disk remove_ownership: Disk 0d.01.13 is not owned by this node.
riddler*> disk reassign -o w-nas-04 -n riddler
disk reassign: Can only be run in maintenance mode or during takeover in advanced mode
riddler*>

So, I guess I have to - again - boot into maintenance mode or is there a trick which would allow me to remove the ownership without having to shut everything down again?

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:47
An: Alexander Griesser <AGriesser@anexia-it.com>; toasters@teaparty.net
Betreff: RE: Used Replacement disk not recognized

Does it appear in "disk show -v"?

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: Alexander Griesser [mailto:AGriesser@anexia-it.com]
Sent: Thursday, October 22, 2015 5:44 PM
To: Borzenkov, Andrei; toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: AW: Used Replacement disk not recognized

Hi Andrei,

the last command states "Disk 0d.01.13 does not exist", I don't think that this is cool...
The problem is that even after the reboot, the disk does not show up anywhere except for sysconfig or storage show disk.

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com> [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:41
An: Alexander Griesser <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: RE: Used Replacement disk not recognized

Firmware is not older or newer - those two disks are from different manufacturers so they have different firmware; both are using the latest versions publically available.

KB https://kb.netapp.com/support/index?page=content&id=2010827&locale=en_US&access=s implies that Drive spinning up may be seen with X302 and is otherwise harmless, although there are so many different variants of X302 it is hard to guess whether it applies in your case. I would suggest open the case.

Last command you pasted suggests that everything is OK now; or what is remaining problem?

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: toasters-bounces@teaparty.net<mailto:toasters-bounces@teaparty.net> [mailto:toasters-bounces@teaparty.net] On Behalf Of Alexander Griesser
Sent: Thursday, October 22, 2015 5:06 PM
To: toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: Used Replacement disk not recognized

Hey everybody,

I just had the pleasure to try to replace a faulty disk in a filer which is out-of-service with a "new" one from after market.
Of course, this didn't work out as easy as it should, so maybe some could help me get this disk going.

Here are the facts:
FAS2040
NetApp Release 8.1.4P8 7-Mode

The disk at 0d.01.13 was broken, so I pulled it a few days ago - now that the replacment disk has arrived, I tried to plug it in and this is what happens:

Thu Oct 22 14:30:10 CEST [riddler:scsi.cmd.notReadyCondition:notice]: Disk device 0d.01.13: Device returns not yet ready: CDB 0x1b: Sense Data SCSI:not ready - Drive spinning up (0x2 - 0x4 0x1 0x28)(2).

The interesting thing, I can see the disk by using `storage show disk`:

riddler> storage show disk
DISK SHELF BAY SERIAL VENDOR MODEL REV
--------------------- --------- ---------------- -------- ---------- ----
0d.01.13 1 13 J838WSRL NETAPP X302_HJUPI NA04

But I cannot see it in disk show.

riddler> disk show 0d.01.13
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------

Sysconfig -v tells me, that there is a newer firmware on the "new" disk:

01.13: NETAPP X302_HJUPI01TSSM NA04 847.5GB (1953525168 512B/sect)
01.14: NETAPP X302_WVULC01TSSA NA03 847.5GB (1953525168 512B/sect)

So I first tried to downgrade firmware using:

disk_fw_update -R 0d.01.13

but that did not really work out, no error message, but just no change on the firmware of the disk - also nothing in the logs.
So I read somewhere that this might be due to a bad/incompatible label on the disk and found out that the only way to get rid of the label is to boot into maintenance mode (thus having a downtime of parts of the HA-pair :-/) and recreate the label there.

So I booted into maintenance mode and have run the following command:

*> label makespare 0d.01.13
*********************************************************
Label editing is highly dangerous and should be attempted
ONLY by NetApp Customer Support Engineers.
Misuse can cause ALL data to be permanently lost.
*********************************************************
'Label save': Error while trying to save Label 1 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 1.
'Label save': Error while trying to save Label 2 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 2.
The current set of labels on the disks have been saved. You can
use the 'label restore' command to revert to these labels, the
'label keep' command to erase these labels, or 'label save' to
preserve the current set of labels.
*********************************************************
*** Run WAFL recovery tools (wafliron or wafliron_oc) after this. ***

Disk 0d.01.13 does not exist.

The last line suggests, that the disk still does not get recognized properly - and additionally, it (for whatever reason) claims to not be able to write the label 1 on disk 0d.03.22, although this is there and online:

riddler> disk show 0d.03.22
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------
0d.03.22 riddler (142248200) Pool0 N002XRML riddler (142248200)

I've stopped my expiriments at this point and am very thankful for fresh ideas :)

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
RE: Used Replacement disk not recognized [ In reply to ]
disk assign 0d.01.13 -f

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: Alexander Griesser [mailto:AGriesser@anexia-it.com]
Sent: Thursday, October 22, 2015 5:50 PM
To: Borzenkov, Andrei; toasters@teaparty.net
Subject: AW: Used Replacement disk not recognized

Bingo:

riddler> disk show -v
DISK OWNER POOL SERIAL NUMBER HOME CHKSUM
------------ ------------- ----- ------------- ------------- ------
0d.01.13 w-nas-04 (142255463) Pool0 J838WSRL w-nas-04 (142255463) Block

„w-nas-04“ is none of my filers – so I need to remove the ownership here now, I guess.

riddler*> disk remove_ownership 0d.01.13 -f
disk remove_ownership: Disk 0d.01.13 is not owned by this node.
riddler*> disk reassign -o w-nas-04 -n riddler
disk reassign: Can only be run in maintenance mode or during takeover in advanced mode
riddler*>

So, I guess I have to – again – boot into maintenance mode or is there a trick which would allow me to remove the ownership without having to shut everything down again?

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com> [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:47
An: Alexander Griesser <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: RE: Used Replacement disk not recognized

Does it appear in “disk show -v”?

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: Alexander Griesser [mailto:AGriesser@anexia-it.com]
Sent: Thursday, October 22, 2015 5:44 PM
To: Borzenkov, Andrei; toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: AW: Used Replacement disk not recognized

Hi Andrei,

the last command states „Disk 0d.01.13 does not exist“, I don’t think that this is cool…
The problem is that even after the reboot, the disk does not show up anywhere except for sysconfig or storage show disk.

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com> [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:41
An: Alexander Griesser <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: RE: Used Replacement disk not recognized

Firmware is not older or newer – those two disks are from different manufacturers so they have different firmware; both are using the latest versions publically available.

KB https://kb.netapp.com/support/index?page=content&id=2010827&locale=en_US&access=s implies that Drive spinning up may be seen with X302 and is otherwise harmless, although there are so many different variants of X302 it is hard to guess whether it applies in your case. I would suggest open the case.

Last command you pasted suggests that everything is OK now; or what is remaining problem?

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: toasters-bounces@teaparty.net<mailto:toasters-bounces@teaparty.net> [mailto:toasters-bounces@teaparty.net] On Behalf Of Alexander Griesser
Sent: Thursday, October 22, 2015 5:06 PM
To: toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: Used Replacement disk not recognized

Hey everybody,

I just had the pleasure to try to replace a faulty disk in a filer which is out-of-service with a „new“ one from after market.
Of course, this didn’t work out as easy as it should, so maybe some could help me get this disk going.

Here are the facts:
FAS2040
NetApp Release 8.1.4P8 7-Mode

The disk at 0d.01.13 was broken, so I pulled it a few days ago – now that the replacment disk has arrived, I tried to plug it in and this is what happens:

Thu Oct 22 14:30:10 CEST [riddler:scsi.cmd.notReadyCondition:notice]: Disk device 0d.01.13: Device returns not yet ready: CDB 0x1b: Sense Data SCSI:not ready - Drive spinning up (0x2 - 0x4 0x1 0x28)(2).

The interesting thing, I can see the disk by using `storage show disk`:

riddler> storage show disk
DISK SHELF BAY SERIAL VENDOR MODEL REV
--------------------- --------- ---------------- -------- ---------- ----
0d.01.13 1 13 J838WSRL NETAPP X302_HJUPI NA04

But I cannot see it in disk show.

riddler> disk show 0d.01.13
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------

Sysconfig –v tells me, that there is a newer firmware on the „new“ disk:

01.13: NETAPP X302_HJUPI01TSSM NA04 847.5GB (1953525168 512B/sect)
01.14: NETAPP X302_WVULC01TSSA NA03 847.5GB (1953525168 512B/sect)

So I first tried to downgrade firmware using:

disk_fw_update –R 0d.01.13

but that did not really work out, no error message, but just no change on the firmware of the disk – also nothing in the logs.
So I read somewhere that this might be due to a bad/incompatible label on the disk and found out that the only way to get rid of the label is to boot into maintenance mode (thus having a downtime of parts of the HA-pair :-/) and recreate the label there.

So I booted into maintenance mode and have run the following command:

*> label makespare 0d.01.13
*********************************************************
Label editing is highly dangerous and should be attempted
ONLY by NetApp Customer Support Engineers.
Misuse can cause ALL data to be permanently lost.
*********************************************************
'Label save': Error while trying to save Label 1 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 1.
'Label save': Error while trying to save Label 2 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 2.
The current set of labels on the disks have been saved. You can
use the 'label restore' command to revert to these labels, the
'label keep' command to erase these labels, or 'label save' to
preserve the current set of labels.
*********************************************************
*** Run WAFL recovery tools (wafliron or wafliron_oc) after this. ***

Disk 0d.01.13 does not exist.

The last line suggests, that the disk still does not get recognized properly – and additionally, it (for whatever reason) claims to not be able to write the label 1 on disk 0d.03.22, although this is there and online:

riddler> disk show 0d.03.22
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------
0d.03.22 riddler (142248200) Pool0 N002XRML riddler (142248200)

I’ve stopped my expiriments at this point and am very thankful for fresh ideas :)

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
AW: Used Replacement disk not recognized [ In reply to ]
Many thanks!

riddler> vol status -s

Pool1 spare disks (empty)

Pool0 spare disks

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
Spare disks for block checksum
spare 0d.01.13 0d 1 13 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816
spare 0d.03.22 0d 3 22 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816

I'm up and running :)

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:52
An: Alexander Griesser <AGriesser@anexia-it.com>; toasters@teaparty.net
Betreff: RE: Used Replacement disk not recognized

disk assign 0d.01.13 -f

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: Alexander Griesser [mailto:AGriesser@anexia-it.com]
Sent: Thursday, October 22, 2015 5:50 PM
To: Borzenkov, Andrei; toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: AW: Used Replacement disk not recognized

Bingo:

riddler> disk show -v
DISK OWNER POOL SERIAL NUMBER HOME CHKSUM
------------ ------------- ----- ------------- ------------- ------
0d.01.13 w-nas-04 (142255463) Pool0 J838WSRL w-nas-04 (142255463) Block

"w-nas-04" is none of my filers - so I need to remove the ownership here now, I guess.

riddler*> disk remove_ownership 0d.01.13 -f
disk remove_ownership: Disk 0d.01.13 is not owned by this node.
riddler*> disk reassign -o w-nas-04 -n riddler
disk reassign: Can only be run in maintenance mode or during takeover in advanced mode
riddler*>

So, I guess I have to - again - boot into maintenance mode or is there a trick which would allow me to remove the ownership without having to shut everything down again?

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com> [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:47
An: Alexander Griesser <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: RE: Used Replacement disk not recognized

Does it appear in "disk show -v"?

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: Alexander Griesser [mailto:AGriesser@anexia-it.com]
Sent: Thursday, October 22, 2015 5:44 PM
To: Borzenkov, Andrei; toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: AW: Used Replacement disk not recognized

Hi Andrei,

the last command states "Disk 0d.01.13 does not exist", I don't think that this is cool...
The problem is that even after the reboot, the disk does not show up anywhere except for sysconfig or storage show disk.

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com> [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:41
An: Alexander Griesser <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: RE: Used Replacement disk not recognized

Firmware is not older or newer - those two disks are from different manufacturers so they have different firmware; both are using the latest versions publically available.

KB https://kb.netapp.com/support/index?page=content&id=2010827&locale=en_US&access=s implies that Drive spinning up may be seen with X302 and is otherwise harmless, although there are so many different variants of X302 it is hard to guess whether it applies in your case. I would suggest open the case.

Last command you pasted suggests that everything is OK now; or what is remaining problem?

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: toasters-bounces@teaparty.net<mailto:toasters-bounces@teaparty.net> [mailto:toasters-bounces@teaparty.net] On Behalf Of Alexander Griesser
Sent: Thursday, October 22, 2015 5:06 PM
To: toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: Used Replacement disk not recognized

Hey everybody,

I just had the pleasure to try to replace a faulty disk in a filer which is out-of-service with a "new" one from after market.
Of course, this didn't work out as easy as it should, so maybe some could help me get this disk going.

Here are the facts:
FAS2040
NetApp Release 8.1.4P8 7-Mode

The disk at 0d.01.13 was broken, so I pulled it a few days ago - now that the replacment disk has arrived, I tried to plug it in and this is what happens:

Thu Oct 22 14:30:10 CEST [riddler:scsi.cmd.notReadyCondition:notice]: Disk device 0d.01.13: Device returns not yet ready: CDB 0x1b: Sense Data SCSI:not ready - Drive spinning up (0x2 - 0x4 0x1 0x28)(2).

The interesting thing, I can see the disk by using `storage show disk`:

riddler> storage show disk
DISK SHELF BAY SERIAL VENDOR MODEL REV
--------------------- --------- ---------------- -------- ---------- ----
0d.01.13 1 13 J838WSRL NETAPP X302_HJUPI NA04

But I cannot see it in disk show.

riddler> disk show 0d.01.13
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------

Sysconfig -v tells me, that there is a newer firmware on the "new" disk:

01.13: NETAPP X302_HJUPI01TSSM NA04 847.5GB (1953525168 512B/sect)
01.14: NETAPP X302_WVULC01TSSA NA03 847.5GB (1953525168 512B/sect)

So I first tried to downgrade firmware using:

disk_fw_update -R 0d.01.13

but that did not really work out, no error message, but just no change on the firmware of the disk - also nothing in the logs.
So I read somewhere that this might be due to a bad/incompatible label on the disk and found out that the only way to get rid of the label is to boot into maintenance mode (thus having a downtime of parts of the HA-pair :-/) and recreate the label there.

So I booted into maintenance mode and have run the following command:

*> label makespare 0d.01.13
*********************************************************
Label editing is highly dangerous and should be attempted
ONLY by NetApp Customer Support Engineers.
Misuse can cause ALL data to be permanently lost.
*********************************************************
'Label save': Error while trying to save Label 1 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 1.
'Label save': Error while trying to save Label 2 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 2.
The current set of labels on the disks have been saved. You can
use the 'label restore' command to revert to these labels, the
'label keep' command to erase these labels, or 'label save' to
preserve the current set of labels.
*********************************************************
*** Run WAFL recovery tools (wafliron or wafliron_oc) after this. ***

Disk 0d.01.13 does not exist.

The last line suggests, that the disk still does not get recognized properly - and additionally, it (for whatever reason) claims to not be able to write the label 1 on disk 0d.03.22, although this is there and online:

riddler> disk show 0d.03.22
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------
0d.03.22 riddler (142248200) Pool0 N002XRML riddler (142248200)

I've stopped my expiriments at this point and am very thankful for fresh ideas :)

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
Re: Used Replacement disk not recognized [ In reply to ]
This command will fix you without having to take the system down.

disk assign 0d.01.13 -s unowned -f

Then, assign the disk.

On Thu, Oct 22, 2015 at 10:49 AM, Alexander Griesser <
AGriesser@anexia-it.com> wrote:

> Bingo:
>
>
>
> riddler> disk show -v
>
> DISK OWNER POOL SERIAL NUMBER
> HOME CHKSUM
>
> ------------ ------------- ----- -------------
> ------------- ------
>
> 0d.01.13 w-nas-04 (142255463) Pool0 J838WSRL
> w-nas-04 (142255463) Block
>
>
>
> „w-nas-04“ is none of my filers – so I need to remove the ownership here
> now, I guess.
>
>
>
> riddler*> disk remove_ownership 0d.01.13 -f
>
> disk remove_ownership: Disk 0d.01.13 is not owned by this node.
>
> riddler*> disk reassign -o w-nas-04 -n riddler
>
> disk reassign: Can only be run in maintenance mode or during takeover in
> advanced mode
>
> riddler*>
>
>
>
> So, I guess I have to – again – boot into maintenance mode or is there a
> trick which would allow me to remove the ownership without having to shut
> everything down again?
>
>
>
> Best,
>
>
>
> *Alexander Griesser*
>
> Head of Systems Operations
>
>
>
> ANEXIA Internetdienstleistungs GmbH
>
>
>
> E-Mail: AGriesser@anexia-it.com
>
> Web: http://www.anexia-it.com
>
>
>
> Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
>
> Geschäftsführer: Alexander Windbichler
>
> Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT
> U63216601
>
>
>
> *Von:* andrei.borzenkov@ts.fujitsu.com [mailto:
> andrei.borzenkov@ts.fujitsu.com]
> *Gesendet:* Donnerstag, 22. Oktober 2015 16:47
> *An:* Alexander Griesser <AGriesser@anexia-it.com>; toasters@teaparty.net
> *Betreff:* RE: Used Replacement disk not recognized
>
>
>
> Does it appear in “disk show -v”?
>
>
>
> ---
>
> With best regards
>
>
>
> *Andre**i** Borzenkov*
>
> Senior system engineer
>
> FTS WEMEAI RUC RU SC TMS FOS
>
> [image: cid:image001.gif@01CBF835.B3FEDA90]
>
> *FUJITSU*
>
> Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
>
> Tel.: +7 495 730 62 20 ( reception)
>
> Mob.: +7 916 678 7208
>
> Fax: +7 495 730 62 14
>
> E-mail: Andrei.Borzenkov@ts.fujitsu.com
>
> Web: ru.fujitsu.com <http://ts.fujitsu.com/>
>
> Company details: ts.fujitsu.com/imprint
> <http://ts.fujitsu.com/imprint.html>
>
> This communication contains information that is confidential, proprietary
> in nature and/or privileged. It is for the exclusive use of the intended
> recipient(s). If you are not the intended recipient(s) or the person
> responsible for delivering it to the intended recipient(s), please note
> that any form of dissemination, distribution or copying of this
> communication is strictly prohibited and may be unlawful. If you have
> received this communication in error, please immediately notify the sender
> and delete the original communication. Thank you for your cooperation.
>
> Please be advised that neither Fujitsu, its affiliates, its employees or
> agents accept liability for any errors, omissions or damages caused by
> delays of receipt or by any virus infection in this message or its
> attachments, or which may otherwise arise as a result of this e-mail
> transmission.
>
>
>
> *From:* Alexander Griesser [mailto:AGriesser@anexia-it.com
> <AGriesser@anexia-it.com>]
> *Sent:* Thursday, October 22, 2015 5:44 PM
> *To:* Borzenkov, Andrei; toasters@teaparty.net
> *Subject:* AW: Used Replacement disk not recognized
>
>
>
> Hi Andrei,
>
>
>
> the last command states „Disk 0d.01.13 does not exist“, I don’t think that
> this is cool…
>
> The problem is that even after the reboot, the disk does not show up
> anywhere except for sysconfig or storage show disk.
>
>
>
> Best,
>
>
>
> *Alexander Griesser*
>
> Head of Systems Operations
>
>
>
> ANEXIA Internetdienstleistungs GmbH
>
>
>
> E-Mail: AGriesser@anexia-it.com
>
> Web: http://www.anexia-it.com
>
>
>
> Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
>
> Geschäftsführer: Alexander Windbichler
>
> Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT
> U63216601
>
>
>
> *Von:* andrei.borzenkov@ts.fujitsu.com [
> mailto:andrei.borzenkov@ts.fujitsu.com <andrei.borzenkov@ts.fujitsu.com>]
> *Gesendet:* Donnerstag, 22. Oktober 2015 16:41
> *An:* Alexander Griesser <AGriesser@anexia-it.com>; toasters@teaparty.net
> *Betreff:* RE: Used Replacement disk not recognized
>
>
>
> Firmware is not older or newer – those two disks are from different
> manufacturers so they have different firmware; both are using the latest
> versions publically available.
>
>
>
> KB
> https://kb.netapp.com/support/index?page=content&id=2010827&locale=en_US&access=s
> implies that Drive spinning up may be seen with X302 and is otherwise
> harmless, although there are so many different variants of X302 it is hard
> to guess whether it applies in your case. I would suggest open the case.
>
>
>
> Last command you pasted suggests that everything is OK now; or what is
> remaining problem?
>
>
>
> ---
>
> With best regards
>
>
>
> *Andre**i** Borzenkov*
>
> Senior system engineer
>
> FTS WEMEAI RUC RU SC TMS FOS
>
> [image: cid:image001.gif@01CBF835.B3FEDA90]
>
> *FUJITSU*
>
> Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
>
> Tel.: +7 495 730 62 20 ( reception)
>
> Mob.: +7 916 678 7208
>
> Fax: +7 495 730 62 14
>
> E-mail: Andrei.Borzenkov@ts.fujitsu.com
>
> Web: ru.fujitsu.com <http://ts.fujitsu.com/>
>
> Company details: ts.fujitsu.com/imprint
> <http://ts.fujitsu.com/imprint.html>
>
> This communication contains information that is confidential, proprietary
> in nature and/or privileged. It is for the exclusive use of the intended
> recipient(s). If you are not the intended recipient(s) or the person
> responsible for delivering it to the intended recipient(s), please note
> that any form of dissemination, distribution or copying of this
> communication is strictly prohibited and may be unlawful. If you have
> received this communication in error, please immediately notify the sender
> and delete the original communication. Thank you for your cooperation.
>
> Please be advised that neither Fujitsu, its affiliates, its employees or
> agents accept liability for any errors, omissions or damages caused by
> delays of receipt or by any virus infection in this message or its
> attachments, or which may otherwise arise as a result of this e-mail
> transmission.
>
>
>
> *From:* toasters-bounces@teaparty.net [
> mailto:toasters-bounces@teaparty.net <toasters-bounces@teaparty.net>] *On
> Behalf Of *Alexander Griesser
> *Sent:* Thursday, October 22, 2015 5:06 PM
> *To:* toasters@teaparty.net
> *Subject:* Used Replacement disk not recognized
>
>
>
> Hey everybody,
>
>
>
> I just had the pleasure to try to replace a faulty disk in a filer which
> is out-of-service with a „new“ one from after market.
>
> Of course, this didn’t work out as easy as it should, so maybe some could
> help me get this disk going.
>
>
>
> Here are the facts:
>
> FAS2040
>
> NetApp Release 8.1.4P8 7-Mode
>
>
>
> The disk at 0d.01.13 was broken, so I pulled it a few days ago – now that
> the replacment disk has arrived, I tried to plug it in and this is what
> happens:
>
>
>
> Thu Oct 22 14:30:10 CEST [riddler:scsi.cmd.notReadyCondition:notice]: Disk
> device 0d.01.13: Device returns not yet ready: CDB 0x1b: Sense Data
> SCSI:not ready - Drive spinning up (0x2 - 0x4 0x1 0x28)(2).
>
>
>
> The interesting thing, I can see the disk by using `storage show disk`:
>
>
>
> riddler> storage show disk
>
> DISK SHELF BAY SERIAL VENDOR MODEL REV
>
> --------------------- --------- ---------------- -------- ---------- ----
>
> 0d.01.13 1 13 J838WSRL NETAPP X302_HJUPI NA04
>
>
>
> But I cannot see it in disk show.
>
>
>
> riddler> disk show 0d.01.13
>
> DISK OWNER POOL SERIAL NUMBER HOME
>
> ------------ ------------- ----- -------------
> -------------
>
>
>
> Sysconfig –v tells me, that there is a newer firmware on the „new“ disk:
>
>
>
> 01.13: NETAPP X302_HJUPI01TSSM NA04 847.5GB (1953525168
> 512B/sect)
>
> 01.14: NETAPP X302_WVULC01TSSA NA03 847.5GB (1953525168
> 512B/sect)
>
>
>
> So I first tried to downgrade firmware using:
>
>
>
> disk_fw_update –R 0d.01.13
>
>
>
> but that did not really work out, no error message, but just no change on
> the firmware of the disk – also nothing in the logs.
>
> So I read somewhere that this might be due to a bad/incompatible label on
> the disk and found out that the only way to get rid of the label is to boot
> into maintenance mode (thus having a downtime of parts of the HA-pair :-/)
> and recreate the label there.
>
>
>
> So I booted into maintenance mode and have run the following command:
>
>
>
> *> label makespare 0d.01.13
>
> *********************************************************
>
> Label editing is highly dangerous and should be attempted
>
> ONLY by NetApp Customer Support Engineers.
>
> Misuse can cause ALL data to be permanently lost.
>
> *********************************************************
>
> 'Label save': Error while trying to save Label 1 on disk 0d.03.22: error
> 15.
>
> Label save: 1 of 35 disks could not save Label 1.
>
> 'Label save': Error while trying to save Label 2 on disk 0d.03.22: error
> 15.
>
> Label save: 1 of 35 disks could not save Label 2.
>
> The current set of labels on the disks have been saved. You can
>
> use the 'label restore' command to revert to these labels, the
>
> 'label keep' command to erase these labels, or 'label save' to
>
> preserve the current set of labels.
>
> *********************************************************
>
> *** Run WAFL recovery tools (wafliron or wafliron_oc) after this.
> ***
>
>
>
> Disk 0d.01.13 does not exist.
>
>
>
> The last line suggests, that the disk still does not get recognized
> properly – and additionally, it (for whatever reason) claims to not be able
> to write the label 1 on disk 0d.03.22, although this is there and online:
>
>
>
> riddler> disk show 0d.03.22
>
> DISK OWNER POOL SERIAL NUMBER HOME
>
> ------------ ------------- ----- -------------
> -------------
>
> 0d.03.22 riddler (142248200) Pool0 N002XRML
> riddler (142248200)
>
>
>
> I’ve stopped my expiriments at this point and am very thankful for fresh
> ideas J
>
>
>
> Best,
>
>
>
> *Alexander Griesser*
>
> Head of Systems Operations
>
>
>
> ANEXIA Internetdienstleistungs GmbH
>
>
>
> E-Mail: AGriesser@anexia-it.com
>
> Web: http://www.anexia-it.com
>
>
>
> Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
>
> Geschäftsführer: Alexander Windbichler
>
> Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT
> U63216601
>
>
>
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> http://www.teaparty.net/mailman/listinfo/toasters
>
>
Re: Used Replacement disk not recognized [ In reply to ]
make sure to add the -s unowned, because the disk is owned, but not by the
local node.

On Thu, Oct 22, 2015 at 10:51 AM, andrei.borzenkov@ts.fujitsu.com <
andrei.borzenkov@ts.fujitsu.com> wrote:

> disk assign 0d.01.13 -f
>
>
>
> ---
>
> With best regards
>
>
>
> *Andre**i** Borzenkov*
>
> Senior system engineer
>
> FTS WEMEAI RUC RU SC TMS FOS
>
> [image: cid:image001.gif@01CBF835.B3FEDA90]
>
> *FUJITSU*
>
> Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
>
> Tel.: +7 495 730 62 20 ( reception)
>
> Mob.: +7 916 678 7208
>
> Fax: +7 495 730 62 14
>
> E-mail: Andrei.Borzenkov@ts.fujitsu.com
>
> Web: ru.fujitsu.com <http://ts.fujitsu.com/>
>
> Company details: ts.fujitsu.com/imprint
> <http://ts.fujitsu.com/imprint.html>
>
> This communication contains information that is confidential, proprietary
> in nature and/or privileged. It is for the exclusive use of the intended
> recipient(s). If you are not the intended recipient(s) or the person
> responsible for delivering it to the intended recipient(s), please note
> that any form of dissemination, distribution or copying of this
> communication is strictly prohibited and may be unlawful. If you have
> received this communication in error, please immediately notify the sender
> and delete the original communication. Thank you for your cooperation.
>
> Please be advised that neither Fujitsu, its affiliates, its employees or
> agents accept liability for any errors, omissions or damages caused by
> delays of receipt or by any virus infection in this message or its
> attachments, or which may otherwise arise as a result of this e-mail
> transmission.
>
>
>
> *From:* Alexander Griesser [mailto:AGriesser@anexia-it.com]
> *Sent:* Thursday, October 22, 2015 5:50 PM
> *To:* Borzenkov, Andrei; toasters@teaparty.net
> *Subject:* AW: Used Replacement disk not recognized
>
>
>
> Bingo:
>
>
>
> riddler> disk show -v
>
> DISK OWNER POOL SERIAL NUMBER
> HOME CHKSUM
>
> ------------ ------------- ----- -------------
> ------------- ------
>
> 0d.01.13 w-nas-04 (142255463) Pool0 J838WSRL
> w-nas-04 (142255463) Block
>
>
>
> „w-nas-04“ is none of my filers – so I need to remove the ownership here
> now, I guess.
>
>
>
> riddler*> disk remove_ownership 0d.01.13 -f
>
> disk remove_ownership: Disk 0d.01.13 is not owned by this node.
>
> riddler*> disk reassign -o w-nas-04 -n riddler
>
> disk reassign: Can only be run in maintenance mode or during takeover in
> advanced mode
>
> riddler*>
>
>
>
> So, I guess I have to – again – boot into maintenance mode or is there a
> trick which would allow me to remove the ownership without having to shut
> everything down again?
>
>
>
> Best,
>
>
>
> *Alexander Griesser*
>
> Head of Systems Operations
>
>
>
> ANEXIA Internetdienstleistungs GmbH
>
>
>
> E-Mail: AGriesser@anexia-it.com
>
> Web: http://www.anexia-it.com
>
>
>
> Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
>
> Geschäftsführer: Alexander Windbichler
>
> Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT
> U63216601
>
>
>
> *Von:* andrei.borzenkov@ts.fujitsu.com [
> mailto:andrei.borzenkov@ts.fujitsu.com <andrei.borzenkov@ts.fujitsu.com>]
> *Gesendet:* Donnerstag, 22. Oktober 2015 16:47
> *An:* Alexander Griesser <AGriesser@anexia-it.com>; toasters@teaparty.net
> *Betreff:* RE: Used Replacement disk not recognized
>
>
>
> Does it appear in “disk show -v”?
>
>
>
> ---
>
> With best regards
>
>
>
> *Andre**i** Borzenkov*
>
> Senior system engineer
>
> FTS WEMEAI RUC RU SC TMS FOS
>
> [image: cid:image001.gif@01CBF835.B3FEDA90]
>
> *FUJITSU*
>
> Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
>
> Tel.: +7 495 730 62 20 ( reception)
>
> Mob.: +7 916 678 7208
>
> Fax: +7 495 730 62 14
>
> E-mail: Andrei.Borzenkov@ts.fujitsu.com
>
> Web: ru.fujitsu.com <http://ts.fujitsu.com/>
>
> Company details: ts.fujitsu.com/imprint
> <http://ts.fujitsu.com/imprint.html>
>
> This communication contains information that is confidential, proprietary
> in nature and/or privileged. It is for the exclusive use of the intended
> recipient(s). If you are not the intended recipient(s) or the person
> responsible for delivering it to the intended recipient(s), please note
> that any form of dissemination, distribution or copying of this
> communication is strictly prohibited and may be unlawful. If you have
> received this communication in error, please immediately notify the sender
> and delete the original communication. Thank you for your cooperation.
>
> Please be advised that neither Fujitsu, its affiliates, its employees or
> agents accept liability for any errors, omissions or damages caused by
> delays of receipt or by any virus infection in this message or its
> attachments, or which may otherwise arise as a result of this e-mail
> transmission.
>
>
>
> *From:* Alexander Griesser [mailto:AGriesser@anexia-it.com
> <AGriesser@anexia-it.com>]
> *Sent:* Thursday, October 22, 2015 5:44 PM
> *To:* Borzenkov, Andrei; toasters@teaparty.net
> *Subject:* AW: Used Replacement disk not recognized
>
>
>
> Hi Andrei,
>
>
>
> the last command states „Disk 0d.01.13 does not exist“, I don’t think that
> this is cool…
>
> The problem is that even after the reboot, the disk does not show up
> anywhere except for sysconfig or storage show disk.
>
>
>
> Best,
>
>
>
> *Alexander Griesser*
>
> Head of Systems Operations
>
>
>
> ANEXIA Internetdienstleistungs GmbH
>
>
>
> E-Mail: AGriesser@anexia-it.com
>
> Web: http://www.anexia-it.com
>
>
>
> Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
>
> Geschäftsführer: Alexander Windbichler
>
> Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT
> U63216601
>
>
>
> *Von:* andrei.borzenkov@ts.fujitsu.com [
> mailto:andrei.borzenkov@ts.fujitsu.com <andrei.borzenkov@ts.fujitsu.com>]
> *Gesendet:* Donnerstag, 22. Oktober 2015 16:41
> *An:* Alexander Griesser <AGriesser@anexia-it.com>; toasters@teaparty.net
> *Betreff:* RE: Used Replacement disk not recognized
>
>
>
> Firmware is not older or newer – those two disks are from different
> manufacturers so they have different firmware; both are using the latest
> versions publically available.
>
>
>
> KB
> https://kb.netapp.com/support/index?page=content&id=2010827&locale=en_US&access=s implies
> that Drive spinning up may be seen with X302 and is otherwise harmless,
> although there are so many different variants of X302 it is hard to guess
> whether it applies in your case. I would suggest open the case.
>
>
>
> Last command you pasted suggests that everything is OK now; or what is
> remaining problem?
>
>
>
> ---
>
> With best regards
>
>
>
> *Andre**i** Borzenkov*
>
> Senior system engineer
>
> FTS WEMEAI RUC RU SC TMS FOS
>
> [image: cid:image001.gif@01CBF835.B3FEDA90]
>
> *FUJITSU*
>
> Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
>
> Tel.: +7 495 730 62 20 ( reception)
>
> Mob.: +7 916 678 7208
>
> Fax: +7 495 730 62 14
>
> E-mail: Andrei.Borzenkov@ts.fujitsu.com
>
> Web: ru.fujitsu.com <http://ts.fujitsu.com/>
>
> Company details: ts.fujitsu.com/imprint
> <http://ts.fujitsu.com/imprint.html>
>
> This communication contains information that is confidential, proprietary
> in nature and/or privileged. It is for the exclusive use of the intended
> recipient(s). If you are not the intended recipient(s) or the person
> responsible for delivering it to the intended recipient(s), please note
> that any form of dissemination, distribution or copying of this
> communication is strictly prohibited and may be unlawful. If you have
> received this communication in error, please immediately notify the sender
> and delete the original communication. Thank you for your cooperation.
>
> Please be advised that neither Fujitsu, its affiliates, its employees or
> agents accept liability for any errors, omissions or damages caused by
> delays of receipt or by any virus infection in this message or its
> attachments, or which may otherwise arise as a result of this e-mail
> transmission.
>
>
>
> *From:* toasters-bounces@teaparty.net [
> mailto:toasters-bounces@teaparty.net <toasters-bounces@teaparty.net>] *On
> Behalf Of *Alexander Griesser
> *Sent:* Thursday, October 22, 2015 5:06 PM
> *To:* toasters@teaparty.net
> *Subject:* Used Replacement disk not recognized
>
>
>
> Hey everybody,
>
>
>
> I just had the pleasure to try to replace a faulty disk in a filer which
> is out-of-service with a „new“ one from after market.
>
> Of course, this didn’t work out as easy as it should, so maybe some could
> help me get this disk going.
>
>
>
> Here are the facts:
>
> FAS2040
>
> NetApp Release 8.1.4P8 7-Mode
>
>
>
> The disk at 0d.01.13 was broken, so I pulled it a few days ago – now that
> the replacment disk has arrived, I tried to plug it in and this is what
> happens:
>
>
>
> Thu Oct 22 14:30:10 CEST [riddler:scsi.cmd.notReadyCondition:notice]: Disk
> device 0d.01.13: Device returns not yet ready: CDB 0x1b: Sense Data
> SCSI:not ready - Drive spinning up (0x2 - 0x4 0x1 0x28)(2).
>
>
>
> The interesting thing, I can see the disk by using `storage show disk`:
>
>
>
> riddler> storage show disk
>
> DISK SHELF BAY SERIAL VENDOR MODEL REV
>
> --------------------- --------- ---------------- -------- ---------- ----
>
> 0d.01.13 1 13 J838WSRL NETAPP X302_HJUPI NA04
>
>
>
> But I cannot see it in disk show.
>
>
>
> riddler> disk show 0d.01.13
>
> DISK OWNER POOL SERIAL NUMBER HOME
>
> ------------ ------------- ----- -------------
> -------------
>
>
>
> Sysconfig –v tells me, that there is a newer firmware on the „new“ disk:
>
>
>
> 01.13: NETAPP X302_HJUPI01TSSM NA04 847.5GB (1953525168
> 512B/sect)
>
> 01.14: NETAPP X302_WVULC01TSSA NA03 847.5GB (1953525168
> 512B/sect)
>
>
>
> So I first tried to downgrade firmware using:
>
>
>
> disk_fw_update –R 0d.01.13
>
>
>
> but that did not really work out, no error message, but just no change on
> the firmware of the disk – also nothing in the logs.
>
> So I read somewhere that this might be due to a bad/incompatible label on
> the disk and found out that the only way to get rid of the label is to boot
> into maintenance mode (thus having a downtime of parts of the HA-pair :-/)
> and recreate the label there.
>
>
>
> So I booted into maintenance mode and have run the following command:
>
>
>
> *> label makespare 0d.01.13
>
> *********************************************************
>
> Label editing is highly dangerous and should be attempted
>
> ONLY by NetApp Customer Support Engineers.
>
> Misuse can cause ALL data to be permanently lost.
>
> *********************************************************
>
> 'Label save': Error while trying to save Label 1 on disk 0d.03.22: error
> 15.
>
> Label save: 1 of 35 disks could not save Label 1.
>
> 'Label save': Error while trying to save Label 2 on disk 0d.03.22: error
> 15.
>
> Label save: 1 of 35 disks could not save Label 2.
>
> The current set of labels on the disks have been saved. You can
>
> use the 'label restore' command to revert to these labels, the
>
> 'label keep' command to erase these labels, or 'label save' to
>
> preserve the current set of labels.
>
> *********************************************************
>
> *** Run WAFL recovery tools (wafliron or wafliron_oc) after this.
> ***
>
>
>
> Disk 0d.01.13 does not exist.
>
>
>
> The last line suggests, that the disk still does not get recognized
> properly – and additionally, it (for whatever reason) claims to not be able
> to write the label 1 on disk 0d.03.22, although this is there and online:
>
>
>
> riddler> disk show 0d.03.22
>
> DISK OWNER POOL SERIAL NUMBER HOME
>
> ------------ ------------- ----- -------------
> -------------
>
> 0d.03.22 riddler (142248200) Pool0 N002XRML
> riddler (142248200)
>
>
>
> I’ve stopped my expiriments at this point and am very thankful for fresh
> ideas J
>
>
>
> Best,
>
>
>
> *Alexander Griesser*
>
> Head of Systems Operations
>
>
>
> ANEXIA Internetdienstleistungs GmbH
>
>
>
> E-Mail: AGriesser@anexia-it.com
>
> Web: http://www.anexia-it.com
>
>
>
> Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
>
> Geschäftsführer: Alexander Windbichler
>
> Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT
> U63216601
>
>
>
> _______________________________________________
> Toasters mailing list
> Toasters@teaparty.net
> http://www.teaparty.net/mailman/listinfo/toasters
>
>
AW: Used Replacement disk not recognized [ In reply to ]
Alright, but now I do have a label issue I guess :)

An even older filer (FAS2040, OnTap 8.0.1P2 7-Mode) is writing this to the log when I insert one of the refurbished disks:

Fri Oct 23 20:24:52 CEST [megatron: raid.disk.missing:info]: Disk 0d.01.5 Shelf 1 Bay 5 [NETAPP X410_HVIPC288A15 NA00] S/N [JXV0TL2J] is missing from the system
Fri Oct 23 20:26:52 CEST [megatron: diskown.changingOwner:info]: changing ownership for disk 0d.01.5 (S/N JXXDSRMJ) from unowned (ID 4294967295) to megatron (ID 142227074)
Fri Oct 23 20:26:52 CEST [megatron: raid.assim.disk.nolabels:error]: Disk 0d.01.5 Shelf 1 Bay 5 [NETAPP X410_HVIPC288A15 NA02] S/N [JXXDSRMJ] has no valid labels. It will be taken out of service to prevent possible data loss.
Fri Oct 23 20:26:52 CEST [megatron: raid.config.disk.bad.label:error]: Disk 0d.01.5 Shelf 1 Bay 5 [NETAPP X410_HVIPC288A15 NA02] S/N [JXXDSRMJ] has bad label.
Fri Oct 23 20:26:52 CEST [megatron: callhome.dsk.label:CRITICAL]: Call home for DISK BAD LABEL
Fri Oct 23 20:27:00 CEST [megatron: monitor.globalStatus.nonCritical:warning]: Disk on adapter 0d, shelf 1, bay 5, failed.

I guess this means that I will have to erase the label here, correct?

aggr status -r
Broken disks

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
bad label 0d.01.5 0d 1 5 SA:B 0 SAS 15000 272000/557056000 274845/562884296

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] Im Auftrag von Alexander Griesser
Gesendet: Donnerstag, 22. Oktober 2015 16:55
An: andrei.borzenkov@ts.fujitsu.com; toasters@teaparty.net
Betreff: AW: Used Replacement disk not recognized

Many thanks!

riddler> vol status -s

Pool1 spare disks (empty)

Pool0 spare disks

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
Spare disks for block checksum
spare 0d.01.13 0d 1 13 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816
spare 0d.03.22 0d 3 22 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816

I'm up and running :)

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com> [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:52
An: Alexander Griesser <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: RE: Used Replacement disk not recognized

disk assign 0d.01.13 -f

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: Alexander Griesser [mailto:AGriesser@anexia-it.com]
Sent: Thursday, October 22, 2015 5:50 PM
To: Borzenkov, Andrei; toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: AW: Used Replacement disk not recognized

Bingo:

riddler> disk show -v
DISK OWNER POOL SERIAL NUMBER HOME CHKSUM
------------ ------------- ----- ------------- ------------- ------
0d.01.13 w-nas-04 (142255463) Pool0 J838WSRL w-nas-04 (142255463) Block

"w-nas-04" is none of my filers - so I need to remove the ownership here now, I guess.

riddler*> disk remove_ownership 0d.01.13 -f
disk remove_ownership: Disk 0d.01.13 is not owned by this node.
riddler*> disk reassign -o w-nas-04 -n riddler
disk reassign: Can only be run in maintenance mode or during takeover in advanced mode
riddler*>

So, I guess I have to - again - boot into maintenance mode or is there a trick which would allow me to remove the ownership without having to shut everything down again?

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com> [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:47
An: Alexander Griesser <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: RE: Used Replacement disk not recognized

Does it appear in "disk show -v"?

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: Alexander Griesser [mailto:AGriesser@anexia-it.com]
Sent: Thursday, October 22, 2015 5:44 PM
To: Borzenkov, Andrei; toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: AW: Used Replacement disk not recognized

Hi Andrei,

the last command states "Disk 0d.01.13 does not exist", I don't think that this is cool...
The problem is that even after the reboot, the disk does not show up anywhere except for sysconfig or storage show disk.

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com> [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:41
An: Alexander Griesser <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: RE: Used Replacement disk not recognized

Firmware is not older or newer - those two disks are from different manufacturers so they have different firmware; both are using the latest versions publically available.

KB https://kb.netapp.com/support/index?page=content&id=2010827&locale=en_US&access=s implies that Drive spinning up may be seen with X302 and is otherwise harmless, although there are so many different variants of X302 it is hard to guess whether it applies in your case. I would suggest open the case.

Last command you pasted suggests that everything is OK now; or what is remaining problem?

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: toasters-bounces@teaparty.net<mailto:toasters-bounces@teaparty.net> [mailto:toasters-bounces@teaparty.net] On Behalf Of Alexander Griesser
Sent: Thursday, October 22, 2015 5:06 PM
To: toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: Used Replacement disk not recognized

Hey everybody,

I just had the pleasure to try to replace a faulty disk in a filer which is out-of-service with a "new" one from after market.
Of course, this didn't work out as easy as it should, so maybe some could help me get this disk going.

Here are the facts:
FAS2040
NetApp Release 8.1.4P8 7-Mode

The disk at 0d.01.13 was broken, so I pulled it a few days ago - now that the replacment disk has arrived, I tried to plug it in and this is what happens:

Thu Oct 22 14:30:10 CEST [riddler:scsi.cmd.notReadyCondition:notice]: Disk device 0d.01.13: Device returns not yet ready: CDB 0x1b: Sense Data SCSI:not ready - Drive spinning up (0x2 - 0x4 0x1 0x28)(2).

The interesting thing, I can see the disk by using `storage show disk`:

riddler> storage show disk
DISK SHELF BAY SERIAL VENDOR MODEL REV
--------------------- --------- ---------------- -------- ---------- ----
0d.01.13 1 13 J838WSRL NETAPP X302_HJUPI NA04

But I cannot see it in disk show.

riddler> disk show 0d.01.13
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------

Sysconfig -v tells me, that there is a newer firmware on the "new" disk:

01.13: NETAPP X302_HJUPI01TSSM NA04 847.5GB (1953525168 512B/sect)
01.14: NETAPP X302_WVULC01TSSA NA03 847.5GB (1953525168 512B/sect)

So I first tried to downgrade firmware using:

disk_fw_update -R 0d.01.13

but that did not really work out, no error message, but just no change on the firmware of the disk - also nothing in the logs.
So I read somewhere that this might be due to a bad/incompatible label on the disk and found out that the only way to get rid of the label is to boot into maintenance mode (thus having a downtime of parts of the HA-pair :-/) and recreate the label there.

So I booted into maintenance mode and have run the following command:

*> label makespare 0d.01.13
*********************************************************
Label editing is highly dangerous and should be attempted
ONLY by NetApp Customer Support Engineers.
Misuse can cause ALL data to be permanently lost.
*********************************************************
'Label save': Error while trying to save Label 1 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 1.
'Label save': Error while trying to save Label 2 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 2.
The current set of labels on the disks have been saved. You can
use the 'label restore' command to revert to these labels, the
'label keep' command to erase these labels, or 'label save' to
preserve the current set of labels.
*********************************************************
*** Run WAFL recovery tools (wafliron or wafliron_oc) after this. ***

Disk 0d.01.13 does not exist.

The last line suggests, that the disk still does not get recognized properly - and additionally, it (for whatever reason) claims to not be able to write the label 1 on disk 0d.03.22, although this is there and online:

riddler> disk show 0d.03.22
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------
0d.03.22 riddler (142248200) Pool0 N002XRML riddler (142248200)

I've stopped my expiriments at this point and am very thankful for fresh ideas :)

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
RE: Used Replacement disk not recognized [ In reply to ]
Alex,

If you were running 8.02 there is a nice fix. ☹

8.01 you are out of luck. https://kb.netapp.com/support/index?page=content&id=2011366&locale=en_US&access=s

Have you got a newer system you can plug it into to clear it?

Regards




Duncan Cummings
NetApp Specialist
Interactive Pty Ltd
Telephone +61 7 3323 0800
Facsimile +61 7 3323 0899
Mobile +61 403 383 050
www.interactive.com.au<http://www.interactive.com.au>

-------Confidentiality & Legal Privilege-------------
"This email is intended for the named recipient only. The information contained in this message may be confidential, or commercially sensitive. If you are not the intended recipient you must not reproduce or distribute any part of the email, disclose its contents to any other party, or take any action in reliance on it. If you have received this email in error, please contact the sender immediately. Please delete this message from your computer. Confidentiality and legal privilege are not waived or lost by reason of mistaken delivery to you."
From: toasters-bounces@teaparty.net [mailto:toasters-bounces@teaparty.net] On Behalf Of Alexander Griesser
Sent: Saturday, 24 October 2015 4:31 AM
To: toasters@teaparty.net
Subject: AW: Used Replacement disk not recognized

Alright, but now I do have a label issue I guess ☺

An even older filer (FAS2040, OnTap 8.0.1P2 7-Mode) is writing this to the log when I insert one of the refurbished disks:

Fri Oct 23 20:24:52 CEST [megatron: raid.disk.missing:info]: Disk 0d.01.5 Shelf 1 Bay 5 [NETAPP X410_HVIPC288A15 NA00] S/N [JXV0TL2J] is missing from the system
Fri Oct 23 20:26:52 CEST [megatron: diskown.changingOwner:info]: changing ownership for disk 0d.01.5 (S/N JXXDSRMJ) from unowned (ID 4294967295) to megatron (ID 142227074)
Fri Oct 23 20:26:52 CEST [megatron: raid.assim.disk.nolabels:error]: Disk 0d.01.5 Shelf 1 Bay 5 [NETAPP X410_HVIPC288A15 NA02] S/N [JXXDSRMJ] has no valid labels. It will be taken out of service to prevent possible data loss.
Fri Oct 23 20:26:52 CEST [megatron: raid.config.disk.bad.label:error]: Disk 0d.01.5 Shelf 1 Bay 5 [NETAPP X410_HVIPC288A15 NA02] S/N [JXXDSRMJ] has bad label.
Fri Oct 23 20:26:52 CEST [megatron: callhome.dsk.label:CRITICAL]: Call home for DISK BAD LABEL
Fri Oct 23 20:27:00 CEST [megatron: monitor.globalStatus.nonCritical:warning]: Disk on adapter 0d, shelf 1, bay 5, failed.

I guess this means that I will have to erase the label here, correct?

aggr status -r
Broken disks

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
bad label 0d.01.5 0d 1 5 SA:B 0 SAS 15000 272000/557056000 274845/562884296

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: toasters-bounces@teaparty.net<mailto:toasters-bounces@teaparty.net> [mailto:toasters-bounces@teaparty.net] Im Auftrag von Alexander Griesser
Gesendet: Donnerstag, 22. Oktober 2015 16:55
An: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: AW: Used Replacement disk not recognized

Many thanks!

riddler> vol status -s

Pool1 spare disks (empty)

Pool0 spare disks

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
Spare disks for block checksum
spare 0d.01.13 0d 1 13 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816
spare 0d.03.22 0d 3 22 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816

I’m up and running ☺

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com> [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:52
An: Alexander Griesser <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: RE: Used Replacement disk not recognized

disk assign 0d.01.13 -f

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: Alexander Griesser [mailto:AGriesser@anexia-it.com]
Sent: Thursday, October 22, 2015 5:50 PM
To: Borzenkov, Andrei; toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: AW: Used Replacement disk not recognized

Bingo:

riddler> disk show -v
DISK OWNER POOL SERIAL NUMBER HOME CHKSUM
------------ ------------- ----- ------------- ------------- ------
0d.01.13 w-nas-04 (142255463) Pool0 J838WSRL w-nas-04 (142255463) Block

„w-nas-04“ is none of my filers – so I need to remove the ownership here now, I guess.

riddler*> disk remove_ownership 0d.01.13 -f
disk remove_ownership: Disk 0d.01.13 is not owned by this node.
riddler*> disk reassign -o w-nas-04 -n riddler
disk reassign: Can only be run in maintenance mode or during takeover in advanced mode
riddler*>

So, I guess I have to – again – boot into maintenance mode or is there a trick which would allow me to remove the ownership without having to shut everything down again?

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com> [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:47
An: Alexander Griesser <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: RE: Used Replacement disk not recognized

Does it appear in “disk show -v”?

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: Alexander Griesser [mailto:AGriesser@anexia-it.com]
Sent: Thursday, October 22, 2015 5:44 PM
To: Borzenkov, Andrei; toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: AW: Used Replacement disk not recognized

Hi Andrei,

the last command states „Disk 0d.01.13 does not exist“, I don’t think that this is cool…
The problem is that even after the reboot, the disk does not show up anywhere except for sysconfig or storage show disk.

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com> [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:41
An: Alexander Griesser <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: RE: Used Replacement disk not recognized

Firmware is not older or newer – those two disks are from different manufacturers so they have different firmware; both are using the latest versions publically available.

KB https://kb.netapp.com/support/index?page=content&id=2010827&locale=en_US&access=s implies that Drive spinning up may be seen with X302 and is otherwise harmless, although there are so many different variants of X302 it is hard to guess whether it applies in your case. I would suggest open the case.

Last command you pasted suggests that everything is OK now; or what is remaining problem?

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: toasters-bounces@teaparty.net<mailto:toasters-bounces@teaparty.net> [mailto:toasters-bounces@teaparty.net] On Behalf Of Alexander Griesser
Sent: Thursday, October 22, 2015 5:06 PM
To: toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: Used Replacement disk not recognized

Hey everybody,

I just had the pleasure to try to replace a faulty disk in a filer which is out-of-service with a „new“ one from after market.
Of course, this didn’t work out as easy as it should, so maybe some could help me get this disk going.

Here are the facts:
FAS2040
NetApp Release 8.1.4P8 7-Mode

The disk at 0d.01.13 was broken, so I pulled it a few days ago – now that the replacment disk has arrived, I tried to plug it in and this is what happens:

Thu Oct 22 14:30:10 CEST [riddler:scsi.cmd.notReadyCondition:notice]: Disk device 0d.01.13: Device returns not yet ready: CDB 0x1b: Sense Data SCSI:not ready - Drive spinning up (0x2 - 0x4 0x1 0x28)(2).

The interesting thing, I can see the disk by using `storage show disk`:

riddler> storage show disk
DISK SHELF BAY SERIAL VENDOR MODEL REV
--------------------- --------- ---------------- -------- ---------- ----
0d.01.13 1 13 J838WSRL NETAPP X302_HJUPI NA04

But I cannot see it in disk show.

riddler> disk show 0d.01.13
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------

Sysconfig –v tells me, that there is a newer firmware on the „new“ disk:

01.13: NETAPP X302_HJUPI01TSSM NA04 847.5GB (1953525168 512B/sect)
01.14: NETAPP X302_WVULC01TSSA NA03 847.5GB (1953525168 512B/sect)

So I first tried to downgrade firmware using:

disk_fw_update –R 0d.01.13

but that did not really work out, no error message, but just no change on the firmware of the disk – also nothing in the logs.
So I read somewhere that this might be due to a bad/incompatible label on the disk and found out that the only way to get rid of the label is to boot into maintenance mode (thus having a downtime of parts of the HA-pair :-/) and recreate the label there.

So I booted into maintenance mode and have run the following command:

*> label makespare 0d.01.13
*********************************************************
Label editing is highly dangerous and should be attempted
ONLY by NetApp Customer Support Engineers.
Misuse can cause ALL data to be permanently lost.
*********************************************************
'Label save': Error while trying to save Label 1 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 1.
'Label save': Error while trying to save Label 2 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 2.
The current set of labels on the disks have been saved. You can
use the 'label restore' command to revert to these labels, the
'label keep' command to erase these labels, or 'label save' to
preserve the current set of labels.
*********************************************************
*** Run WAFL recovery tools (wafliron or wafliron_oc) after this. ***

Disk 0d.01.13 does not exist.

The last line suggests, that the disk still does not get recognized properly – and additionally, it (for whatever reason) claims to not be able to write the label 1 on disk 0d.03.22, although this is there and online:

riddler> disk show 0d.03.22
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------
0d.03.22 riddler (142248200) Pool0 N002XRML riddler (142248200)

I’ve stopped my expiriments at this point and am very thankful for fresh ideas ☺

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
AW: Used Replacement disk not recognized [ In reply to ]
Hi Duncan,

unfortunately, I do not have a newer system with a free slot to plug the disk in there, but I’ll try to find one before I start the upgrade to 8.0.2 ☺

Thanks, will keep you posted!

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: Duncan Cummings [mailto:dcummings@interactive.com.au]
Gesendet: Sonntag, 25. Oktober 2015 02:45
An: Alexander Griesser <AGriesser@anexia-it.com>; toasters@teaparty.net
Betreff: RE: Used Replacement disk not recognized

Alex,

If you were running 8.02 there is a nice fix. ☹

8.01 you are out of luck. https://kb.netapp.com/support/index?page=content&id=2011366&locale=en_US&access=s

Have you got a newer system you can plug it into to clear it?

Regards




Duncan Cummings
NetApp Specialist
Interactive Pty Ltd
Telephone +61 7 3323 0800
Facsimile +61 7 3323 0899
Mobile +61 403 383 050
www.interactive.com.au<http://www.interactive.com.au>

-------Confidentiality & Legal Privilege-------------
"This email is intended for the named recipient only. The information contained in this message may be confidential, or commercially sensitive. If you are not the intended recipient you must not reproduce or distribute any part of the email, disclose its contents to any other party, or take any action in reliance on it. If you have received this email in error, please contact the sender immediately. Please delete this message from your computer. Confidentiality and legal privilege are not waived or lost by reason of mistaken delivery to you."
From: toasters-bounces@teaparty.net<mailto:toasters-bounces@teaparty.net> [mailto:toasters-bounces@teaparty.net] On Behalf Of Alexander Griesser
Sent: Saturday, 24 October 2015 4:31 AM
To: toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: AW: Used Replacement disk not recognized

Alright, but now I do have a label issue I guess ☺

An even older filer (FAS2040, OnTap 8.0.1P2 7-Mode) is writing this to the log when I insert one of the refurbished disks:

Fri Oct 23 20:24:52 CEST [megatron: raid.disk.missing:info]: Disk 0d.01.5 Shelf 1 Bay 5 [NETAPP X410_HVIPC288A15 NA00] S/N [JXV0TL2J] is missing from the system
Fri Oct 23 20:26:52 CEST [megatron: diskown.changingOwner:info]: changing ownership for disk 0d.01.5 (S/N JXXDSRMJ) from unowned (ID 4294967295) to megatron (ID 142227074)
Fri Oct 23 20:26:52 CEST [megatron: raid.assim.disk.nolabels:error]: Disk 0d.01.5 Shelf 1 Bay 5 [NETAPP X410_HVIPC288A15 NA02] S/N [JXXDSRMJ] has no valid labels. It will be taken out of service to prevent possible data loss.
Fri Oct 23 20:26:52 CEST [megatron: raid.config.disk.bad.label:error]: Disk 0d.01.5 Shelf 1 Bay 5 [NETAPP X410_HVIPC288A15 NA02] S/N [JXXDSRMJ] has bad label.
Fri Oct 23 20:26:52 CEST [megatron: callhome.dsk.label:CRITICAL]: Call home for DISK BAD LABEL
Fri Oct 23 20:27:00 CEST [megatron: monitor.globalStatus.nonCritical:warning]: Disk on adapter 0d, shelf 1, bay 5, failed.

I guess this means that I will have to erase the label here, correct?

aggr status -r
Broken disks

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
bad label 0d.01.5 0d 1 5 SA:B 0 SAS 15000 272000/557056000 274845/562884296

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: toasters-bounces@teaparty.net<mailto:toasters-bounces@teaparty.net> [mailto:toasters-bounces@teaparty.net] Im Auftrag von Alexander Griesser
Gesendet: Donnerstag, 22. Oktober 2015 16:55
An: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: AW: Used Replacement disk not recognized

Many thanks!

riddler> vol status -s

Pool1 spare disks (empty)

Pool0 spare disks

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
Spare disks for block checksum
spare 0d.01.13 0d 1 13 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816
spare 0d.03.22 0d 3 22 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816

I’m up and running ☺

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com> [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:52
An: Alexander Griesser <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: RE: Used Replacement disk not recognized

disk assign 0d.01.13 -f

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: Alexander Griesser [mailto:AGriesser@anexia-it.com]
Sent: Thursday, October 22, 2015 5:50 PM
To: Borzenkov, Andrei; toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: AW: Used Replacement disk not recognized

Bingo:

riddler> disk show -v
DISK OWNER POOL SERIAL NUMBER HOME CHKSUM
------------ ------------- ----- ------------- ------------- ------
0d.01.13 w-nas-04 (142255463) Pool0 J838WSRL w-nas-04 (142255463) Block

„w-nas-04“ is none of my filers – so I need to remove the ownership here now, I guess.

riddler*> disk remove_ownership 0d.01.13 -f
disk remove_ownership: Disk 0d.01.13 is not owned by this node.
riddler*> disk reassign -o w-nas-04 -n riddler
disk reassign: Can only be run in maintenance mode or during takeover in advanced mode
riddler*>

So, I guess I have to – again – boot into maintenance mode or is there a trick which would allow me to remove the ownership without having to shut everything down again?

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com> [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:47
An: Alexander Griesser <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: RE: Used Replacement disk not recognized

Does it appear in “disk show -v”?

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: Alexander Griesser [mailto:AGriesser@anexia-it.com]
Sent: Thursday, October 22, 2015 5:44 PM
To: Borzenkov, Andrei; toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: AW: Used Replacement disk not recognized

Hi Andrei,

the last command states „Disk 0d.01.13 does not exist“, I don’t think that this is cool…
The problem is that even after the reboot, the disk does not show up anywhere except for sysconfig or storage show disk.

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com> [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:41
An: Alexander Griesser <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: RE: Used Replacement disk not recognized

Firmware is not older or newer – those two disks are from different manufacturers so they have different firmware; both are using the latest versions publically available.

KB https://kb.netapp.com/support/index?page=content&id=2010827&locale=en_US&access=s implies that Drive spinning up may be seen with X302 and is otherwise harmless, although there are so many different variants of X302 it is hard to guess whether it applies in your case. I would suggest open the case.

Last command you pasted suggests that everything is OK now; or what is remaining problem?

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
[cid:image001.gif@01CBF835.B3FEDA90]
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: toasters-bounces@teaparty.net<mailto:toasters-bounces@teaparty.net> [mailto:toasters-bounces@teaparty.net] On Behalf Of Alexander Griesser
Sent: Thursday, October 22, 2015 5:06 PM
To: toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: Used Replacement disk not recognized

Hey everybody,

I just had the pleasure to try to replace a faulty disk in a filer which is out-of-service with a „new“ one from after market.
Of course, this didn’t work out as easy as it should, so maybe some could help me get this disk going.

Here are the facts:
FAS2040
NetApp Release 8.1.4P8 7-Mode

The disk at 0d.01.13 was broken, so I pulled it a few days ago – now that the replacment disk has arrived, I tried to plug it in and this is what happens:

Thu Oct 22 14:30:10 CEST [riddler:scsi.cmd.notReadyCondition:notice]: Disk device 0d.01.13: Device returns not yet ready: CDB 0x1b: Sense Data SCSI:not ready - Drive spinning up (0x2 - 0x4 0x1 0x28)(2).

The interesting thing, I can see the disk by using `storage show disk`:

riddler> storage show disk
DISK SHELF BAY SERIAL VENDOR MODEL REV
--------------------- --------- ---------------- -------- ---------- ----
0d.01.13 1 13 J838WSRL NETAPP X302_HJUPI NA04

But I cannot see it in disk show.

riddler> disk show 0d.01.13
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------

Sysconfig –v tells me, that there is a newer firmware on the „new“ disk:

01.13: NETAPP X302_HJUPI01TSSM NA04 847.5GB (1953525168 512B/sect)
01.14: NETAPP X302_WVULC01TSSA NA03 847.5GB (1953525168 512B/sect)

So I first tried to downgrade firmware using:

disk_fw_update –R 0d.01.13

but that did not really work out, no error message, but just no change on the firmware of the disk – also nothing in the logs.
So I read somewhere that this might be due to a bad/incompatible label on the disk and found out that the only way to get rid of the label is to boot into maintenance mode (thus having a downtime of parts of the HA-pair :-/) and recreate the label there.

So I booted into maintenance mode and have run the following command:

*> label makespare 0d.01.13
*********************************************************
Label editing is highly dangerous and should be attempted
ONLY by NetApp Customer Support Engineers.
Misuse can cause ALL data to be permanently lost.
*********************************************************
'Label save': Error while trying to save Label 1 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 1.
'Label save': Error while trying to save Label 2 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 2.
The current set of labels on the disks have been saved. You can
use the 'label restore' command to revert to these labels, the
'label keep' command to erase these labels, or 'label save' to
preserve the current set of labels.
*********************************************************
*** Run WAFL recovery tools (wafliron or wafliron_oc) after this. ***

Disk 0d.01.13 does not exist.

The last line suggests, that the disk still does not get recognized properly – and additionally, it (for whatever reason) claims to not be able to write the label 1 on disk 0d.03.22, although this is there and online:

riddler> disk show 0d.03.22
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------
0d.03.22 riddler (142248200) Pool0 N002XRML riddler (142248200)

I’ve stopped my expiriments at this point and am very thankful for fresh ideas ☺

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601
AW: AW: Used Replacement disk not recognized [ In reply to ]
Hey,

this did work, indeed – at least the disk could be unfailed with `disk unfail –s` here on 8.1.4 in a different filer; I’m currently zeroing them and will then remove_ownership and bring them back to the older filer.

Thanks for your help!

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Samstag, 24. Oktober 2015 10:08
An: Alexander Griesser <AGriesser@anexia-it.com>
Cc: toasters@teaparty.net
Betreff: Re: AW: Used Replacement disk not recognized

Assuming it is indeed too new label (it does not look exactly as one) your options are

- update to 8.0.2 or above and use "disk unfail -s"

- put disk in filer with newer version and make it zeroed unowned spare


Messing with labels directly is unlikely going to help.

https://kb.netapp.com/support/index?page=content&id=2011366&locale=en_US

Отправлено с iPhone

23 окт. 2015 г., в 21:45, Alexander Griesser <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>> написал(а):
Alright, but now I do have a label issue I guess ☺

An even older filer (FAS2040, OnTap 8.0.1P2 7-Mode) is writing this to the log when I insert one of the refurbished disks:

Fri Oct 23 20:24:52 CEST [megatron: raid.disk.missing:info]: Disk 0d.01.5 Shelf 1 Bay 5 [NETAPP X410_HVIPC288A15 NA00] S/N [JXV0TL2J] is missing from the system
Fri Oct 23 20:26:52 CEST [megatron: diskown.changingOwner:info]: changing ownership for disk 0d.01.5 (S/N JXXDSRMJ) from unowned (ID 4294967295) to megatron (ID 142227074)
Fri Oct 23 20:26:52 CEST [megatron: raid.assim.disk.nolabels:error]: Disk 0d.01.5 Shelf 1 Bay 5 [NETAPP X410_HVIPC288A15 NA02] S/N [JXXDSRMJ] has no valid labels. It will be taken out of service to prevent possible data loss.
Fri Oct 23 20:26:52 CEST [megatron: raid.config.disk.bad.label:error]: Disk 0d.01.5 Shelf 1 Bay 5 [NETAPP X410_HVIPC288A15 NA02] S/N [JXXDSRMJ] has bad label.
Fri Oct 23 20:26:52 CEST [megatron: callhome.dsk.label:CRITICAL]: Call home for DISK BAD LABEL
Fri Oct 23 20:27:00 CEST [megatron: monitor.globalStatus.nonCritical:warning]: Disk on adapter 0d, shelf 1, bay 5, failed.

I guess this means that I will have to erase the label here, correct?

aggr status -r
Broken disks

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
bad label 0d.01.5 0d 1 5 SA:B 0 SAS 15000 272000/557056000 274845/562884296

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: toasters-bounces@teaparty.net<mailto:toasters-bounces@teaparty.net> [mailto:toasters-bounces@teaparty.net] Im Auftrag von Alexander Griesser
Gesendet: Donnerstag, 22. Oktober 2015 16:55
An: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: AW: Used Replacement disk not recognized

Many thanks!

riddler> vol status -s

Pool1 spare disks (empty)

Pool0 spare disks

RAID Disk Device HA SHELF BAY CHAN Pool Type RPM Used (MB/blks) Phys (MB/blks)
--------- ------ ------------- ---- ---- ---- ----- -------------- --------------
Spare disks for block checksum
spare 0d.01.13 0d 1 13 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816
spare 0d.03.22 0d 3 22 SA:B 0 BSAS 7200 847555/1735794176 847884/1736466816

I’m up and running ☺

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com> [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:52
An: Alexander Griesser <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: RE: Used Replacement disk not recognized

disk assign 0d.01.13 -f

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
<image001.gif>
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: Alexander Griesser [mailto:AGriesser@anexia-it.com]
Sent: Thursday, October 22, 2015 5:50 PM
To: Borzenkov, Andrei; toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: AW: Used Replacement disk not recognized

Bingo:

riddler> disk show -v
DISK OWNER POOL SERIAL NUMBER HOME CHKSUM
------------ ------------- ----- ------------- ------------- ------
0d.01.13 w-nas-04 (142255463) Pool0 J838WSRL w-nas-04 (142255463) Block

„w-nas-04“ is none of my filers – so I need to remove the ownership here now, I guess.

riddler*> disk remove_ownership 0d.01.13 -f
disk remove_ownership: Disk 0d.01.13 is not owned by this node.
riddler*> disk reassign -o w-nas-04 -n riddler
disk reassign: Can only be run in maintenance mode or during takeover in advanced mode
riddler*>

So, I guess I have to – again – boot into maintenance mode or is there a trick which would allow me to remove the ownership without having to shut everything down again?

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com> [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:47
An: Alexander Griesser <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: RE: Used Replacement disk not recognized

Does it appear in “disk show -v”?

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
<image001.gif>
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: Alexander Griesser [mailto:AGriesser@anexia-it.com]
Sent: Thursday, October 22, 2015 5:44 PM
To: Borzenkov, Andrei; toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: AW: Used Replacement disk not recognized

Hi Andrei,

the last command states „Disk 0d.01.13 does not exist“, I don’t think that this is cool…
The problem is that even after the reboot, the disk does not show up anywhere except for sysconfig or storage show disk.

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

Von: andrei.borzenkov@ts.fujitsu.com<mailto:andrei.borzenkov@ts.fujitsu.com> [mailto:andrei.borzenkov@ts.fujitsu.com]
Gesendet: Donnerstag, 22. Oktober 2015 16:41
An: Alexander Griesser <AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>>; toasters@teaparty.net<mailto:toasters@teaparty.net>
Betreff: RE: Used Replacement disk not recognized

Firmware is not older or newer – those two disks are from different manufacturers so they have different firmware; both are using the latest versions publically available.

KB https://kb.netapp.com/support/index?page=content&id=2010827&locale=en_US&access=s implies that Drive spinning up may be seen with X302 and is otherwise harmless, although there are so many different variants of X302 it is hard to guess whether it applies in your case. I would suggest open the case.

Last command you pasted suggests that everything is OK now; or what is remaining problem?

---
With best regards

Andrei Borzenkov
Senior system engineer
FTS WEMEAI RUC RU SC TMS FOS
<image001.gif>
FUJITSU
Zemlyanoy Val Street, 9, 105 064 Moscow, Russian Federation
Tel.: +7 495 730 62 20 ( reception)
Mob.: +7 916 678 7208
Fax: +7 495 730 62 14
E-mail: Andrei.Borzenkov@ts.fujitsu.com<mailto:Andrei.Borzenkov@ts.fujitsu.com>
Web: ru.fujitsu.com<http://ts.fujitsu.com/>
Company details: ts.fujitsu.com/imprint<http://ts.fujitsu.com/imprint.html>
This communication contains information that is confidential, proprietary in nature and/or privileged. It is for the exclusive use of the intended recipient(s). If you are not the intended recipient(s) or the person responsible for delivering it to the intended recipient(s), please note that any form of dissemination, distribution or copying of this communication is strictly prohibited and may be unlawful. If you have received this communication in error, please immediately notify the sender and delete the original communication. Thank you for your cooperation.
Please be advised that neither Fujitsu, its affiliates, its employees or agents accept liability for any errors, omissions or damages caused by delays of receipt or by any virus infection in this message or its attachments, or which may otherwise arise as a result of this e-mail transmission.

From: toasters-bounces@teaparty.net<mailto:toasters-bounces@teaparty.net> [mailto:toasters-bounces@teaparty.net] On Behalf Of Alexander Griesser
Sent: Thursday, October 22, 2015 5:06 PM
To: toasters@teaparty.net<mailto:toasters@teaparty.net>
Subject: Used Replacement disk not recognized

Hey everybody,

I just had the pleasure to try to replace a faulty disk in a filer which is out-of-service with a „new“ one from after market.
Of course, this didn’t work out as easy as it should, so maybe some could help me get this disk going.

Here are the facts:
FAS2040
NetApp Release 8.1.4P8 7-Mode

The disk at 0d.01.13 was broken, so I pulled it a few days ago – now that the replacment disk has arrived, I tried to plug it in and this is what happens:

Thu Oct 22 14:30:10 CEST [riddler:scsi.cmd.notReadyCondition:notice]: Disk device 0d.01.13: Device returns not yet ready: CDB 0x1b: Sense Data SCSI:not ready - Drive spinning up (0x2 - 0x4 0x1 0x28)(2).

The interesting thing, I can see the disk by using `storage show disk`:

riddler> storage show disk
DISK SHELF BAY SERIAL VENDOR MODEL REV
--------------------- --------- ---------------- -------- ---------- ----
0d.01.13 1 13 J838WSRL NETAPP X302_HJUPI NA04

But I cannot see it in disk show.

riddler> disk show 0d.01.13
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------

Sysconfig –v tells me, that there is a newer firmware on the „new“ disk:

01.13: NETAPP X302_HJUPI01TSSM NA04 847.5GB (1953525168 512B/sect)
01.14: NETAPP X302_WVULC01TSSA NA03 847.5GB (1953525168 512B/sect)

So I first tried to downgrade firmware using:

disk_fw_update –R 0d.01.13

but that did not really work out, no error message, but just no change on the firmware of the disk – also nothing in the logs.
So I read somewhere that this might be due to a bad/incompatible label on the disk and found out that the only way to get rid of the label is to boot into maintenance mode (thus having a downtime of parts of the HA-pair :-/) and recreate the label there.

So I booted into maintenance mode and have run the following command:

*> label makespare 0d.01.13
*********************************************************
Label editing is highly dangerous and should be attempted
ONLY by NetApp Customer Support Engineers.
Misuse can cause ALL data to be permanently lost.
*********************************************************
'Label save': Error while trying to save Label 1 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 1.
'Label save': Error while trying to save Label 2 on disk 0d.03.22: error 15.
Label save: 1 of 35 disks could not save Label 2.
The current set of labels on the disks have been saved. You can
use the 'label restore' command to revert to these labels, the
'label keep' command to erase these labels, or 'label save' to
preserve the current set of labels.
*********************************************************
*** Run WAFL recovery tools (wafliron or wafliron_oc) after this. ***

Disk 0d.01.13 does not exist.

The last line suggests, that the disk still does not get recognized properly – and additionally, it (for whatever reason) claims to not be able to write the label 1 on disk 0d.03.22, although this is there and online:

riddler> disk show 0d.03.22
DISK OWNER POOL SERIAL NUMBER HOME
------------ ------------- ----- ------------- -------------
0d.03.22 riddler (142248200) Pool0 N002XRML riddler (142248200)

I’ve stopped my expiriments at this point and am very thankful for fresh ideas ☺

Best,

Alexander Griesser
Head of Systems Operations

ANEXIA Internetdienstleistungs GmbH

E-Mail: AGriesser@anexia-it.com<mailto:AGriesser@anexia-it.com>
Web: http://www.anexia-it.com

Anschrift Hauptsitz Klagenfurt: Feldkirchnerstraße 140, 9020 Klagenfurt
Geschäftsführer: Alexander Windbichler
Firmenbuch: FN 289918a | Gerichtsstand: Klagenfurt | UID-Nummer: AT U63216601

_______________________________________________
Toasters mailing list
Toasters@teaparty.net<mailto:Toasters@teaparty.net>
http://www.teaparty.net/mailman/listinfo/toasters