Dead Node Diagnostics (7200c)

[duck_of_death]

Hi everyone,

I've had 2 7200c nodes die on me lately, one didn't come back after a move to a new server rack and the replacement for that node failed during a power cycle. When power is applied the blue/amber/green lights on the rear of the node are displayed but within a few seconds I get the blinking blue and amber lights.

I'm very interested in what actually happened to these nodes as it seems like they died very suddenly and the failure seems to prevent the FPGA (Cyclone III) from starting up.

I've probed a few pins on the J2 connector which is located beside the FPGA but other than a 12.5 MHz clock I can't seem to find any sort of serial console. I also see the OKI ARM microcontoller (MCU) on the board but I can't seem to find a serial console for it either, I see some MAX3221e ICs near the wider "MFG" ports (not the board RJ45 jack console) and am wondering if those ports contain the serial console.

I've also looked through the nemoe_fw.bin-4.8.29 file, which I think in the firmware for the OKI MCU. There are some references in it to different serial ports as well as the power up order for the FPGA and Intel CPU. My guess is that this MCU is detecting some fault condition and not powering on the FPGA.

I saw an older post on here regarding the 5 green "POR STATE" LEDs but there wasn't any useful information as to what those LEDs indicate. I'm guessing POR is Power on Reset status for the FPGA but that's just a guess.

Does anyone have information they could share about the meaning of these LEDs? Or, if someone has repaired nodes in the past (HPE techs?) could they share some tips about reviving a dead node or connecting to the serial console?

Thanks!

John


Edit: It just occurred to me that there might be multiple serial consoles on the single RJ45 port... hmm

This was one of the strings in the firmware file:

Serial-port S1 is for engineering use. Use Serial-port S0 for 'Console Menu' access.

[duck_of_death]

Good news!

I'm fairly sure that J2 is a ByteBlaster II-compatible pin-out for the Cyclone III FPGA.



I recall the GND on pins 2 and 10 and I saw the clock on pin 1.