;)
Field Notice: FN - 63916 AireOS 8.0.100.0 or Cisco IOS-XE 3.6.0E - AP Unable to Join WLC or AP Stuck in Downloading State - Software Update Required
Friday, January 16, 2015 at 2:30PM NOTICE:
THIS FIELD NOTICE IS PROVIDED ON AN "AS IS" BASIS AND DOES NOT IMPLY ANY KIND OF GUARANTEE OR WARRANTY, INCLUDING THE WARRANTY OF MERCHANTABILITY. YOUR USE OF THE INFORMATION ON THE FIELD NOTICE OR MATERIALS LINKED FROM THE FIELD NOTICE IS AT YOUR OWN RISK. CISCO RESERVES THE RIGHT TO CHANGE OR UPDATE THIS FIELD NOTICE AT ANY TIME.
Revision History
| Revision | Date | Comment |
|---|---|---|
|
1.0
|
12-JAN-2015
|
Initial Public Release
|
Products Affected
| Products Affected |
|---|
|
Cisco Aironet 1530 Series
|
|
Cisco Aironet 1550 Series
|
|
Cisco Aironet 1600 Series
|
|
Cisco Aironet 1700 Series
|
|
Cisco Aironet 2600 Series
|
|
Cisco Aironet 2700 Series
|
|
Cisco Aironet 3500 Series
|
|
Cisco Aironet 3600 Series
|
|
Cisco Aironet 3700 Series
|
Problem Description
Some Wireless Access Points (APs) manufactured between August 2014 and October 2014 might have an incorrectly programmed SHA-2 certificate.
The affected product families are:
- Cisco Aironet 1530 Series
- Cisco Aironet 1550 Series
- Cisco Aironet 1600 Series
- Cisco Aironet 1700 Series
- Cisco Aironet 2600 Series
- Cisco Aironet 2700 Series
- Cisco Aironet 3500 Series
- Cisco Aironet 3600 Series
- Cisco Aironet 3700 Series
Issue 1
After you upgrade a Wireless LAN Controller (WLC) to software version 8.0.100.0 or 3.6.0E
AND
after the Wireless APs download the new software version, any Wireless AP with an incorrectly programmed SHA-2 certificate disconnects from the WLC and is not able to rejoin the WLC if the WLC has a SHA-2 certificate.
Issue 2
Any new Wireless AP with software version 8.0.100.0 and with an incorrectly programmed SHA-2 certificate fails to validate the image downloaded from the WLC. The result is that the AP is unable to establish a connection to a WLC with version 8.0.100.0 software.
If the AP has an incorrectly programmed SHA-2 certificate and the WLC has version 8.0.100.0 or 3.6.0E, the likelihood of this issue being observed is 100%.
Background
Between August and October 2014, a manufacturing change was added to support SHA-2 certificates. In the certificate chain transition, some APs were manufactured with incorrect certificate information. Prior to this change, the APs only had a SHA-1 device ID certificate. After the change the APs had both SHA-1 and SHA-2, but the SHA-2 was incorrectly programmed on the affected units.
The available fixed code ensures that the APs continue to function as APs that were manufactured prior to August 2014.
The affected APs are fully functional and equivalent to APs manufactured prior to August 2014.
In the future, Cisco will provide support for SHA-2 authentication between APs and more recently manufactured WLCs.
Problem Symptoms
New Aironet APs with factory installed recovery Cisco IOS® are able to join the controller that runs software version 8.0.100.0 or 3.6.0E and download version 15.3(3)JA or 15.3(3)JN IOS. However after the AP reload, the APs are unable to join the controller. On the AP, logs similar to these are seen:
*Oct 16 12:39:06.231: AP has SHA2 MIC certificate - Using SHA2 MIC certificate for DTLS.
*Oct 16 13:14:56.000: %CAPWAP-5-DTLSREQSEND: DTLS connection request sent peer_ip: ***.***.***.*** peer_port: 5246Peer certificate verification failed FFFFFFFF
*Oct 16 13:14:56.127: DTLS_CLIENT_ERROR: ../capwap/base_capwap/capwap/base_capwap_wtp_dtls.c:496 Certificate verified failed!
*Oct 16 13:14:56.127: %DTLS-5-SEND_ALERT: Send FATAL : Bad certificate Alert to ***.***.***.***:5246
*Oct 16 13:14:56.127: %DTLS-5-SEND_ALERT: Send FATAL : Close notify Alert to ***.***.***.***:5246
Another symptom of this issue is that the AP might be able to join the software version 8.0.100.0 controller, download a new Cisco IOS code, and boot up and join the controller correctly; however when it goes to upgrade to the newer 8.x code it gets stuck in a loop and fails the download.
*Nov 11 10:13:53.003: Currently running a Release Image
*Nov 11 10:13:53.027: Using SHA-2 signed certificate for image signing validation.
*Nov 11 10:13:53.091: Image signing certificate validation failed (FFFFFFFF).
*Nov 11 10:13:53.091: Failed to validate signature
*Nov 11 10:13:53.091: Digital Signature Failed Validation (flash:/update/ap3g2-k9w8-mx.v153_80mr.201410311616/final_hash)
*Nov 11 10:13:53.091: AP image integrity check FAILED Aborting Image Download
Download image failed, notify controller!!! From:8.0.100.0 to 8.0.102.34, FailureCode:3 archive download: takes 339 seconds
*Nov 11 10:14:02.399: capwap_image_proc: problem extracting tar file
Workaround/Solution
AireOS
In order to avoid this issue, if the WLC runs software version 7.6 or earlier and you have APs affected by this issue, do not upgrade to version 8.0.100.x train. Wait for the next Cisco Connection Online (CCO) release.
Workaround for AireOS
If the WLC has been upgraded to version 8.0.100.x and the APs are supported in AireOS 7.6, downgrade to this version.
Solution for AireOS
If the WLC has software version 7.6 or earlier, upgrade the WLC to version 8.0.110.0.
If the WLC has software version 8.0.100.x, follow these steps:
- Upgrade the WLC to software version 8.0.104.0:
- Allow all APs to join the WLC and upgrade to software version 8.0.104.0.
- Upgrade the WLC to software version 8.0.110.0.
Note: Step 2 is required to push the 8.0.104.0 special software version onto the APs in order to allow all future upgrades.
Cisco IOS-XE
In order to avoid this issue, if the WLC has software version 3.3.x or earlier and you have APs affected by this issue, do not upgrade to version 3.6.0E.
Workaround for Cisco IOS-XE
If the WLC has been upgraded to version 3.6.0E and APs are supported in Cisco IOS-XE Version 3.3.x, downgrade to this version.
Solution for Cisco IOS-XE
If the WLC has software version 3.6.0E, follow these steps:
- Upgrade to version 3.6.1 or 3.7.0 or later.
- Enter the wireless security certificate force-sha1-cert command from the prompt.
CDETS
To follow the bug ID link below and see detailed bug information, you must be a registered customer and you must be logged in.
| CDETS | Description |
|---|---|
|
CSCur43050 (registered customers only)
|
APs mfg in Aug./Sept./Oct. 2014 unable to join an AireOS controller
|
|
CSCur50946 (registered customers only)
|
APs mfg in Aug./Sept./Oct. 2014 unable to join an IOS-XE controller
|
How To Identify Hardware Levels
From the AP CLI, enter the show version command and look for the "Top Assembly Serial Number". An example of a Top Assembly Serial Number is FTX1613GJGA.
If the AP is joined to an AireOS controller:
- From the CLI, enter the show ap inventory APNAME command.
- From the GUI, select Wireless > All APs > APNAME > Inventory in order to view the serial number.
If the AP is joined to a Cisco IOS-XE controller:
- From the controller CLI , enter the show ap name APNAME inventory command and look for the "Cisco AP" serial number.
- From the GUI, select Configuration > Wireless > Access Points > All APs > APNAME > Inventory in order to view the serial number.
Alternately, the serial number can be found on the back/bottom of the AP:
Confirm that your serial number is affected with the Serial Number Validation Tool.
For More Information
If you require further assistance, or if you have any further questions regarding this field notice, please contact the Cisco Systems Technical Assistance Center (TAC) by one of the following methods:
Reader Comments