So lets say that you have an ILM instance and the load balanced ADAM servers have a cert that expired..1am is not the best time to be creating a playbook! A few engineers took swipes at it and ended up with some kind of corrupt private/public key pair situation, along with multiple certificates matching the same subject name. We ended up taking notes of what was done, backing it out, and walking through it slowly with Microsoft Support's blessing.
To replace an ADAM certificate that already expired..
1) Locate, document, and remove existing certificate (hint, check service account's cert store..)
2) Create new web server template based certificate request (hint, check SAN attribute)
3) Create response from CA
4) Import response from requesting computer and export the resulting .pfx with public & private key.
5) Import public+private key to ADAM server
6) Grant rights to the account ADAM is running as for the private key of the cert via certificates MMC ideally. In our case we found the old cert in a service account store, but it was running as Network Service. Placing the new cert in the local machine store + granting Network Service access to the private key worked in this case. Others may have a functional/used domain service account & related key store.
7) ADAM should resume accepting connections on 636. Events in the ADAM log should confirm.
Note: If using SAN attribute on cert, be sure subject name is matched. If the subject name changes some apps (Weblogic, lookin' at you!) will not honor the SAN attribute and reject the connection. Thankfully you can disable host name validation if needed, so it seems.
Monday, June 17, 2013
NBSTAT, Printers, and Go Away We're Not Home
New ISP, new issues. While investigating why my connection goes brain dead every so often I saw some odd hits in the AT&T UVerse 2wire gateway's logs:
src=162.1.2.3 dst=192.168.0.108 ipprot=6 sport=61913 dport=445 Drop traffic to 192.168.0.0/16
The src IP is the WAN IP of my router (2wire is bridging, sort of.) and the destination is an unused private network range. Odd, I thought, that my Linksys was trying to connect to private ranges, and even odder that it was heading out the WAN interface towards the internet.
Firing up Wireshark I immediately saw a flood of packets to AND FROM that private range. I have no routes to that range, nothing in the arp tables, and from all I can tell that network just doesn't exist.
The connections were coming from the SYSTEM process as well, so not much more digging to be easily done there..
Enter: Reset cause: Go away, we're not home
My PC was making a query to two addresses in this private range and getting responses..rather troubling responses at that. The hardware address of the responses belonged to my Linksys router too.
38817 2315.479819 my.workstation.IP 192.168.0.100 TCP 66 61934 > microsoft-ds [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=4 SACK_PERM=1
38818 2315.479819 my.workstation.IP 192.168.0.100 TCP 66 61935 > netbios-ssn [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=4 SACK_PERM=1
38819 2315.482288 192.168.0.100 my.workstation.ip TCP 77 [TCP Retransmission] netbios-ssn > 61935 [RST, ACK] Seq=1 Ack=1 Win=0 Len=23
38820 2315.482291 192.168.0.100 my.workstation.ip TCP 77 [TCP Retransmission] microsoft-ds > 61934 [RST, ACK] Seq=1 Ack=1 Win=0 Len=23
The payload in the responses contained: "Go away, we're not home". A bit of googling led me to many hits related to the STORM virus. While that's very dated I still ran all of the standard scans, a few manual checks, investigation of a few drivers..Nothing found at all. That IP range in question was somewhat familiar, as it was my old private network before moving and switching things around.
Nearing defeat, I started a registry search for that IP address. Bingo. A pair of mapped printers with static IPs from my old network configuration. Removing them & restarting spooler cleared up all of those odd connections.
The root of that cryptic response seems to be how the AT&T UVerse 2Wire gateway responds to traffic it drops. The firewall on that device was dropping traffic to that subnet, and Go away, we're not home was how it was responding. If I had dropped a sniffer between that gateway and the Linksys I probably would have found it more quickly, but live and learn..
src=162.1.2.3 dst=192.168.0.108 ipprot=6 sport=61913 dport=445 Drop traffic to 192.168.0.0/16
The src IP is the WAN IP of my router (2wire is bridging, sort of.) and the destination is an unused private network range. Odd, I thought, that my Linksys was trying to connect to private ranges, and even odder that it was heading out the WAN interface towards the internet.
Firing up Wireshark I immediately saw a flood of packets to AND FROM that private range. I have no routes to that range, nothing in the arp tables, and from all I can tell that network just doesn't exist.
The connections were coming from the SYSTEM process as well, so not much more digging to be easily done there..
Enter: Reset cause: Go away, we're not home
My PC was making a query to two addresses in this private range and getting responses..rather troubling responses at that. The hardware address of the responses belonged to my Linksys router too.
38817 2315.479819 my.workstation.IP 192.168.0.100 TCP 66 61934 > microsoft-ds [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=4 SACK_PERM=1
38818 2315.479819 my.workstation.IP 192.168.0.100 TCP 66 61935 > netbios-ssn [SYN] Seq=0 Win=8192 Len=0 MSS=1460 WS=4 SACK_PERM=1
38819 2315.482288 192.168.0.100 my.workstation.ip TCP 77 [TCP Retransmission] netbios-ssn > 61935 [RST, ACK] Seq=1 Ack=1 Win=0 Len=23
38820 2315.482291 192.168.0.100 my.workstation.ip TCP 77 [TCP Retransmission] microsoft-ds > 61934 [RST, ACK] Seq=1 Ack=1 Win=0 Len=23
The payload in the responses contained: "Go away, we're not home". A bit of googling led me to many hits related to the STORM virus. While that's very dated I still ran all of the standard scans, a few manual checks, investigation of a few drivers..Nothing found at all. That IP range in question was somewhat familiar, as it was my old private network before moving and switching things around.
Nearing defeat, I started a registry search for that IP address. Bingo. A pair of mapped printers with static IPs from my old network configuration. Removing them & restarting spooler cleared up all of those odd connections.
The root of that cryptic response seems to be how the AT&T UVerse 2Wire gateway responds to traffic it drops. The firewall on that device was dropping traffic to that subnet, and Go away, we're not home was how it was responding. If I had dropped a sniffer between that gateway and the Linksys I probably would have found it more quickly, but live and learn..
Subscribe to:
Posts (Atom)