Hi!
I've upgraded our VCSA appliance from 5.5U2 to 6 and everything seemed to went fine but after it ran about 12 hours, the appliance began to eat up the two vCPUs almost completely, rendering the web client as well as any SOAP interfaces pretty useless:
Diagnosing the cause is a little bit difficult for me because even the SSH daemon refuses to accept any connections during those high load periods.
Sometimes I've managed it to get a VIMtop output which was basically telling me that the "CIM health service" is taking up more than 100% cpu time. After this, I've noticed that the health stats for my HP ProLiant G5 (still using an 5.5U2/HP image) were missing when using the web client, although they are still accessible using WBEM directly on the host. Then I've simply deactivated the CIM server on that specific host ... but it didn't help.
When look at VCSA's dmesg output, there are dozens of messages like this one:
IPfilter Dropped: IN=eth OUT= MAC:ff:ff:ff:ff:ff:ff:ff:ff (...) SRC=(varies) DST=255.255.255.255 LEN=68 TOS=0x00 PREC=0x00 TTL=128 ID=26090 PROTO=UDP SPT=58980 DPT=1947 LEN=48
Is this somehow related?
So my question is if anyone has an idea how to troubleshoot this issue in order to track down the root cause?