Saturday, April 16, 2011

Uploading Diagnostic information to Vmware

FTP the bundles to Vmware

http://kb.vmware.com/selfservice/microsites/search.do?cmd=displayKC&docType=kc&externalId=1008525&sliceId=1&docTypeID=DT_KB_1_1&dialogID=175002379&stateId=0%200%20175000597

Exporting VCenter Logs
Exporting ESX and ESXi logs
Creating VMware Support Log File Bundles

Collecting Diagnostic information for Vmware products

Below links gives you information about collecting Diagnostic information of Various products in Vmware.

http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1008524

Vcenter and ESX and ESXi logfiles

Vcenter Log files :
-------------------
Windows 2008 C:\programdata\vmware\vmware Virtualcenter\Logs
Windows 2003 C:\Documents and settings\All Users\Application Data\Vmware\VirtualCenter\logs
Size,location,name and rotation can be manipulated with vpxd.cfg (5MB)file.

Vmware KB Article :
1021804

ESX log Files:-
---------------
/var/log/vmkernel  ---Vkernel messages
/var/log/vmkwarning
/var/log/vmksummary
/var/log/vmware/hostd.log--ESX service
/var/log/messages ---Service Console
/var/log/vmware/vpx/vpxa.log---vSphere Client Agent
/var/log/vmware/aam/vmware_<hstname>-xxx.log---HA
/var/log/vmkiscsid.og---iscsi
/var/log/boot-logs/sysboot.log ---boot log

Vmware KB Article:
1021800

ESXi log files :-
-----------------
/var/log/vmware/hostd.log--ESXi service
/var/log/messages ---Combinaton of Vkernel messages ,Hostd,Vmkwarning
/var/log/vmware/vpx/vpxa.log---vSphere Client Agent
/var/log/vmware/aam/vmware_<hstname>-xxx.log---HA
/var/log/boot-logs/sysboot.log ---boot log
ESXi logs vanish when reboot or poweroff.
Vmware KB Article:
1021801

Configuring Log rotation for the log files :
--------------------------------------------
/etc/logrotate.conf
hostd.log
/etc/vmware/hostd/config.xml
vpxa.log
/etc/opt/vmware/vpxa/vpxa.cfg


Acessing the logs:
-------------------
Vsphre clinet
SSH
DCUI with ESXi service Console

Tuesday, April 12, 2011

Troubleshooting Vmware HA

Common VMHA problems :

1)Misconfigured admission control
2)Lack of management network redendancy

VMHA agents monitor other hosts in the cluster with heartbeats sent on the management network
ESX it is vswif and ESXi it is vmk.After 15 secs if no heart is recived host is failed vms are failed over.

3)Misunderstood host roles (primary/secondary)
4)HA agent not loaded

VMHA Admission Control polices:
1)Amonut of host failures
2)Percentage reserved
3)Designated failover host (Reserving the Host for failure)

Saturday, April 9, 2011

Troubleshooting NFS

Adding an NFS volume from ESX:
esxcfg-nas -a <datastore name> -o <nfs servername or ip> -s <mount point>
esxcfg-nas -l   list NFS volumes

Adding an NFS volume from VCLI:
vicfg-nas.pl --server <esx hostname> --add nfsshare --nasserver <ip/dns name> --share <mount point>

NFS logs :
ESX --->\var\log\vmkernel
ESXi--->\var\log\messages
cat \var\log\vmkernel | grep nfs

Troubleshooting steps for NFS :
1) Refresh the storage
2)Check VMkernel is configured correclty
3)Verify firewall ports UDP/TCP 111 & 2049 are open
4)Verify NFS client is open in Security profile/firewall
5) Test connectivity -vmkping from esx to nas server
6) Verify share & permissions on NAS server
7)Check the messages or vmkernel log file

References :
Vmware KB 1003967

Troubleshooting Vcenter issues

Vcenter Service VPXD  is the main services  that should run for the Vcenter to function properly.

Location of config files and logs in Win2008:
C:\programdata\Vmware\VmwareVirtualCenter\vpxd.cfg
C:\programdata\Vmware\VmwareVirtualCenter\logs

1) Make sure VPXD services in Vcenter is started.
2) Look for the credentials used for VPXD are  correct
3)Verify the Hostname or IP of the Database changed
4)Vcenter Database name changed
5)Username and Password changed for database
6)DSN changed (Data Source Name)
7)Verify type of DSN (32 or 64 bit)
8)Check the port 80,443,906 on Vcenter if they are used by some other services
9)Out of space on SQL servers or DB

Referrences :
Refer Vmware KB articles 1003979 & 1003928 & 1003926

Notes :-
netstat -an | more to check all listening ports
TCPVIEW from MS sysinternals to view all the ports with names.

Wednesday, April 6, 2011

ESX & ESXi Performance

Absolute best tool for ESX/ESXi Performance .Best way to see the performance of 
VM on a Host is using .This gives you real-time resource utilization statistics

esxtop for ESX
resxtop for ESXi  using VMA

Q- Quit
C- Switch to CPU Utlization Screen
1)look for CPU Ready value this is the amount of time it waits for Disk  or Network.
2)The %CSTP value represents The %CSTP value represents the amount of time a virtual machine with multiple virtual CPUs is waiting to be scheduled on multiple cores on the physical host. The higher the value, the longer it waits and the worse its performance.  Lowering the number of vCPUs reduces the scheduling wait time. the amount of time a virtual machine with multiple virtual CPUs is waiting to be scheduled on multiple cores on the physical host. The higher the value, the longer it waits and the worse its performance.  Lowering the number of vCPUs reduces the scheduling wait time.

M- Switch to memory Utilization Screen
D- Switch to Stroage (disk) adapter resource Utilization Screen
U- Switch to Stroage (disk) device resource Utilization Screen
V-Switch to Stroage (disk) VM resource Utilization Screen
N-Switch to Network resource Utilization Screen
I-Switch to interrupt Utilization Screen
P-Switch to Power Utilization Screen

These options apply both for ESX and ESXi
http://communities.vmware.com/docs/DOC-9279