-
Notifications
You must be signed in to change notification settings - Fork 3.3k
Description
Problem
Windows 2012 reboots are not detected properly on packer 1.1.2 with vmware-workstation (linux).
Windows 2012 reboots are detected properly on packer 1.1.2 with vmware-fusion(OsX).
packer 1.0.4 and 1.1.1 work properly on both platforms.
Failures
packer | vmware | os |
---|---|---|
1.1.2 | workstation 12.5.4 | ubuntu 16.04 |
I have reproduced this over 12 times with 100% consistency. I'm using the same packer definition that has been working for months with 1.0.4
Packer does not fail if using 1.0.4/1.1.1 OR if running 1.1.2 with OSX with vmware-fusion
Successes
packer | vmware | os |
---|---|---|
1.0.4 | workstation 12.5.4 | ubuntu 16.04 |
1.1.1 | workstation 12.5.4 | ubuntu 16.04 |
1.0.4 | vmware-fusion 8 | Mac OSX |
1.1.2 | vmware-fusion 8 | Mac OSX |
Symptom
This is easy, but very time consuming to reproduce, because it only happens after windows 2012 has completed installing windows updates, and then packer attempts to reboot the machine.
{
"script": "scripts/windows/common/stop-windows-updates.ps1",
"type": "powershell"
},
{
"script": "scripts/windows/common/install-chocolatey.ps1",
"type": "powershell"
},
{
"type": "powershell",
"inline": ["choco install pswindowsupdate --verbose -y"]
},
{
"script": "scripts/windows/common/windows-updates.ps1",
"type": "powershell"
},
{
"type": "windows-restart",
"pause_before": "1m",
"restart_timeout": "1h"
},
Watching the build process, windows always gets stuck shutting down after installing windows updates, and shows error. (Sorry for the picture, the ubuntu build machine doesn't have VNC).
After 5 to 30 minutes, the windows error always clears itself and windows reboots successfully. However packer 1.1.2 never detects that the vm has successfully rebooted, even with a 1 hour timeout.
vmware-iso: A system shutdown is in progress.(1115)
2017/11/21 14:22:57 ui: vmware-iso: A system shutdown is in progress.(1115)
2017/11/21 14:22:57 packer-1.1.2: 2017/11/21 14:22:57 Reboot already in progress, waiting...
2017/11/21 14:23:07 packer-1.1.2: 2017/11/21 14:23:07 Check if machine is rebooting...
2017/11/21 14:23:07 packer-1.1.2: 2017/11/21 14:23:07 [INFO] starting remote command: shutdown.exe -f -r -t 60
2017/11/21 14:23:07 [INFO] 0 bytes written for 'stdout'
2017/11/21 14:23:07 [INFO] 40 bytes written for 'stderr'
2017/11/21 14:23:07 packer-1.1.2: 2017/11/21 14:23:07 [INFO] command 'shutdown.exe -f -r -t 60' exited with code: 1115
2017/11/21 14:23:07 packer-1.1.2: 2017/11/21 14:23:07 [INFO] RPC endpoint: Communicator ended with: 1115
2017/11/21 14:23:07 [INFO] RPC client: Communicator ended with: 1115
2017/11/21 14:23:07 [INFO] RPC endpoint: Communicator ended with: 1115
2017/11/21 14:23:07 ui: vmware-iso: A system shutdown is in progress.(1115)
2017/11/21 14:23:07 packer-1.1.2: 2017/11/21 14:23:07 [INFO] 0 bytes written for 'stdout'
2017/11/21 14:23:07 packer-1.1.2: 2017/11/21 14:23:07 [INFO] 40 bytes written for 'stderr'
2017/11/21 14:23:07 packer-1.1.2: 2017/11/21 14:23:07 [INFO] RPC client: Communicator ended with: 1115
vmware-iso: A system shutdown is in progress.(1115)
2017/11/21 14:23:07 packer-1.1.2: 2017/11/21 14:23:07 Reboot already in progress, waiting...
2017/11/21 14:23:17 packer-1.1.2: 2017/11/21 14:23:17 Check if machine is rebooting...
2017/11/21 14:23:27 [INFO] 0 bytes written for 'stdout'
2017/11/21 14:23:27 [INFO] 0 bytes written for 'stderr'
2017/11/21 14:23:27 packer-1.1.2: 2017/11/21 14:23:27 [INFO] 0 bytes written for 'stderr'
2017/11/21 14:23:27 packer-1.1.2: 2017/11/21 14:23:27 Waiting for machine to reboot with timeout: 5m0s
2017/11/21 14:23:27 packer-1.1.2: 2017/11/21 14:23:27 Waiting for machine to become available...
2017/11/21 14:23:27 packer-1.1.2: 2017/11/21 14:23:27 Checking that communicator is connected with: 'powershell.exe -EncodedCommand ZQBjAGgAbwAgACIAJAB7AGUAbgB2ADoAQwBPAE0AUABVAFQARQBSAE4AQQBNAEUAfQAgAHIAZQBzAHQAYQByAHQAZQBkAC4AIgA='
2017/11/21 14:23:27 packer-1.1.2: 2017/11/21 14:23:27 [INFO] 0 bytes written for 'stdout'
2017/11/21 14:24:02 [INFO] 0 bytes written for 'stdout'
2017/11/21 14:24:02 [INFO] 0 bytes written for 'stderr'
2017/11/21 14:24:02 packer-1.1.2: 2017/11/21 14:24:02 [INFO] 0 bytes written for 'stdout'
2017/11/21 14:24:02 packer-1.1.2: 2017/11/21 14:24:02 [INFO] 0 bytes written for 'stderr'
==> vmware-iso: Timeout waiting for machine to restart.
2017/11/21 14:27:46 ui error: ==> vmware-iso: Timeout waiting for machine to restart.
2017/11/21 14:27:46 [INFO] (telemetry) ending windows-restart
2017/11/21 14:27:46 [INFO] (telemetry) found error: Timeout waiting for machine to restart.
2017/11/21 14:27:46 packer-1.1.2: 2017/11/21 14:27:46 Executing: /usr/bin/vmrun -T ws list
2017/11/21 14:27:46 packer-1.1.2: 2017/11/21 14:27:46 stdout: Total running VMs: 1
2017/11/21 14:27:46 packer-1.1.2: /root/golden-images-vagrant/output-win2012r2-vmware/packer-vmware-iso.vmx
2017/11/21 14:27:46 packer-1.1.2: 2017/11/21 14:27:46 stderr:
2017/11/21 14:27:46 ui: ==> vmware-iso: Stopping virtual machine...
2017/11/21 14:27:46 packer-1.1.2: 2017/11/21 14:27:46 Executing: /usr/bin/vmrun -T ws stop output-win2012r2-vmware/packer-vmware-iso.vmx hard
==> vmware-iso: Stopping virtual machine...
2017/11/21 14:27:48 packer-1.1.2: 2017/11/21 14:27:48 stdout:
2017/11/21 14:27:48 packer-1.1.2: 2017/11/21 14:27:48 stderr:
2017/11/21 14:27:48 packer-1.1.2: 2017/11/21 14:27:48 Deleting floppy disk: /tmp/packer990518872
==> vmware-iso: Deleting output directory...
2017/11/21 14:27:48 ui: ==> vmware-iso: Deleting output directory...
2017/11/21 14:27:49 packer-1.1.2: 2017/11/21 14:27:49 Error removing output dir: remove output-win2012r2-vmware: directory not empty
2017/11/21 14:27:51 [INFO] (telemetry) ending vmware-iso
2017/11/21 14:27:51 [INFO] (telemetry) found error: Timeout waiting for machine to restart.
2017/11/21 14:27:51 ui error: Build 'vmware-iso' errored: Timeout waiting for machine to restart.
2017/11/21 14:27:51 Builds completed. Waiting on interrupt barrier...
2017/11/21 14:27:51 machine readable: error-count []string{"1"}
2017/11/21 14:27:51 ui error:
==> Some builds didn't complete successfully and had errors:
2017/11/21 14:27:51 machine readable: vmware-iso,error []string{"Timeout waiting for machine to restart."}
2017/11/21 14:27:51 ui error: --> vmware-iso: Timeout waiting for machine to restart.
2017/11/21 14:27:51 ui:
==> Builds finished but no artifacts were created.
2017/11/21 14:27:51 [INFO] (telemetry) Finalizing.
Build 'vmware-iso' errored: Timeout waiting for machine to restart.
==> Some builds didn't complete successfully and had errors:
--> vmware-iso: Timeout waiting for machine to restart.
==> Builds finished but no artifacts were created.
2017/11/21 14:27:51 waiting for all plugin processes to complete...
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 [ERR] Error decoding response stream 39: EOF
2017/11/21 14:27:51 [ERR] Error decoding response stream 36: EOF
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
2017/11/21 14:27:51 /opt/packer/packer-1.1.2: plugin process exited
Installed 1.15.7 of the vmware VIX api library
vmware-installer -t
Component Name Component Long Name Component Version
============================== ================================================= ====================
vmware-installer VMware Installer 2.1.0.5087745
vmware-player-setup VMware Player Setup 12.5.4.5192485
vmware-vmx VMware VMX 12.5.4.5192485
vmware-vix-core VMware VIX Core Library 1.15.7.5192485
vmware-network-editor VMware Network Editor 12.5.4.5192485
vmware-network-editor-ui VMware Network Editor User Interface 12.5.4.5192485
vmware-tools-netware VMware Tools for NetWare 10.1.5.5192485
vmware-tools-linuxPreGlibc25 VMware Tools for legacy Linux 10.1.5.5192485
vmware-tools-winPreVista VMware Tools for Windows 2000, XP and Server 2003 10.1.5.5192485
vmware-tools-winPre2k VMware Tools for Windows 95, 95, Me and NT 10.1.5.5192485
vmware-tools-freebsd VMware Tools for FreeBSD 10.1.5.5192485
vmware-tools-windows VMware Tools for Windows Vista or later 10.1.5.5192485
vmware-tools-solaris VMware Tools for Solaris 10.1.5.5192485
vmware-tools-linux VMware Tools for Linux 10.1.5.5192485
vmware-usbarbitrator VMware USB Arbitrator 15.2.0.5192485
vmware-player-app VMware Player Application 12.5.4.5192485
vmware-workstation-server VMware Workstation Server 12.5.4.5192485
vmware-ovftool VMware OVF Tool component for Linux 4.1.0.3634792
vmware-vprobe VMware VProbes component for Linux 12.5.4.5192485
vmware-vix-lib-Workstation1200 VMware VIX Workstation-12.0.0 Library 1.15.7.5192485
vmware-workstation VMware Workstation 12.5.4.5192485
Workaround
I've had to roll back to 1.0.4 or 1.1.1 to get windows 2012 images working again.
Theories
These are all the milestones from 1.1.2
1.1.2
improved scripts after restarts in #5272. Perhaps the check command is too specific- make restart command work correctly even if user has their own check … #5563 looks suspiciously related
What other information can I provide to help debug this? I want packer 1.1.2 to work with vmware workstation on linux.