cygwin python is no longer required under Windows to run the expect scripts.

Numerous cleanups and bug fixes to the expect scripts.

The following bugs have been fixed:

http://bugzilla.zeroc.com/bugzilla/show_bug.cgi?id=3091 - under windows a timeout does not kill the spawned servers.
http://bugzilla.zeroc.com/bugzilla/show_bug.cgi?id=3167 - cleanup IceBox expect scripts.
http://bugzilla.zeroc.com/bugzilla/show_bug.cgi?id=3131 - demoscript environment setup.
http://bugzilla.zeroc.com/bugzilla/show_bug.cgi?id=3130 - config/DemoUtil.py should be in demoscript.

Squashed commit of the following:

commit c82e5b70dab99b69caf4044341f6453e0a2b8192
Author: U-MARCH4\matthew <matthew@march4.(none)>
Date:   Wed Jun 11 14:42:34 2008 +0800

    fix bug with multicast demo script.

commit c6e61dddf2fc73088e5ecacd096c01fc30c5477a
Author: Matthew Newhook <matthew@zeroc.com>
Date:   Wed Jun 11 14:14:52 2008 +0800

    guess the build mode if not set on the command line under Windows.

commit 6e797cdca50d6a30493f56e978da4b6f8a08e70d
Author: Matthew Newhook <matthew@zeroc.com>
Date:   Tue Jun 10 17:16:49 2008 +0800

    use dirname, not split.

commit 862fb56fac680d42037d251c54938ed294596690
Author: Matthew Newhook <matthew@zeroc.com>
Date:   Tue Jun 10 17:07:24 2008 +0800

    simplify environment setup.

commit d647b35588019ab841c5fe076950e7c19dcbf22c
Author: Matthew Newhook <matthew@zeroc.com>
Date:   Tue Jun 10 16:38:40 2008 +0800

    use iceHome not sourcedist in the environment setup.

commit 18a82ad794517406f80add4071f46848220bae27
Author: U-MARCH4\matthew <matthew@march4.(none)>
Date:   Tue Jun 10 16:00:49 2008 +0800

    fix another hasInterruptSupport problem.

commit 18b276f2c5dcf8a2c38c79e7dc8e8b5f20ce884e
Author: Matthew Newhook <matthew@centosvm4.matthew.zeroc.com>
Date:   Tue Jun 10 15:28:08 2008 +0800

    Fix printing of the environment.

commit 42d5f59dbd2db1811abdd759387e53081858edb0
Author: Matthew Newhook <matthew@centosvm4.matthew.zeroc.com>
Date:   Tue Jun 10 14:12:19 2008 +0800

    remove -u from iceca. Put another workaround in the makecerts script.

commit 1bfcc656f25ec3f4f49b5c534bd1c50d50801a4c
Author: Matthew Newhook <matthew@zeroc.com>
Date:   Mon Jun 9 12:19:31 2008 +0800

    New method to find the top level directory.

commit 1c771768e29b7ff9d141b39f8e03b6790564b4ea
Author: Matthew Newhook <matthew@zeroc.com>
Date:   Mon Jun 9 11:28:25 2008 +0800

    Print environment variables in a more sane manner. Don't add cs/bin to the PATH & java/lib to CLASSPATH if not using a source dist.

commit a477ceac74b04f297cb342bc7229e2a3a70695e1
Author: Matthew Newhook <matthew@zeroc.com>
Date:   Fri Jun 6 16:32:09 2008 +0800

    fix various problems when testing with the demo dist.

commit 9bdb41f02130a5716a52ea0ae161fd88697277e5
Author: Matthew Newhook <matthew@zeroc.com>
Date:   Fri Jun 6 13:35:41 2008 +0800

    Stop copying obsolete DemoUtil.py.

commit d1b4eabc643d0ea04374640ef3f3a2452aaa19bd
Author: U-MARCH4\matthew <matthew@march4.(none)>
Date:   Fri Jun 6 13:20:59 2008 +0800

    fix win32 problem.

commit f92d9bf033e253cb8e3ee8dbce1583c7a695ad9f
Author: Matthew Newhook <matthew@zeroc.com>
Date:   Fri Jun 6 13:08:27 2008 +0800

    more cleanups.

commit 8ecea1446b8a64afdf029f4e722d473c58149f84
Author: Matthew Newhook <matthew@zeroc.com>
Date:   Fri Jun 6 12:45:58 2008 +0800

    get rid of the processCmdLine.

commit a868ebcd7f868d7b9340f8df6d47ab87d43d37d3
Author: Matthew Newhook <matthew@zeroc.com>
Date:   Fri Jun 6 12:38:16 2008 +0800

    fix preamble.

commit 148f2922a794fa0ef696e8b4ba51749d4b15cb2c
Author: Matthew Newhook <matthew@zeroc.com>
Date:   Fri Jun 6 12:30:07 2008 +0800

    fixed some errors in the scripts.

commit 3b9fb9a6b755fe3d84864cdd6853b2a59588a089
Author: Matthew Newhook <matthew@centosvm4.matthew.zeroc.com>
Date:   Thu Jun 5 14:47:39 2008 +0800

    python 2.3 support.

commit 56956b0a56dd2de5753db060bdadfdb4ddb242a8
Author: Matthew Newhook <matthew@zeroc.com>
Date:   Thu Jun 5 14:05:51 2008 +0800

    change setenv to addenv. Use hasInterruptSupport in Except.

commit e2907ffd17bc59eaa9d5dd74fb74dbb7ac9addba
Author: Matthew Newhook <matthew@zeroc.com>
Date:   Thu Jun 5 13:02:24 2008 +0800

    minor cleanup and optimizations.

commit 97fbc27302ee1f74572b634e3b7d4efe6654540f
Author: Matthew Newhook <matthew@zeroc.com>
Date:   Thu Jun 5 12:52:57 2008 +0800

    fix top level allDemos.

commit 04630047daffafe524e3aed7833dd48e82b90bf2
Author: Matthew Newhook <matthew@zeroc.com>
Date:   Thu Jun 5 12:39:57 2008 +0800

    Align scripts with the testsuite.

commit 2909a30d682cb6f9cb192f21bf652c3b8037371d
Author: U-MARCH4\matthew <matthew@march4.(none)>
Date:   Thu Jun 5 10:55:13 2008 +0800

    Some Win32 fixes.

commit bc760c357a33a11f3f645169c291c69e1f16fb43
Author: Matthew Newhook <matthew@zeroc.com>
Date:   Wed Jun 4 18:03:12 2008 +0800

    lots of cleanups.

commit c69fbac2457b5a14cc8a48beae3381652f50f599
Author: U-MARCH4\matthew <matthew@march4.(none)>
Date:   Tue Jun 3 16:54:45 2008 +0800

    fix some python problems.

commit 326a95af7c720c0c740975c11251feb2f9b4762b
Author: Matthew Newhook <matthew@zeroc.com>
Date:   Tue Jun 3 16:16:13 2008 +0800

    use python -u for iceca.

commit 990c03c6d314c7d95137a69ebce03425e0eff56c
Author: Matthew Newhook <matthew@zeroc.com>
Date:   Tue Jun 3 14:47:00 2008 +0800

    first set of unix changes.

commit aa02eaec6c5d44d526b3236908a53c855754217c
Author: U-MARCH4\matthew <matthew@march4.(none)>
Date:   Tue Jun 3 14:46:51 2008 +0800

    Get rid of \r in the input stream. Fix scripts accordingly. Fix signal handling under windows.

commit 558015fb4131980f1f4b3fa23c3ddf366d186d20
Author: U-MARCH4\matthew <matthew@march4.(none)>
Date:   Mon Jun 2 16:38:21 2008 +0800

    fix signal handling.

commit cadfb7bcabb4c68317fc77c9076607dfb2201e2b
Author: U-MARCH4\matthew <matthew@march4.(none)>
Date:   Fri May 30 17:40:38 2008 +0800

    working on a fix for hanging.

commit 9b9ffa736d1540c8bfdaa3693df3022e6377cdec
Author: U-MARCH4\matthew <matthew@march4.(none)>
Date:   Thu May 29 17:53:11 2008 +0800

    Removed dependence on cygwin.

commit cce2d732036585fe8e34f92d6b75ae9c27465622
Author: U-MARCH4\matthew <matthew@march4.(none)>
Date:   Thu May 29 16:17:42 2008 +0800

    windows expect script changes.

1 files changed, 445 insertions, 0 deletions

diff --git a/demoscript/Expect.py b/demoscript/Expect.py
new file mode 100755
index 00000000000..a76c6d3e89b
--- /dev/null
+++ b/demoscript/Expect.py
@@ -0,0 +1,445 @@
+# **********************************************************************
+#
+# Copyright (c) 2003-2008 ZeroC, Inc. All rights reserved.
+#
+# This copy of Ice is licensed to you under the terms described in the
+# ICE_LICENSE file included in this distribution.
+#
+# **********************************************************************
+
+import threading
+import subprocess
+import StringIO
+import string
+import time
+import re
+import traceback
+import sys
+import os
+import signal
+import sys
+
+__all__ = ["Expect", "EOF", "TIMEOUT" ]
+
+win32 = (sys.platform == "win32")
+if win32:
+    # We use this to remove the reliance on win32api.
+    import ctypes
+
+class EOF:
+    """Raised when EOF is read from a child.
+    """
+    def __init__(self, value):
+        self.value = value
+    def __str__(self):
+        return str(self.value)
+
+class TIMEOUT:
+    """Raised when a read time exceeds the timeout.
+    """
+    def __init__(self, value):
+        self.value = value
+    def __str__(self):
+        return str(self.value)
+
+def escape(s):
+    if s == TIMEOUT:
+        return "<TIMEOUT>"
+    o = StringIO.StringIO()
+    for c in s:
+	if c == '\\':
+	    o.write('\\\\')
+	elif c == '\'':
+	    o.write("\\'")
+	elif c == '\"':
+	    o.write('\\"')
+	elif c == '\b':
+	    o.write('\\b')
+	elif c == '\f':
+	    o.write('\\f')
+	elif c == '\n':
+	    o.write('\\n')
+	elif c == '\r':
+	    o.write('\\r')
+	elif c == '\t':
+	    o.write('\\t')
+	else:
+	    if c in string.printable:
+		o.write(c)
+	    else:
+		o.write('\\%03o' % ord(c))
+    return o.getvalue()
+
+class reader(threading.Thread):
+    def __init__(self, desc, p, logfile):
+	self.desc = desc
+	self.buf = StringIO.StringIO()
+    	self.cv = threading.Condition()
+	self.p = p
+	self.logfile = logfile
+        threading.Thread.__init__(self)
+
+    def run(self):
+        try:
+	    while True:
+		c = self.p.stdout.read(1)
+		if not c: break
+                if c == '\r': continue
+
+		self.cv.acquire()
+		try:
+		    self.buf.write(c)
+		    self.cv.notify()
+		finally:
+		    self.cv.release()
+        except IOError, e:
+	    print e
+
+    def getbuf(self):
+	self.cv.acquire()
+	try:
+	    buf = self.buf.getvalue()
+	finally:
+	    self.cv.release()
+	return buf
+
+    def match(self, pattern, timeout, matchall = False):
+	"""pattern is a list of string, regexp duples.
+	"""
+
+	if timeout is not None:
+	    end = time.time() + timeout
+	start = time.time()
+
+	# Trace the match
+	if self.logfile:
+            if timeout is None:
+                tdesc = "<infinite>"
+            else:
+                tdesc = "%.2fs" % timeout
+            p = [ escape(s) for (s, r) in pattern ]
+            pdesc = StringIO.StringIO()
+            if len(p) == 1:
+                pdesc.write(escape(p[0]))
+            else:
+                pdesc.write('[');
+                for pat in p:
+                    if pat != p[0]:
+                        pdesc.write(',');
+                    pdesc.write(escape(pat))
+                pdesc.write(']');
+	    self.logfile.write('%s: expect: "%s" timeout: %s\n' % (self.desc, pdesc.getvalue(), tdesc))
+	    self.logfile.flush()
+
+        maxend = None
+    	self.cv.acquire()
+	try:
+	    try: # This second try/except block is necessary because of python 2.3
+		while True:
+		    buf = self.buf.getvalue()
+
+		    # Try to match on the current buffer.
+		    olen = len(pattern)
+		    for index, p in enumerate(pattern):
+			s, regexp = p
+			if s == TIMEOUT:
+			    continue
+			m = regexp.search(buf)
+			if m is not None:
+			    before = buf[:m.start()]
+			    matched = buf[m.start():m.end()]
+			    after = buf[m.end():]
+
+			    if maxend is None or m.end() > maxend:
+				maxend = m.end()
+
+			    # Trace the match
+			    if self.logfile:
+				if len(pattern) > 1:
+				    self.logfile.write('%s: match found in %.2fs.\npattern: "%s"\nbuffer: "%s||%s||%s"\n'%
+						       (self.desc, time.time() - start, escape(s), escape(before),
+							escape(matched), escape(after)))
+				else:
+				    self.logfile.write('%s: match found in %.2fs.\nbuffer: "%s||%s||%s"\n' %
+						       (self.desc, time.time() - start, escape(before), escape(matched),
+							escape(after)))
+
+			    if matchall:
+				del pattern[index]
+				# If all patterns have been found then
+				# truncate the buffer to the longest match,
+				# and then return.
+				if len(pattern) == 0:
+				    self.buf.truncate(0)
+				    self.buf.write(buf[maxend:])
+				    return buf
+				break
+
+			    # Consume matched portion of the buffer.
+			    self.buf.truncate(0)
+			    self.buf.write(after)
+			    
+			    return buf, before, after, m, index
+
+		    # If a single match was found then the match.
+		    if len(pattern) != olen:
+			continue
+
+		    if timeout is None:
+			self.cv.wait()
+		    else:
+			self.cv.wait(end - time.time())
+			if time.time() >= end:
+			    # Log the failure
+			    if self.logfile:
+				self.logfile.write('%s: match failed.\npattern: "%s"\nbuffer: "%s"\n"' %
+						   (self.desc, escape(s), escape(buf)))
+				self.logfile.flush()
+			    raise TIMEOUT ('timeout exceeded in match\npattern: "%s"\nbuffer: "%s"\n"' %
+					   (escape(s), escape(buf)))
+	    except TIMEOUT, e:
+		if (TIMEOUT, None) in pattern:
+		    return buf, buf, TIMEOUT, None, pattern.index((TIMEOUT, None))
+		raise e
+	finally:
+	    self.cv.release()
+
+class Expect (object):
+    def __init__(self, command, timeout=30, logfile=None, mapping = None, desc = None, cwd = None):
+	self.buf = "" # The part before the match
+	self.before = "" # The part before the match
+	self.after = "" # The part after the match
+	self.matchindex = 0 # the index of the matched pattern
+	self.match = None # The last match
+        self.mapping = mapping # The mapping of the test.
+        self.exitstatus = None # The exitstatus, either -signal or, if positive, the exit code.
+        self.killed = None # If killed, the signal that was sent.
+	self.desc = desc
+	self.logfile = logfile
+	self.timeout = timeout
+
+	if self.logfile:
+	    self.logfile.write('spawn: "%s"\n' % command)
+	    self.logfile.flush()
+
+        if win32:
+            # Don't rely on win32api
+            #import win32process
+                                      #creationflags = win32process.CREATE_NEW_PROCESS_GROUP)
+            CREATE_NEW_PROCESS_GROUP = 512
+            self.p = subprocess.Popen(command, cwd = cwd, shell=False, bufsize=0, stdin=subprocess.PIPE,
+                                      stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
+                                      creationflags = 512) # CREATE_NEW_PROCESS_GROUP
+        else:
+            self.p = subprocess.Popen(command, cwd = cwd, shell=True, bufsize=0, stdin=subprocess.PIPE,
+                                      stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
+	self.r = reader(desc, self.p, logfile)
+
+	# The thread is marked as a daemon thread. This is done so that if
+	# an expect script runs off the end of main without kill/wait on each
+	# spawned process the script will not hang tring to join with the
+	# reader thread. Instead __del__ (below) will be called which
+	# terminates and joins with the reader thread.
+	self.r.setDaemon(True)
+
+	self.r.start()
+
+    def __del__(self):
+	# Terminate and clean up.
+	if self.p is not None:
+	    self.terminate()
+
+    def expect(self, pattern, timeout = 10):
+    	"""pattern is either a string, or a list of string regexp patterns.
+
+	   timeout == None expect can block indefinitely.
+
+	   timeout == -1 then the default is used.
+	"""
+	if timeout == -1:
+	    timeout = self.timeout
+
+	if type(pattern) != list:
+	    pattern = [ pattern ]
+        def compile(s):
+            if type(s) == str:
+                return re.compile(s, re.S)
+            return None
+	pattern = [ ( p, compile(p) ) for p in pattern ]
+        try:
+            self.buf, self.before, self.after, self.match, self.matchindex = self.r.match(pattern, timeout)
+        except TIMEOUT, e:
+            self.buf = ""
+            self.before = ""
+            self.after = ""
+            self.match = None
+            self.matchindex = 0
+            raise e
+	return self.matchindex
+
+    def expectall(self, pattern, timeout = 10):
+    	"""pattern is a list of string regexp patterns.
+
+	   timeout == None expect can block indefinitely.
+
+	   timeout == -1 then the default is used.
+	"""
+	if timeout == -1:
+	    timeout = self.timeout
+
+	pattern = [ ( p, re.compile(p, re.S) ) for p in pattern ]
+        try:
+            self.buf = self.r.match(pattern, timeout, matchall = True)
+            self.before = ""
+            self.after = ""
+            self.matchindex = 0
+            self.match = None
+        except TIMEOUT, e:
+            self.buf = ""
+            self.before = ""
+            self.after = ""
+            self.matchindex = 0
+            self.match = None
+            raise e
+
+    def sendline(self, data):
+    	"""send data to the application.
+	"""
+	if self.logfile:
+	    self.logfile.write('%s: sendline: "%s"\n' % (self.desc, escape(data)))
+	    self.logfile.flush()
+	self.p.stdin.write("%s\n" % data)
+
+    def wait(self, timeout = None):
+    	"""Wait for the application to terminate for up to timeout seconds, or
+           raises a TIMEOUT exception. If timeout is None, the wait is
+           indefinite.
+
+           The exit status is returned. A negative exit status means
+           the application was killed by a signal.
+           """
+	if self.p is not None:
+
+            # Unfortunately, with the subprocess module there is no
+            # better method of doing a timed wait.
+            if timeout is not None:
+                end = time.time() + timeout
+                while time.time() < end and self.p.poll() is None:
+                    time.sleep(0.1)
+                if self.p.poll() is None:
+                    raise TIMEOUT ('timedwait exceeded timeout')
+
+	    self.exitstatus = self.p.wait()
+
+            # A Windows application with a negative exit status means
+            # killed by CTRL_BREAK. Fudge the exit status.
+            if win32 and self.exitstatus < 0:
+                assert self.killed is not None
+                self.exitstatus = -self.killed
+	    self.p = None
+	    self.r.join()
+	    # Simulate a match on EOF
+	    self.buf = self.r.getbuf()
+	    self.before = self.buf
+	    self.after = ""
+	    self.r = None
+	return self.exitstatus
+
+    def terminate(self):
+	"""Terminate the process."""
+        # First try to break the app. Don't bother if this is win32
+        # and we're using java. It won't break (BREAK causes a stack
+        # trace).
+        if self.hasInterruptSupport():
+            try:
+                if win32:
+                    # We BREAK since CTRL_C doesn't work (the only way to make
+                    # that work is with remote code injection).
+                    #
+                    # Using the ctypes module removes the reliance on the
+                    # python win32api
+                    #win32console.GenerateConsoleCtrlEvent(win32console.CTRL_BREAK_EVENT, self.p.pid)
+                    ctypes.windll.kernel32.GenerateConsoleCtrlEvent(1, self.p.pid) # 1 is CTRL_BREAK_EVENT
+                else:
+                   os.kill(self.p.pid, signal.SIGINT)
+            except:
+                traceback.print_exc(file=sys.stdout)
+
+            # If the break does not terminate the process within 5
+            # seconds, then terminate the process.
+            try:
+                self.wait(timeout = 5)
+                return
+            except TIMEOUT, e:
+                pass
+
+	try:
+            if win32:
+                # Next kill the app.
+                if Util.hasInterruptSupport():
+                    print "%s: did not respond to break. terminating: %d" % (self.desc, self.p.pid)
+                subprocess.TerminateProcess(self.p._handle, -1)
+            else:
+               os.kill(self.p.pid, signal.SIGKILL)
+            self.wait()
+	except:
+            traceback.print_exc(file=sys.stdout)
+
+    def kill(self, sig):
+        """Send the signal to the process."""
+        self.killed = sig # Save the sent signal.
+        if win32:
+            # Signals under windows are all turned into CTRL_BREAK_EVENT,
+            # except with Java since CTRL_BREAK_EVENT generates a stack
+            # trace.
+            #
+            # We BREAK since CTRL_C doesn't work (the only way to make
+            # that work is with remote code injection).
+            if self.hasInterruptSupport():
+                try:
+                    #
+                    # Using the ctypes module removes the reliance on the
+                    # python win32api
+                    ctypes.windll.kernel32.GenerateConsoleCtrlEvent(1, self.p.pid) # 1 is CTRL_BREAK_EVENT
+                    #win32console.GenerateConsoleCtrlEvent(win32console.CTRL_BREAK_EVENT, self.p.pid)
+                except:
+                    traceback.print_exc(file=sys.stdout)
+            else:
+                subprocess.TerminateProcess(self.p._handle, sig)
+        else:
+            os.kill(self.p.pid, sig)
+
+    # status == 0 is normal exit status for C++
+    #
+    # status == 130 is normal exit status for a Java app that was
+    # SIGINT interrupted.
+    #
+    def waitTestSuccess(self, exitstatus = 0, timeout = None):
+        """Wait for the process to terminate for up to timeout seconds, and
+        validate the exit status is as expected."""
+
+        def test(result, expected):
+            if expected != result:
+                print "unexpected exit status: expected: %d, got %d" % (expected, result)
+                assert False
+
+        self.wait(timeout)
+        if self.mapping == "java":
+            if self.killed is not None:
+                if win32:
+                    test(self.exitstatus, self.killed)
+                else:
+                    if self.killed == signal.SIGINT:
+                        test(130, self.exitstatus)
+                    else:
+                        assert False
+            else:
+                assert self.exitstatus == exitstatus
+        else:
+            test(self.exitstatus, exitstatus)
+
+    def hasInterruptSupport(self):
+        """Return True if the application gracefully terminated, False otherwise."""
+        if win32 and self.mapping == "java":
+            return False
+        return True