[SPARK-4745] Fix get_existing_cluster() function with multiple security groups

The current get_existing_cluster() function would only find an instance belonged to a cluster if the instance's security groups == cluster_name + "-master" (or "-slaves"). This fix allows for multiple security groups by checking if the cluster_name + "-master" security group is in the list of groups for a particular instance.

Author: alexdebrie <[email protected]>

Closes #3596 from alexdebrie/master and squashes the following commits:

9d51232 [alexdebrie] Fix get_existing_cluster() function with multiple security groups

(cherry picked from commit 794f3ae)
Signed-off-by: Josh Rosen <[email protected]>

Author: alexdebrie

ec2/spark_ec2.py

@@ -506,9 +506,9 @@ def get_existing_cluster(conn, opts, cluster_name, die_on_error=True):
         active = [i for i in res.instances if is_active(i)]
         for inst in active:
             group_names = [g.name for g in inst.groups]
-            if group_names == [cluster_name + "-master"]:
+            if (cluster_name + "-master") in group_names:
                 master_nodes.append(inst)
-            elif group_names == [cluster_name + "-slaves"]:
+            elif (cluster_name + "-slaves") in group_names:
                 slave_nodes.append(inst)
     if any((master_nodes, slave_nodes)):
         print "Found %d master(s), %d slaves" % (len(master_nodes), len(slave_nodes))