Skip to content

Instantly share code, notes, and snippets.

@Kulasangar
Created November 4, 2018 19:00
Show Gist options
  • Select an option

  • Save Kulasangar/171b517fc5115e0504dba3c2ecabd98f to your computer and use it in GitHub Desktop.

Select an option

Save Kulasangar/171b517fc5115e0504dba3c2ecabd98f to your computer and use it in GitHub Desktop.
import boto3
connection = boto3.client(
'emr',
region_name='us-east-1',
aws_access_key_id='YOUR ACCESS KEY',
ws_secret_access_key='YOUR SECRET KEY',
)
cluster_id = 'give the cluster id'
if cluster_id != '':
step = {
'Name': 'file-copy-step-5',
'ActionOnFailure': 'CONTINUE',
'HadoopJarStep': {
'Jar': 's3://kula-emr-test/jars/CopyFilesS3-1.0-SNAPSHOT-jar-with-dependencies.jar',
'Args': ['test.xml', 'kula-emr-test', 'kula-emr-test-2']
}
}
action = connection.add_job_flow_steps(JobFlowId=cluster_id, Steps=[step])
print "Added step: %s" % (action)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment