lucasrcosta

# -*- coding: utf-8 -*-
# /var/runtime/awslambda/bootstrap.py
"""
aws_lambda.bootstrap.py
Amazon Lambda

Copyright (c) 2013 Amazon. All rights reserved.

Lambda runtime implemention
"""

darahayes

KMS Encrypt and Decrypt Filters for Ansible

This Gist shows how you can encrypt/decrypt variables with KMS during your Ansible playbook execution. While google searching I couldn't find anything useful on the subject.

Let's say your project has a folder of playbooks called plays.

In that folder, create a folder called filter_plugins and insert kms_filters.py (included in this gist)

nmarley

import boto3
import base64

if __name__ == '__main__':
    session = boto3.session.Session()

    kms = session.client('kms')

    encrypted_password = 'AQECAHjgTiiE7TYRGp5Irf8jQ3HzlaQaHGYgsUJDaavnHcFm0gAAAGswaQYJKoZIhvcNAQcGoFwwWgIBADBVBgkqhkiG9w0BBwEwHgYJYIZIAWUDBAEuMBEEDDwxVQuG0oVwpkU7nQIBEIAoVGk1/wpserb+GVUOzE7PiL/Nr9fTDFKZfpKpF0ip2ct4B2q0Wn6ZZw=='
    binary_data = base64.b64decode(encrypted_password)

jvwing

<?xml version="1.0" ?>
<template encoding-version="1.0">
  <description>Template for processing events from Amazon CloudTrail.

See https://www.batchiq.com/process-cloudtrail-events-with-nifi.html</description>
  <groupId>f46d6392-0158-1000-73ba-16e80594f898</groupId>
  <name>CloudTrail_Event_Processing_Starter_Kit</name>
  <snippet>
    <processGroups>
      <id>363e44cf-ed77-17f2-0000-000000000000</id>

alopresto

Certificate and key commands

1. Verify connection:

$ openssl s_client -connect <host:port> -debug -state -cert <path_to_your_cert.pem> -key <path_to_your_key.pem> -CAfile <path_to_your_CA_cert.pem>

2. Export client cert from PKCS12 keystore to PEM:

ijokarumawak

NiFi Flow Example: Move Files between S3 Buckets

Please be careful with specifying the right bucket name, region and credential.

When I misconfigured region, I got the following error:

2016-11-17 10:51:07,828 ERROR [Timer-Driven Process Thread-9] o.a.nifi.processors.aws.s3.PutS3Object
com.amazonaws.services.s3.model.AmazonS3Exception: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint. (Service: Amazon S3; Status Code: 301; ErrorCode: PermanentRedirect; Request ID: 99A62426D8544997)

alopresto

hw12203:/Users/alopresto/Workspace/scratch/csr-demo (master) alopresto
🔓 3s @ 16:33:45 $ keytool -genkey -alias nifi -keyalg RSA -keysize 2048 -keystore keystore.jks
Enter keystore password:
Re-enter new password:
What is your first and last name?
  [Unknown]:  nifi.apache.org
What is the name of your organizational unit?
  [Unknown]:  NiFi
What is the name of your organization?
  [Unknown]:  Apache

simplesteph

{
  "swagger" : "2.0",
  "info" : {
    "description" : "The Rest Api provides programmatic access to command and control a NiFi instance in real time. Start and \n                                            stop processors, monitor queues, query provenance data, and more. Each endpoint below includes a description,\n                                            definitions of the expected input and output, potential response codes, and the authorizations required\n                                            to invoke each service.",
    "version" : "1.0.0",
    "title" : "NiFi Rest Api",
    "contact" : {
      "url" : "https://nifi.apache.org",
      "email" : "dev@nifi.apache.org"
    },

albertmenglongli

'''
This script performs efficient concatenation of files stored in S3. Given a
folder, output location, and optional suffix, all files with the given suffix
will be concatenated into one file stored in the output location.

Concatenation is performed within S3 when possible, falling back to local
operations when necessary.

Run `python combineS3Files.py -h` for more info.
'''

ijin

from __future__ import print_function  
import os  
import sys  
import subprocess  
import boto3  
  
from botocore.client import Config  
  
from dulwich.errors import (  
    SendPackError,