It is good practice to deny traffic that is not HTTPS to your S3 bucket. For this reason, I wrote a script that I can use to apply a standard policy to each of my S3 buckets. While the script could be more robust to iterate through each bucket in my account, I decided to… Continue reading Adding a Standard Bucket Policy
Category: Python
Security Group ID Finder
I have been working on deploying resources to a lot of AWS accounts lately where each account has the same network infrastructure. When deploying Lambdas, I had the common name of the security group but not the ID. I wrote this utility to get the security group ID for me quickly. This is a simple… Continue reading Security Group ID Finder
A Dynamo Data Migration Tool
Have you ever wanted to migrate data from one Dynamo DB table to another? I haven’t seen an AWS tool to do this so I wrote one using Python. The process is pretty simple. First, we get all of our data from our source table. We store this in a list. Next, we iterate over… Continue reading A Dynamo Data Migration Tool
Querying and Editing a Single Dynamo Object
I have a workflow that creates a record inside of a DynamoDB table as part of a pipeline within AWS. The record has a primary key of the Code Pipeline job. Later in the pipeline I wanted to edit that object to append the status of resources created by this pipeline. In order to do… Continue reading Querying and Editing a Single Dynamo Object
Searching S3 Buckets for an Object
I wrote this code for a project that I was working on for a client. The goal was to look in an S3 bucket to find objects that match a specific identification number. Specifically, they are looking to return audio logs from calls in an AWS Connect instance. In this script, we are utilizing Boto3… Continue reading Searching S3 Buckets for an Object
Subscribing All SES Identities to an SNS Topic
I recently ran across an issue where I was experiencing many bounced emails on my Amazon SES account. So much so that Amazon reached out and put me on a warning notice. I realized that I had no logging in place to handle this. In order to create a logging mechanism I decided to send… Continue reading Subscribing All SES Identities to an SNS Topic
Building Dynamic DNS with Route53 and PFSense
I use PFSense as my home router, firewall, VPN and much more. I’m sure a lot of my readers do as well. One thing that I have always set up is an entry in Route53 that points to my public IP address on my PFSense box. However, I use Comcast so, my IP address is… Continue reading Building Dynamic DNS with Route53 and PFSense
Pandas & NumPy with AWS Lambda
Fun fact: Pandas and NumPy don’t work out of the box with Lambda. The libraries that you might download from your development machine probably won’t work either. The standard Lambda Python environment is very barebones by default. There is no point in loading in a bunch of libraries if they aren’t needed. This is why… Continue reading Pandas & NumPy with AWS Lambda
Concatenating Multi-Sheet Excel Files with Python
I recently came across a data source that used multi-sheets within an Excel file. My dashboard cannot read a multi-sheet Excel file so I needed to combine them into one sheet. The file is being uploaded into an S3 bucket and then needs to move through the data lake to be read into the dashboard.… Continue reading Concatenating Multi-Sheet Excel Files with Python
A File Management Architecture
This post is a continuation of my article: “A File Extraction Project”. This project has been a great learning experience for both frontend and backend application architecture and design. Below you will find a diagram and an explanation of all the pieces that make this work. The entire architecture is powered by Flask on an… Continue reading A File Management Architecture