I was working with a customer the other day trying to come up with a way to import a bunch of IP addresses into a white list on AWS. We came up with the approach of using Managed Prefix Lists in VPC. I wrote some Python in order to grab some code from an API… Continue reading Updating AWS Managed Prefix Lists
Tag: python
Automating Security Group Rule Removal
I’m using an Amazon Web Services Security Group as a way to allow traffic into an EC2 instance for the instance’s users. The users can give themselves access through a web interface that I wrote for them. Maybe I’ll cover that in a different post. I found recently that the Security Group was nearing its… Continue reading Automating Security Group Rule Removal
EC2 Reservation Notification
I realized today that I haven’t updated my EC2 reservations recently. Wondering why I never did this I came to understand that there was no way that I was getting notified that the reservations were expiring. I spent the day putting together a script that would look through my reservations, assess the time of their… Continue reading EC2 Reservation Notification
Security Group ID Finder
I have been working on deploying resources to a lot of AWS accounts lately where each account has the same network infrastructure. When deploying Lambdas, I had the common name of the security group but not the ID. I wrote this utility to get the security group ID for me quickly. This is a simple… Continue reading Security Group ID Finder
A Dynamo Data Migration Tool
Have you ever wanted to migrate data from one Dynamo DB table to another? I haven’t seen an AWS tool to do this so I wrote one using Python. The process is pretty simple. First, we get all of our data from our source table. We store this in a list. Next, we iterate over… Continue reading A Dynamo Data Migration Tool
Querying and Editing a Single Dynamo Object
I have a workflow that creates a record inside of a DynamoDB table as part of a pipeline within AWS. The record has a primary key of the Code Pipeline job. Later in the pipeline I wanted to edit that object to append the status of resources created by this pipeline. In order to do… Continue reading Querying and Editing a Single Dynamo Object
Pandas & NumPy with AWS Lambda
Fun fact: Pandas and NumPy don’t work out of the box with Lambda. The libraries that you might download from your development machine probably won’t work either. The standard Lambda Python environment is very barebones by default. There is no point in loading in a bunch of libraries if they aren’t needed. This is why… Continue reading Pandas & NumPy with AWS Lambda
Concatenating Multi-Sheet Excel Files with Python
I recently came across a data source that used multi-sheets within an Excel file. My dashboard cannot read a multi-sheet Excel file so I needed to combine them into one sheet. The file is being uploaded into an S3 bucket and then needs to move through the data lake to be read into the dashboard.… Continue reading Concatenating Multi-Sheet Excel Files with Python
Where Is It 5 O’Clock Pt: 4
As much as I’ve scratched my head working on this project it has been fun to learn some new things and build something that isn’t infrastructure automation. I’ve learned some frontend web development some backend development and utilized some new Amazon Web Services products. With all that nice stuff said I’m proud to announce that… Continue reading Where Is It 5 O’Clock Pt: 4
Where Is It Five O’Clock Pt: 3
So I left this project at a point where I felt it needed to be re-architected based on the fact that Flask only executes the function once and not every time the page loads. I re-architected the application in my head to include an API that calls the Lambda function and returns a list of… Continue reading Where Is It Five O’Clock Pt: 3