I recently ran across an issue where I was experiencing many bounced emails on my Amazon SES account. So much so that Amazon reached out and put me on a warning notice. I realized that I had no logging in place to handle this. In order to create a logging mechanism I decided to send… Continue reading Subscribing All SES Identities to an SNS Topic
Category: Python
Building Dynamic DNS with Route53 and PFSense
I use PFSense as my home router, firewall, VPN and much more. I’m sure a lot of my readers do as well. One thing that I have always set up is an entry in Route53 that points to my public IP address on my PFSense box. However, I use Comcast so, my IP address is… Continue reading Building Dynamic DNS with Route53 and PFSense
Pandas & NumPy with AWS Lambda
Fun fact: Pandas and NumPy don’t work out of the box with Lambda. The libraries that you might download from your development machine probably won’t work either. The standard Lambda Python environment is very barebones by default. There is no point in loading in a bunch of libraries if they aren’t needed. This is why… Continue reading Pandas & NumPy with AWS Lambda
Concatenating Multi-Sheet Excel Files with Python
I recently came across a data source that used multi-sheets within an Excel file. My dashboard cannot read a multi-sheet Excel file so I needed to combine them into one sheet. The file is being uploaded into an S3 bucket and then needs to move through the data lake to be read into the dashboard.… Continue reading Concatenating Multi-Sheet Excel Files with Python
A File Management Architecture
This post is a continuation of my article: “A File Extraction Project”. This project has been a great learning experience for both frontend and backend application architecture and design. Below you will find a diagram and an explanation of all the pieces that make this work. The entire architecture is powered by Flask on an… Continue reading A File Management Architecture
A File Extraction Project
I had a client approach me regarding a set of files they had. The files were a set of certificates to support their products. They deliver these files to customers in the sales process. The workflow currently involves manually packaging the files up into a deliverable format. The client asked me to automate this process… Continue reading A File Extraction Project
A Self Hosted Server Health Check
I’m not big on creating dashboards. I find that I don’t look at them enough to warrant hosting the software on an instance and having to have the browser open to the page all the time. Instead, I prefer to be alerted via Slack as much as possible. I wrote scripts to collect DNS records… Continue reading A Self Hosted Server Health Check
Collecting DNS Records from Route53
I had an idea the other day to query my web servers to get a response code and potentially alert me based on the result. Ideally, I would get a Slack notification on any response that is not “200”. I was thinking that I could run this either on a CRON so that I have… Continue reading Collecting DNS Records from Route53
Where Is It 5 O’Clock Pt: 4
As much as I’ve scratched my head working on this project it has been fun to learn some new things and build something that isn’t infrastructure automation. I’ve learned some frontend web development some backend development and utilized some new Amazon Web Services products. With all that nice stuff said I’m proud to announce that… Continue reading Where Is It 5 O’Clock Pt: 4
Where Is It Five O’Clock Pt: 3
So I left this project at a point where I felt it needed to be re-architected based on the fact that Flask only executes the function once and not every time the page loads. I re-architected the application in my head to include an API that calls the Lambda function and returns a list of… Continue reading Where Is It Five O’Clock Pt: 3