Things I have automated at work
Reviewing firewall rules across AWS, GCP and Azure
Tools: AWS CDK, Wiz, AWS, Google Drive, Slack
Languages: TypeScript, Python
To ensure SOC 2 compliance, we conduct multiple firewall reviews throughout the year. Automation has increased the frequency of these reviews and reduced the time required from a full day to a maximum of one hour.
Wiz exports firewall rules from AWS, GCP, and Azure to AWS S3. Lambdas process the raw file and compare it with previous reports to generate a diff of rules, writing back to S3. A different lambda copies that file to Google Drive for easy collaboration.
A Slack message is then also sent to both the Security team and the Security Champions for review.
Process
sequenceDiagram Wiz->>S3: Export S3<<->>Lambda: Process S3<<->>Lambda: Diff Lambda->>Google Drive: Upload Lambda-->>Slack: Send notification
Example
Closing tickets referencing AMIs that need patching that are no longer used
Tools: AWS CDK, Wiz, AWS, Jira
Languages: TypeScript, Python
EKS and Karpenter allow nodes to have short lifecycles, which is great for new AMIs. The vulnerability management program creates tickets to monitor AMIs with critical or high CVEs, and using a newer AMI can resolve many of those issues, but some require manual intervention. Some tickets may remain “in progress” despite no longer using the AMI.
This automation gets all the Jira Issues part of the VM Program containing AMIs and checks against Wiz to see if it’s still in use. If it’s no longer found in the current snapshot of resources, the ticket is closed and a Slack message sent. This saves us a bit of time and effort as it stops us from following up on AMIs we no longer use.
Process
sequenceDiagram Lambda<<->>Jira: Get issues Lambda<<->>Wiz: Check AMIs Lambda->>Jira: Resolve issue Lambda-->>Slack: Send notification
Example
A Slackbot that tags the relevant team on new messages in the Security channel
Tools: AWS CDK, Jupyter, Zapier, Slack, AWS
Languages: TypeScript, Python
This is my favourite thing I’ve automated so far. Our Security Org has a few teams, with people spread across multiple countries and time-zones. We use a shared Slack channel for company-wide messaging, but we sometimes miss an odd message or two. To prevent that from happening, I had the idea of having a bot tag the relevant team on incoming messages.
It took me a few weeks to review the past two years of Slack messages in the security channel. I added a “classify” reaction and stored the message and the relevant team in a sheet using a Zap. Once I had enough data, I started researching on how to classify text and came across scikit-learn. A Jupiter notebook and multiple tutorials later, I had something that somewhat resembled what I was after. I then set up a Zap to trigger off a new Slack message, which would make a POST request to my API endpoint to classify the message. To not cause any notification fatigue from misclassifications, I set it to not tag the team on confidence ratings below 60%.
Every so often, I add more data or try new things with scikit-learn and end up AB testing the new model against the current one.
Process
Labelling of data
sequenceDiagram Slack->>Zapier: "Classify" reaction added to message Zapier->>Zapier: Store message and team in Tables
Tagging team on new messages
sequenceDiagram Slack->>Zapier: New message in Security channel Zapier<<->>AWS: POST request to model Zapier->>Slack: Tag team
Example
Checking permissions of shared documents shared in Slack
Tools: Zapier, Slack
Languages: Python
One thing I see people do is scope internal documents to “Anyone with the link” rather than “Anyone in the company”. These links can often end up in places they shouldn’t, so I built a Zap to check the permissions of the document when a link is posted in Slack. The Zap filters links matching our “document management system,” and then, using ‘Code by Zapier,’ it makes a GET request to the URL to check for a HTTP 200 (public) or HTTP 302 (redirects to a login page) response.
Process
sequenceDiagram Slack->>Zapier: New message containing "tool" URL Zapier->>Tool: GET request to URL Tool-->>Zapier: HTTP 200 - NO AUTH Tool-->>Zapier: HTTP 302 - AUTH Zapier->>Slack: Notification for "NO AUTH"
Example
A Slackbot for GitLab Merge Requests including their statuses
Tools: AWS CDK, GitLab, Slack
Languages: Golang, TypeScript
Each morning, I review Wiz and review any code review requests assigned to me overnight (we’re a distributed team). To see what interesting things other teams in my Zone are doing, I built a Slackbot that posts a message to a channel when a new merge request is opened. The bot also adds reactions to the message depending on whether the merge request is approved, merged, or closed.
As it catches all Merge Request events from GitLab, I ended up setting up multiple feed channels for:
- Security
- SRE
- Data
Process
sequenceDiagram GitLab->>AWS: Merge Request Event Note over AWS,Slack: MR Opened AWS->>Slack: New Slack message Note over AWS,Slack: MR Approved AWS->>Slack: "Approved" reaction added Note over AWS,Slack: MR Merged AWS->>Slack: All reactions removed AWS->>Slack: "Merged" reaction removed Note over AWS,Slack: MR Closed AWS->>Slack: All reactions removed AWS->>Slack: "Closed" reaction removed
Example
A CLI tool to check tagging and labelling compliance
Languages: Golang
This was a tool I built when testing Cursor. I wanted to see how far I could get with writing as little code as possible. It’s quite a useful tool that I run from time to time to check if we’ve missed any tags or labels in our code. It supports Terraform, Helm and Dockerfiles, which covers most of our use-cases.
Example
Reacting to a Slack message to create a ticket
Tools: Zapier, Slack
This is something most teams have set up at Zapier. It’s a simple Zap which triggers off a Slack reaction to a message and creates a Jira ticket. It saves us a bit of time and effort, as we can pre-configure the random required fields beforehand.
Process
sequenceDiagram Slack->>Zapier: Reaction added Zapier<<->>Jira: Create Issue Zapier->>Slack: Comment Issue