https://www.infraspec.dev/blog Field notes on observability, infrastructure, cloud, reliability and software engineering. en https://www.infraspec.dev/blog/scaling-otel-collectors-push-and-pull https://www.infraspec.dev/blog/scaling-otel-collectors-push-and-pull Sat, 17 Oct 2026 00:00:00 GMT Arihant Hirawat This came out of a conversation with one of our engineers about a production incident she'd just been through. It sounded small at first (a pod got… https://www.infraspec.dev/blog/taking-back-control https://www.infraspec.dev/blog/taking-back-control Thu, 01 Jan 2026 00:00:00 GMT Nimisha G J The story of every growing system is fundamentally a story about losing and regaining control. https://www.infraspec.dev/blog/pragmatism-over-perfection https://www.infraspec.dev/blog/pragmatism-over-perfection Mon, 21 Apr 2025 00:00:00 GMT Aleric Cusher We as Engineers often chase perfection. It fuels our curiosity, sharpens our skills, and makes us feel good about the things we build. But at the same… https://www.infraspec.dev/blog/apache-spark-unleashing-big-data-with-rdds-dataframes-and-beyond https://www.infraspec.dev/blog/apache-spark-unleashing-big-data-with-rdds-dataframes-and-beyond Sun, 24 Nov 2024 00:00:00 GMT Shivani Banke Before I knew Spark, I mostly used only Pandas and NumPy for data processing. They worked well for small datasets, and I never really had any… https://www.infraspec.dev/blog/multi-tenant-system-with-aws-cdk https://www.infraspec.dev/blog/multi-tenant-system-with-aws-cdk Fri, 22 Nov 2024 00:00:00 GMT Mufaddal Shakir In this blog, I will be taking you on a journey of building the scalable and efficient IAC solution that we build for our multi-tenant system. Here we… https://www.infraspec.dev/blog/hive-to-unity-catalog-data-migration-databricks https://www.infraspec.dev/blog/hive-to-unity-catalog-data-migration-databricks Sat, 09 Nov 2024 00:00:00 GMT Arihant Hirawat We had to migrate extensive data from Hive Metastore to Unity Catalog in a regulated large-scale enterprise environment. This task required precise… https://www.infraspec.dev/blog/clickhouse-benchmarking https://www.infraspec.dev/blog/clickhouse-benchmarking Mon, 21 Oct 2024 00:00:00 GMT Rohit Nagmoti Imagine being a Formula One driver, racing at breakneck speeds, but without any telemetry data to guide you. It’s a thrilling ride, but one wrong turn or… https://www.infraspec.dev/blog/cache-strategies https://www.infraspec.dev/blog/cache-strategies Wed, 25 Sep 2024 00:00:00 GMT Sanjay Singh Caching is a technique that enhances application performance by temporarily storing frequently accessed data in high-speed data storage hardware like… https://www.infraspec.dev/blog/terragrunt-envs-management https://www.infraspec.dev/blog/terragrunt-envs-management Tue, 10 Sep 2024 00:00:00 GMT Nimisha G J Managing multiple environments was a never-ending headache for me. Like many others in the DevOps world, I was responsible for deploying applications… https://www.infraspec.dev/blog/nfs-fs-as-docker-volume https://www.infraspec.dev/blog/nfs-fs-as-docker-volume Mon, 19 Aug 2024 00:00:00 GMT Nimisha G J As an Infrastructure engineer, I’ve had my fair share of experiences with containerized environments and the challenges of managing data persistence. One… https://www.infraspec.dev/blog/aws-ecs-fargate-vs-self-managed-ec2 https://www.infraspec.dev/blog/aws-ecs-fargate-vs-self-managed-ec2 Tue, 13 Aug 2024 00:00:00 GMT Saumitra Topinkatti The Amazon Elastic Container Service (ECS) from Amazon deploys and manages containerised applications in two distinct modes of operation: Fargate and… https://www.infraspec.dev/blog/terraform_secrets_management_guide https://www.infraspec.dev/blog/terraform_secrets_management_guide Tue, 06 Aug 2024 00:00:00 GMT Rahul Patil Imagine you’re working on a project where you need to deploy resources to aws using terraform. In a rush to get things done, you decide to hard-code your… https://www.infraspec.dev/blog/tag-strat-blog https://www.infraspec.dev/blog/tag-strat-blog Mon, 29 Jul 2024 00:00:00 GMT Rohit Nagmoti Managing cloud resources efficiently in AWS requires a robust tagging strategy. Tags are key-value pairs attached to resources, providing crucial… https://www.infraspec.dev/blog/genai-dictionary-part1-llm https://www.infraspec.dev/blog/genai-dictionary-part1-llm Fri, 21 Jun 2024 00:00:00 GMT Nandhitha Kamal We bring to you this weekly series of articles, to help understand and demystify the lexicon in the GenAI space. Here is part 1 - Large Language Model. https://www.infraspec.dev/blog/building-a-discord-gpt-bot https://www.infraspec.dev/blog/building-a-discord-gpt-bot Fri, 31 May 2024 00:00:00 GMT Nandhitha Kamal Amidst the excitement surrounding AI, we were eager to delve into this field ourselves. As engineers, we wanted more than just a casual conversation with… https://www.infraspec.dev/blog/argo-rollout-aws-alb https://www.infraspec.dev/blog/argo-rollout-aws-alb Mon, 27 May 2024 00:00:00 GMT Mufaddal Shakir The blog discusses resolving a deployment issue with 502 errors on AWS EKS using AWS ALB and Argo Rollouts. It details the root cause, attempted… https://www.infraspec.dev/blog/header-based-traffic-routing-using-argo-rollouts https://www.infraspec.dev/blog/header-based-traffic-routing-using-argo-rollouts Mon, 27 May 2024 00:00:00 GMT Mufaddal Shakir This blog explores how to use Argo Rollouts for deploying software updates smoothly. It covers the challenges faced when rolling back updates and… https://www.infraspec.dev/blog/kubernetes-rbac https://www.infraspec.dev/blog/kubernetes-rbac Mon, 29 Apr 2024 00:00:00 GMT Sudeep Kulkarni Role-Based Access Control (RBAC) is a crucial feature in Kubernetes that allows administrators to define and manage permissions for users and services… https://www.infraspec.dev/blog/guide-to-implement-winston-logger https://www.infraspec.dev/blog/guide-to-implement-winston-logger Mon, 29 Apr 2024 00:00:00 GMT Shivani Banke “The only thing more satisfying than fixing a bug is reading the frustrated log messages that led you there.” This humorous yet insightful quote… https://www.infraspec.dev/blog/github-runners https://www.infraspec.dev/blog/github-runners Thu, 25 Apr 2024 00:00:00 GMT Rahul Patil In today’s fast-paced world of software development, automation is key to staying competitive. GitHub, a leading platform for collaboration and version… https://www.infraspec.dev/blog/docker-deep-dive https://www.infraspec.dev/blog/docker-deep-dive Wed, 24 Apr 2024 00:00:00 GMT Sudeep Kulkarni This blog will cover the foundational concepts of Docker layers, caching mechanisms, and Dockerfile optimization. Additionally, provide insights on… https://www.infraspec.dev/blog/securing-and-rotating-secrets-with-aws-secrets-manager-part-2 https://www.infraspec.dev/blog/securing-and-rotating-secrets-with-aws-secrets-manager-part-2 Mon, 22 Apr 2024 00:00:00 GMT Saumitra Topinkatti In Part 1 , we discussed upon configuring AWS Secrets Manager, AWS Lambda, and Automatic Rotation for our Secret. We also defined permissions for our… https://www.infraspec.dev/blog/securing-and-rotating-secrets-with-aws-secrets-manager-part-1 https://www.infraspec.dev/blog/securing-and-rotating-secrets-with-aws-secrets-manager-part-1 Sun, 21 Apr 2024 00:00:00 GMT Saumitra Topinkatti Managing secrets has always been a significant aspect when deploying applications. Typically, we set up a pipeline to provide secrets to our application… https://www.infraspec.dev/blog/tftp-overview https://www.infraspec.dev/blog/tftp-overview Fri, 19 Apr 2024 00:00:00 GMT Nimisha G J TFTP known as Trivial File transfer protocol is a simple file transfer protocol with features such as get or put files. Unlike FTP or SFTP which can… https://www.infraspec.dev/blog/rpi-netboot-automation https://www.infraspec.dev/blog/rpi-netboot-automation Wed, 17 Apr 2024 00:00:00 GMT Rohit Nagmoti In this blog, we’ll delve into automating the netbooting process using a bash script ( pxeService.sh ) and an address list ( addresslist.txt ) to enhance… https://www.infraspec.dev/blog/rpi-netboot-deep-dive https://www.infraspec.dev/blog/rpi-netboot-deep-dive Tue, 16 Apr 2024 00:00:00 GMT Rohit Nagmoti The Preboot Execution Environment (PXE) specification describes a standardized client-server environment that boots a software assembly retrieved from a… https://www.infraspec.dev/blog/cron-jobs https://www.infraspec.dev/blog/cron-jobs Mon, 15 Apr 2024 00:00:00 GMT Shivani Banke In this article, tailored specifically for users of Unix-like systems such as Linux, we’ll explore everything you need to know about cron jobs. From… https://www.infraspec.dev/blog/linux-memory-swap https://www.infraspec.dev/blog/linux-memory-swap Mon, 15 Apr 2024 00:00:00 GMT Nimisha G J This blog will help you figure out how to check how much memory is left on your Linux machine. Knowing this will tell you if your server is running low… https://www.infraspec.dev/blog/building-internal-tools https://www.infraspec.dev/blog/building-internal-tools Thu, 11 Apr 2024 00:00:00 GMT Abhishek Sangwan This article takes you through what it is like to build an internal tool while dealing with a lot of legacy systems where a lot of tech debts already… https://www.infraspec.dev/blog/java-streams https://www.infraspec.dev/blog/java-streams Tue, 05 Mar 2024 00:00:00 GMT Rahul Patil Imagine a lazy river carrying data elements one by one. That’s essentially what a stream is – a sequence of elements that you can process on the fly,… https://www.infraspec.dev/blog/tdd-design-benefits https://www.infraspec.dev/blog/tdd-design-benefits Wed, 18 Jan 2023 00:00:00 GMT Chetan Kothari Most of us think of TDD as a tool for software testing and verification. But if used effectively it is more than that. https://www.infraspec.dev/blog/lightweight-k3s-cluster-raspberry-pi https://www.infraspec.dev/blog/lightweight-k3s-cluster-raspberry-pi Mon, 02 Jan 2023 00:00:00 GMT Rajat Tomar As a developer at Infraspec, I am responsible for managing internal tooling and ensuring the smooth operation of local tooling and network services in… https://www.infraspec.dev/blog/is-dns-migration-a-tricky-affair https://www.infraspec.dev/blog/is-dns-migration-a-tricky-affair Thu, 15 Dec 2022 00:00:00 GMT Never trust a friend who says DNS migration is easy. Your instinct says it’s easy, well don’t trust your instinct. In this post, I will be sharing my… https://www.infraspec.dev/blog/office-home-lab-internet-failover-setup https://www.infraspec.dev/blog/office-home-lab-internet-failover-setup Thu, 15 Dec 2022 00:00:00 GMT Arihant Hirawat An overview of how to set up internet failover for an office or home lab. https://www.infraspec.dev/blog/setting-up-ingress-on-eks https://www.infraspec.dev/blog/setting-up-ingress-on-eks Wed, 30 Nov 2022 00:00:00 GMT Rajat Tomar An Ingress in Kubernetes provides external access to your Kubernetes services. You configure access by creating a collection of routing rules that define… https://www.infraspec.dev/blog/container-networking-deep-dive-p3 https://www.infraspec.dev/blog/container-networking-deep-dive-p3 Mon, 21 Nov 2022 00:00:00 GMT Arihant Hirawat In part 3 of this series, we will see how containers running on the host communicates with the outside world i.e., the internet https://www.infraspec.dev/blog/container-networking-deep-dive-p2 https://www.infraspec.dev/blog/container-networking-deep-dive-p2 Sun, 20 Nov 2022 00:00:00 GMT Arihant Hirawat In part 2 of this series, we will demystify how multiple containers running on the same host communicates with the host and vice versa. https://www.infraspec.dev/blog/container-networking-deep-dive-p1 https://www.infraspec.dev/blog/container-networking-deep-dive-p1 Sat, 19 Nov 2022 00:00:00 GMT Arihant Hirawat In part 1 of this series, we will demystify how a container communicates with the host and vice versa.