Connecting a Flask Application to AWS RDS (MySQL)
Project Description Amazon Web Services (AWS) provides a managed SQL database service called Amazon Relational Database Service (Amazon RDS). Amazon RDS uses various database engines to manage the data, migrations, backup, and recovery. In this project, we will create a database instance of SQL over RDS. After that, we will create a key pair value, use a security group and make an EC2 instance using Amazon Machine Image (AMI). Then, we’ll use the endpoint of that database to connect it with the EC2 instance and install the required packages for the Flask application. In the end, we’ll deploy the Flask application on EC2 and connect the application to use the AWS RDS (MySQL) database. The AWS Command Line Interface (CLI) is a powerful utility to manage all AWS services through a shell prompt. You can use the CLI to access and configure various AWS services and automate tasks using scripts. For your convenience, the AWS CLI has been downloaded and installed using the official link Verify that the utility is properly configured by running the following command: When you’ve successfully run this command, you’ll see the version details of the AWS CLI. This ensures that the AWS CLI has been installed correctly. Use the following command to list all commands available through the AWS CLI: Press the “Q” key to exit and return to the shell prompt. Note: In this project, you’ll create and export some environment variables using the environment.sh bash script. Step 2: Configuration: To configure the AWS account, run the configure command available through the AWS CLI and enter credentials, region, and output format. A few sample regions available in the AWS CLI are listed in the following table: Region Name Region US East (Ohio) us-east-2 US East (N. Virginia) us-east-1 US West (N. California) us-west-1 Some sample output formats available through AWS CLI are shown below: Note: It’s recommended to create a new AWS AccessKeyId and SecretAccessKey by creating a new IAM User for this project. To learn how to generate these keys, follow this link. Make sure to set up the AmazonEC2FullAccess user policy for the IAM user. Type the following command in the terminal: After executing this command, add the following parameters: Step 3: Create a New RDS Amazon Web Services (AWS) provides managed SQL database services the called Amazon Relational Database Service (Amazon RDS). Amazon RDS provides various database engines to manage the data, migrations, backup, and recovery. Let’s create a new database using the aws rds command. Add the following parameters while creating a new RDS: After creating the RDS instance, assign the value of VpcSecurityGroups.VpcSecurityGroupId to the SG_ID variable in the environment.sh file. After adding the value to the file, use the following command to export the value: source environment.sh Step 4 : Describe DB Instances Use the aws rds command to list all DB instances associated with the AWS account. After describing the RDS instance, assign the value of Endpoint.Address to the RDS_ADDRESS variable in the environment.sh file. Note: The endpoint may take some time to show up because instances created in the previous task will take some time to be initialized. Run the following: source environment.sh Step 5: Add a New Inbound Rule to the Security Groups The security group has some default inbound and outbound rules. MySQL that will be running on port 3306. The inbound rule will specify the port number and IP range of the incoming traffic that the EC2 security group will allow. Add an inbound rule to the security group with the following parameters: Add two more inbound rules to access the EC2 with SSH and the Flask application. Use a command from aws ec2 to add these three inbound rules to the security group. Type the following command to add an inbound rule for Flask: Type the following command to add an inbound rule for SSH: Step 6: Create a New Key Pair In AWS, a key pair is a combination of public and private keys. The public key is encrypts data, while the private key decrypts data. These are the security credentials used to connect to the EC2 instance. Amazon EC2 instance stores the public key, and the user has their private key to connect to the instance. Let’s create a new key pair in the account using the aws ec2 command. You need to pass a key name to the command to create a key pair. The name of the key pair must be unique. Step 7: List all Key Pairs Let’s verify the creation of key pairs by listing all available key pairs in the account using the aws ec2 command. Step 8: Run an EC2 Instance Let’s launch a template using Amazon Machine Image (AMI) in this task. You can only launch instances of those AMIs for which you have permission. To launch an instance, get the AMI ID of the required operating system. Add the AMI ID in the AMI_ID variable in the environment.sh file. Use the following command to export the value: source environment.sh After getting Amazon Ubuntu’s AMI ID, use the aws ec2 command and pass these parameters to launch the instance: Type the following command to run an instance: After running the instance, copy the InstanceId from the output and assign it to the INSTANCE_ID variable in the environment.sh file. Use the following command to export the values: Step 9: Check the state of the EC2 instance After running an instance, we can check the state of the instance using the aws ec2 command. This will accept the InstanceId as the argument and output the complete details of the instance. Check the PublicIpAddress and State.name of the instance. If the state is not running, wait for a while and list the attributes of the instance again. After two minutes, notice the status of the instance again. It should be in running now. Note: Copy the PublicIpAddress of the instance and place it in the environment.sh file in the PUBLIC_IP. Next, export the variable using the following command: source environment.sh Step 10: Copy the Data from a Local Machine to EC2 machine: To deploy the Flask application over the EC2 instance, upload the application from a local machine to the EC2 instance. In this task, perform the following steps: GitHUB: https://github.com/skillupwithsachin/aws_rds_project_skill_up_with_sachin.git https://github.com/skillupwithsachin/aws_rds_project_skill_up_with_sachin# Type the following command to change the access permission of the file: Use the following command in the terminal to upload the zipped Flask application: After these commands,
Breaking Down Kubernetes Interviews – One Pod at a Time!
Introduction: Why Container Orchestration? Problem Statement:As microservices-based applications scale, managing containers across multiple environments manually becomes inefficient and error-prone. Solution:Container orchestration automates the deployment, scaling, networking, and lifecycle management of containers. Key Benefits of Kubernetes Orchestration: Virtual Machines vs Containers vs Kubernetes Virtual Machines Docker Containers Kubernetes Hardware-level virtualization OS-level virtualization Container orchestration Heavyweight Lightweight and fast Automates container ops Boot time: Minutes Boot time: Seconds Self-healing, scalable Key Insight:Containers solve the portability problem. Kubernetes solves the scalability and reliability problem of containers in production. Storage in Kubernetes (Dynamic & CSI) Problem Statement:How do we abstract and dynamically provision storage in Kubernetes without being tied to a specific cloud or on-premise provider? Solution: Flow:App → PVC → StorageClass + CSI → PV Reference: https://kubernetes.io/blog/2019/01/15/container-storage-interface-ga/ Kubernetes Architecture Control Plane (Master Node): Together, these components form the master control plane, which acts as the brain and command center of the Kubernetes cluster. Worker Node (Data Plane): Worker nodes, also known as worker machines or worker servers, are the heart of a Kubernetes cluster. They are responsible for running containers and executing the actual workloads of your applications. Architecture Flow Example: Triggers → API Server → Scheduler → Etcd → Node → Kubelet → Container Runtime Instruction Flow (From YAML to Running Pod) PODS Pods are fundamental building blocks in Kubernetes that group one or more containers together and provide a shared environment for them to run within the same network and storage context. Allows you to colocate containers that need to work closely together within the same network namespace. They can communicate using localhost and share the same IP address and port space. Containers within a Pod share the same storage volumes, which allows them to easily exchange data and files. Volumes are attached to the Pod and can be used by any of the containers within it. Kubernetes schedules Pods as the smallest deployable unit. If you want to scale or manage your application, you work with Pod replicas, not individual containers. A Pod can include init containers, which are containers that run before the main application containers. Kubernetes High Availability & Failure Scenarios Component Failure Impact Recovery API Server Cluster becomes unmanageable Restart or HA deployment Etcd State loss, no new scheduling Restore from backup, use HA etcd Scheduler No new pods scheduled Restart scheduler Controller Manager Auto-scaling and replication broken Restart or recover HA Kubelet Node disconnected, unmonitored pods Restart kubelet or reboot node Kube-Proxy Service communication broken Restart kube-proxy CoreDNS DNS lookup failure for services Restart CoreDNS Reference: https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/high-availability/ Kubernetes Services In Kubernetes, Services are a fundamental concept that enables communication and load balancing between different sets of Pods, making your applications easily discover able and resilient. Why Do We Need Kubernetes Services? Types of Services: Cluster IP: The default service type. It provides internal access within the cluster. NodePort: Opens a port (30000–32767) on each node, allowing external access to services. Make sure to configure security groups accordingly. LoadBalancer: Distributes incoming traffic across multiple pods, ensuring high availability and better performance. Ingress: HTTP routing with host/path rules Network Policies (Ingress & Egress) Problem Statement:How do we secure communication between microservices in a Kubernetes cluster? Use Case: 3-Tier Microservice Architecture Ingress Policy: Egress Policy: Secrets & ConfigMaps Resource Purpose Security Level Config Map Store non-sensitive config Plain text in etcd Secret Store Sensitive Data Base-64 encoded, more secure Practical Use Case: Kubernetes CI/CD Integration (Brief Outline) Problem Statement: How do we automate builds, tests, and deployments on Kubernetes? Approach: How to handle CrashLoopBackOff Error ? Error Message: kubectl get podsNAME READY STATUS RESTARTS AGEmy-app-5c78f8d6f5-xyz12 0/1 CrashLoopBackOff 5 3m Cause: Application inside the container is crashing repeatedly.Missing dependencies, incorrect configuration, or resource limitations.Fix: Check logs for error messages: kubectl logs my-app-5c78f8d6f5-xyz12 Describe the pod for more details: kubectl describe pod my-app-5c78f8d6f5-xyz12 Fix application errors or adjust resource limits. How to fix ImagePullBackOff Error ? Error Message: kubectl get podsNAME READY STATUS RESTARTS AGEmy-app-5c78f8d6f5-xyz12 0/1 ImagePullBackOff 0 3m Cause: Fix: kubectl describe pod my-app-5c78f8d6f5-xyz12 containers: – name: my-app image: myregistry.com/my-app:latest kubectl create secret docker-registry regcred \ –docker-server=myregistry.com \ –docker-username=myuser \ –docker-password=mypassword How to fix Pod Stuck in “Pending” State ? Error Message: kubectl get podsNAME READY STATUS RESTARTS AGEmy-app-5c78f8d6f5-xyz12 0/1 Pending 0 5m Cause: Fix: kubectl describe pod my-app-5c78f8d6f5-xyz12 kubectl get nodes kubectl get pvc How to fix Node Not Ready ? Error Message: kubectl get nodesNAME STATUS ROLES AGE VERSIONnode-1 NotReady <none> 50m v1.27.2 Cause: Fix: kubectl describe node node-1 journalctl -u kubelet -n 100 systemctl restart kubelet df -h How to fix Service Not Accessible error? Error Message: curl: (7) Failed to connect to my-service port 80: Connection refused Cause: Fix: kubectl get svc my-service kubectl describe svc my-servic kubectl get pods -o wide How to fix “OOMKilled” (Out of Memory) ? Error Message: kubectl get pod my-app-xyz12 -o jsonpath='{.status.containerStatuses[0].state.terminated.reason}’OOMKilled Cause: Fix: resources: limits: memory: “512Mi” requests: memory: “256Mi” kubectl top pod my-app-xyz12 What do you know about kubeconfig file in Kubernetes ? A file used to configure access to a cluster is called a kubeconfig file. This is the generic way of referring to a configuration file. This doesn’t mean the file name is kubeconfig. K8s components like kubectl, kubelet, or kube-controller-manager use the kubeconfig file to interact with the K8s API. The default location of the kubeconfig file is ~/.kube/config. There are other ways to specify the kubeconfig location, such as the KUBECONFIG environment variable or the kubectl —kubeconfig parameter. The kubeconfig file a YAML file contains groups of clusters, users, and contexts. The clusters section lists all clusters that you already connected. The users section lists all users already used to connect to a cluster. There are some possible keys for the user: The context section links a user and a cluster and can set a default namespace. The context name is arbitrary, but the user and cluster must be predefined in the kubeconfig file. If the namespace doesn’t exist, commands will fail with an error. What are Selectors & Labels in Kubernetes? Services use selectors and labels to identify the Pods they should target.
Understanding printf in Scripting: Usage, Examples, and Alternatives
When it comes to printing output in programming, printf is one of the most commonly used functions, especially in languages like C, Shell scripting, and Java. Understanding its functionality, capabilities, and alternatives can significantly enhance your coding experience. What is printf? The printf function stands for “print formatted” and is used to print formatted output to the console. It provides a powerful way to display text, numbers, and other data types in a customized format. Primarily, it is a standard library function in C but is also available in shell scripting for Unix/Linux environments. Syntax in Shell Scripts: Unlike echo, which simply prints text, printf provides advanced formatting capabilities. Common Usage and Examples in Shell Scripts Basic Printing The simplest use of printf is to display static text: Note: Unlike echo, you must explicitly include \n for a new line. Printing Variables You can use format specifiers to print variable values: Formatting Numbers printf allows precise control over numerical output: Creating Aligned Tables You can use width specifiers to align output: Output: Using %q in Shell printf The %q specifier in printf escapes special characters in a string, making it useful for safe and predictable output, especially when dealing with untrusted input or special characters. Example: Output: This is particularly helpful in scripts where inputs might include spaces, quotes, or other characters requiring escaping. Combining %q with Other Specifiers: Output: Format Specifiers in Shell printf Here are some common placeholders used in shell scripting with printf: Differences Between printf and echo Alternatives to printf in Shell Scripts While printf is versatile, there are alternatives for simpler tasks: 1. “ The echo command is simpler and often sufficient for basic output. 2. “** for Advanced Formatting** awk can be used for complex text processing and formatting. 3. “** for Static Text** For displaying static text files or strings, cat is an option: When to Use printf in Shell Scripts Scenario-Based Interview Questions and Answers 1. How would you use “ to escape special characters in a user input string? Answer: Use the %q format specifier to ensure that special characters are escaped. This will output: Hello,\ \$USER! 2. How can you format a floating-point number to show exactly three decimal places? Answer: Use %.3f in the format specifier. This will output: 3.142 3. How can you create a table with aligned columns using “? Answer: Use width specifiers to align the text. 4. What happens if a format specifier does not match the argument type? Answer: The output may be unpredictable, as printf does not perform type checking. For example: This could cause an error or display an unintended result. 5. How do you print a literal ** character using **? Answer: Use %% in the format string. This will output: Progress: 50% The printf command is a powerful tool in shell scripting, offering advanced formatting capabilities beyond what echo can provide. Its versatility makes it a go-to choice for scripts that require precision and control over the output format. While simpler alternatives exist, understanding and leveraging printf ensures your shell scripts are robust and professional. Experiment with printf in your shell scripts and discover how it can streamline and enhance your output!
Useful Tips & Tricks for Building Resume & Getting Job
Check out this informative video on DevOps by me !! In the video, I have talked about tips and tricks for DevOps, as well as interview preparation and resume building. Additionally, they discuss various DevOps projects to practice. Don’t miss out on this valuable resource! Tips and Tricks for Mastering DevOps Building a Strong DevOps Resume Your resume is your first impression. Make sure it reflects your skills and experience effectively. By following the tips and tricks outlined here, preparing thoroughly for interviews, and building a resume that showcases your expertise, you’ll be well on your way to a successful career in DevOps. Remember, the key to success in DevOps lies in fostering a collaborative culture, embracing automation, and always striving for continuous improvement. Happy learning! #skillupwithsachin #resume #learning #tipsandtricks #resumebuilding #interviewtricks
