Linux Namespaces, Container Runtime, and Kubernetes Interactions
Linux namespaces are fundamental to isolating system resources for processes. Each namespace type isolates a different system resource: The Role of Container Runtime Container runtime, on the other hand, is the software that executes containers and manages container images on a node. Container runtimes provide the necessary interfaces to interact with the Linux kernel namespaces, … Continue reading Linux Namespaces, Container Runtime, and Kubernetes Interactions →
Linux namespaces are fundamental to isolating system resources for processes. Each namespace type isolates a different system resource: PID namespace: Isolates the process ID number space, meaning that processes in different PID namespaces can have the same PID. Network namespace (NET): Provides isolation of network controllers, system resources associated with networking, firewall rules, and routing tables. Mount namespace (MNT): Isolates the set of filesystem mount points seen by a group of processes so that processes in different mount namespaces can have different views of the filesystem hierarchy. Interprocess Communication namespace (IPC): Separates the system resources associated with System V IPC and POSIX message queues. User namespace (USER): Isolates the user and group ID number spaces, meaning that a process’s user and group IDs can be different inside and outside a user namespace. UTS namespace: Allows each container to have its own hostname and domain name, isolating the system identifiers nodename and domainname. The Role of Container Runtime Container runtime, on the other hand, is the software that executes containers and manages container images on a node. Container runtimes provide the necessary interfaces to interact with the Linux kernel namespaces, cgroups, and other components to create containers. In the context of Kubernetes, it uses a container runtime to create and manage containers. Kubernetes supports several container runtimes: Docker, containerd, CRI-O, and any implementation of the Kubernetes CRI (Container Runtime Interface). How Kubernetes Creates Containers Using Namespaces In Kubernetes, a Pod is the smallest and simplest unit that you deploy on a Kubernetes cluster. A Pod represents a single instance of a running process in a cluster and can contain one or more containers. These containers in a Pod share the same network, IPC, and optionally UTS and PID namespaces. It’s essentially a co-located group of applications that share the same context and resources. When a Pod gets scheduled on a worker node, the kubelet on that node will ask the configured container runtime (let’s assume Docker for this example) to start the containers defined in the PodSpec. Docker will then interact with the Linux kernel to create the necessary namespaces and isolate the resources for each container in the pod. YAML to Create Sample Pods Now let’s create a sample Kubernetes pod that hosts a single container. Below is an example of a simple Pod configuration in YAML. apiVersion: v1 kind: Pod metadata: name: mypod spec: containers: - name: mycontainer image: nginx To create this Pod, save the YAML in a file named mypod.yaml and then apply it with kubectl: kubectl apply -f mypod.yaml Inspecting Container Processes Assume that the Pod is scheduled and running. We can inspect the running processes of the “mypod” using a few commands. First, we need to identify the container’s runtime ID. This can be done with the following command: kubectl describe pod mypod The output will be something like: bashCopy code Name: mypod ... Containers: mycontainer: Container ID: docker://f0fb8a8a97ec9a9b6b14116f6391255771c5f1fd2f31d752c29660f5a9d96b1a Image: nginx ... The Container ID is what we are interested in. In this case, it is f0fb8a8a97ec9a9b6b14116f6391255771c5f1fd2f31d752c29660f5a9d96b1a. Now, let’s inspect the running processes in this container using the Docker ‘top’ command: docker top f0fb8a8a97ec9a9b6b14116f6391255771c5f1fd2f31d752c29660f5a9d96b1a You’ll see something like: PID USER TIME COMMAND 5317 root 0:00 nginx: master process nginx -g daemon off; 5370 101 0:00 nginx: worker process This shows us the processes running inside our container. Finally, if you want to check the container’s PID namespace, use the following command: docker inspect --format '{{.State.Pid}}' f0fb8a8a97ec9a9b6b14116f6391255771c5f1fd2f31d752c29660f5a9d96b1a This might return a value like 5315, which corresponds to the PID namespace in which the container’s processes are running. Understanding the relationship between Linux namespaces, container runtime, and Kubernetes’ orchestration can offer invaluable insight into the internal workings of containerized applications, enabling effective management and troubleshooting.
Kubernetes has become a leading platform for orchestrating containers in development, staging, and production environments. With its increased adoption, understanding its configuration files is essential. Two key components in the Kubernetes ecosystem are Kubeconfig files and certificate files. These are primarily used for authentication with the Kubernetes API Server. In this post, we will delve … Continue reading Kubeconfig Files and Cert Files →
Kubernetes has become a leading platform for orchestrating containers in development, staging, and production environments. With its increased adoption, understanding its configuration files is essential. Two key components in the Kubernetes ecosystem are Kubeconfig files and certificate files. These are primarily used for authentication with the Kubernetes API Server. In this post, we will delve deep into what these files are, why they are necessary, and how to use them effectively. What is a Kubeconfig File? A Kubeconfig file is a YAML format file that stores configurations for accessing multiple clusters. It’s the file that kubectl uses to determine the cluster it communicates with and the credentials it needs to do so. By default, it’s located in the ~/.kube/config directory on your system. Each Kubeconfig file can contain multiple ‘context’ entries. A ‘context’ combines a user (as defined in the Kubeconfig) with a cluster, allowing kubectl to target different Kubernetes clusters based on the context. The basic structure of a Kubeconfig file consists of the following: Clusters: Defines the details of the clusters. This includes the name of the cluster and the server URL. Users: Defines the user details used for authentication, such as client certificates, bearer tokens, or username and password. Contexts: A combination of clusters and users, referred to by a unique name. What are Cert Files? Cert files, also known as TLS certificates, are used for authenticating a client (or a user) to the Kubernetes API server. They are one of the types of credentials that can be embedded into the Kubeconfig file. Kubernetes API server supports multiple authentication methods, but the client certificate authentication is widely used because of its robustness and support for automation. In this method, a client certificate (public key) and a private key are generated for a user. The certificate is then signed by the Kubernetes cluster’s Certificate Authority (CA). To validate the client’s identity, the API server verifies if the client’s certificate was signed by the trusted CA. This process is commonly known as Certificate-based authentication. Using Kubeconfig and Cert Files for Authentication Now that we understand what these files are, let’s explore how we can use them. Generate a Certificate and Key Firstly, you need to generate a private key and a certificate signing request (CSR) for the user. You can use openssl to achieve this: openssl req -new -newkey rsa:4096 -nodes -keyout myuser.key -out myuser.csr -subj "/CN=myuser/O=myorganization" In the above command, CN is the user’s name and O is the user’s group. Sign the Certificate using the Kubernetes CA Next, you need to get the CSR signed by the Kubernetes CA. Kubernetes provides a ‘CertificateSigningRequest’ resource to automate this process: kubectl create csr myuser --cert myuser.csr --key myuser.key kubectl certificate approve myuser Once approved, you can download the certificate: kubectl get csr myuser -o jsonpath='{.status.certificate}' | base64 --decode > myuser.crt Embed the Credentials in the Kubeconfig The last step is to embed these credentials into the Kubeconfig file. Here’s how to add a user: kubectl config set-credentials myuser --client-certificate=myuser.crt --client-key=myuser.key And then set the context: kubectl config set-context mycontext --cluster=mycluster --user=myuser Remember to replace ‘mycontext’, ‘mycluster’, and ‘myuser’ with your own values. Once done, you can switch between contexts using the following command: kubectl config use-context mycontext In conclusion, Kubeconfig files and cert files are crucial components when working with Kubernetes, specifically for communication and authentication with the Kubernetes API server. Understanding how to generate, sign, and embed certificates into Kubeconfig will equip you with a significant part of Kubernetes security and authentication. This also plays a pivotal role in automating deployments and maintaining high security in your Kubernetes environment.
Pod Lifecycle, Probes, and Health Checks
In Kubernetes, a Pod stands as the smallest, simplest unit that can be created and managed. Every Pod corresponds to a fraction of a workload operating on your cluster. Grasping the Pod Lifecycle allows us to proficiently control our applications and comprehend the situation when things veer off course. Here’s a high-level look at the … Continue reading Pod Lifecycle, Probes, and Health Checks →
In Kubernetes, a Pod stands as the smallest, simplest unit that can be created and managed. Every Pod corresponds to a fraction of a workload operating on your cluster. Grasping the Pod Lifecycle allows us to proficiently control our applications and comprehend the situation when things veer off course. Here’s a high-level look at the Pod Lifecycle: Pending: The Pod has been acknowledged by the Kubernetes system, but some containers are yet to be readied for running. Running: The Pod has been allocated to a node, all the containers have been created, and at least one container is either running or is in the startup/restart process. Succeeded: All containers within the Pod have exited, and they did so successfully, meaning they won’t restart. Failed: All containers within the Pod have ceased to run. At least one container has terminated due to a failure or system termination. Unknown: The Pod’s state could not be determined. Dissecting Liveness, Readiness, and Startup Probes A Probe in Kubernetes is a diagnostic conducted periodically by the kubelet on a Container. There are three distinct types of probes: liveness, readiness, and startup. Kubernetes employs these probes to determine the Pod’s health and make subsequent decisions based on the Pod’s state. Liveness Probe The liveness probe serves as an indicator of when to restart a container. If a liveness probe encounters a failure, the kubelet terminates the container, which is then subjected to its designated restart policy. If no liveness probe is provided by a Container, the default state is ‘Success’. Here’s an example of a liveness probe in a Kubernetes Pod specification: apiVersion: v1 kind: Pod metadata: name: liveness-pod spec: containers: - name: liveness-container image: k8s.gcr.io/busybox args: - /bin/sh - -c - touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600 livenessProbe: exec: command: - cat - /tmp/healthy initialDelaySeconds: 5 periodSeconds: 5 Readiness Probe Readiness probes signal when a container is prepared to start accepting traffic. A Pod is deemed ready when all its Containers are ready. If a Container doesn’t specify a readiness probe, the default state is ‘Success’. Here’s a simple readiness probe in a Pod spec: apiVersion: v1 kind: Pod metadata: name: readiness-pod spec: containers: - name: readiness-container image: k8s.gcr.io/busybox args: - /bin/sh - -c - sleep 30; touch /tmp/ready; readinessProbe: exec: command: - cat - /tmp/ready initialDelaySeconds: 5 periodSeconds: 5 Startup Probe Startup probes denote whether the application within the container has started. If a startup probe is provided, all other probes are disabled until it succeeds. If it fails, the kubelet terminates the container, which then adheres to its restart policy. Here’s an example of a startup probe: apiVersion: v1 kind: Pod metadata: name: startup-pod spec: containers: - name: startup-container image: k8s.gcr.io/busybox args: - /bin/sh - -c - sleep 30; touch /tmp/started; startupProbe: exec: command: - cat - /tmp/started initialDelaySeconds: 5 periodSeconds: 5 Navigating Health Checks in Kubernetes Health checks in Kubernetes verify that application instances can handle requests. They inform the Kubernetes scheduler about the health of the instances, indicating whether they’re ready to receive requests. Readiness probe: Signifies whether the application is ready to serve requests. If the readiness probe fails, the endpoints controller will remove the container’s IP address from the endpoints of all services. Liveness probe: Shows whether the application is running. If the liveness probe fails, the kubelet will terminate the container, which will be subjected to its restart policy. In conclusion, the understanding of Pod Lifecycle and various probes is key for application management in Kubernetes. It directly influences the high availability of applications, optimal resource usage, and the overall reliability of the system. By defining your liveness, readiness, and startup probes accurately, you can ensure that your applications are functioning as anticipated, and any issues can be detected and resolved promptly.
Understanding Standard Streams: stdin, stdout, and stderr in Linux
Today we’re going to dive into an important topic for anyone using a Linux-based operating system or writing software for such a system: the standard streams – stdin, stdout, and stderr. These three data streams are the primary means of communication between your Linux operating system and the processes it runs. What are Standard Streams? … Continue reading Understanding Standard Streams: stdin, stdout, and stderr in Linux →
Today we’re going to dive into an important topic for anyone using a Linux-based operating system or writing software for such a system: the standard streams – stdin, stdout, and stderr. These three data streams are the primary means of communication between your Linux operating system and the processes it runs. What are Standard Streams? Standard streams are communication channels between a program and its environment. When a Linux process starts, three streams are created: stdin (Standard Input): This is the data stream where a program reads its input. By default, stdin refers to the keyboard input. stdout (Standard Output): This is where a program writes its output data. The default destination is the text terminal that initiated the program. stderr (Standard Error): This is another output stream typically used by programs to output error messages or diagnostics. Just like stdout, the default destination is the text terminal that initiated the program. How Do Standard Streams Work? When you execute a command in your terminal, the system initiates a process to run the command. This process comes with an input channel (stdin) and two output channels (stdout and stderr). These channels function like pipelines, enabling the smooth flow of data. Let’s delve into some practical examples of how these streams are used in Linux: Redirecting stdin: The < operator is used to redirect the input of a command. For example, if you type sort < file.txt, the system will use the contents of file.txt as input to the sort command, as opposed to keyboard input. Redirecting stdout: The > operator serves to redirect the output of a command. For instance, ls > files.txt will write the output of the ls command to a file named files.txt instead of displaying it in the terminal. Redirecting stderr: You can redirect error output of a command using the 2> operator. For example, ls /nonexistent 2> error.txt will direct any error message to the file error.txt. Piping: The | operator is used to feed stdout of one command into the stdin of another. This powerful feature of Unix-like systems allows you to chain commands together. For example, ls | sort will output a sorted list of files in the current directory. These streams (stdin, stdout, stderr) are established when the shell or other process starts and typically last for the entire lifespan of the process. In programming, they can be used for interprocess communication, both between different programs and between different parts of the same program. Conclusion Understanding standard streams is vital to mastering command-line operations and scripting in a Linux environment. By using these streams effectively, you can fine-tune your input and output management, and even leverage them for sophisticated inter-process communications. Practice using and redirecting stdin, stdout, and stderr Get a feel for how these streams can empower your command line usage and Linux programming skills. Happy coding!
What is Helm? Helm is a package manager for Kubernetes. A package manager generally aims to provide a single command to install some software. Installing software likely has dependencies – a package manager should be able to resolve those dependencies. A package manager is supposed to abstract some complexities of installing software and making it … Continue reading All Things Helm 3.0 →
What is Helm? Helm is a package manager for Kubernetes. A package manager generally aims to provide a single command to install some software. Installing software likely has dependencies – a package manager should be able to resolve those dependencies. A package manager is supposed to abstract some complexities of installing software and making it easier to update, upgrade and remove the software. What can Helm do for us? Single command installation – helm install Ability to roll back – helm rollback Get state information – helm status Simplified upgrades- helm upgrade Single command uninstall – helm uninstall Want to install helm? Grab the installation script from the official guide, and run it. curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3 chmod 700 get_helm.sh ./get_helm.sh #Confirm that the installation was successful helm version version.BuildInfo{Version:"v3.5.0", GitCommit:"32c22239423b3b4ba6706d450bd044baffdcf9e6", GitTreeState:"clean", GoVersion:"go1.15.6"} Helm Charts Charts is the packaging format for Helm. A chart is nothing but a collection of files with a specific file structure that represents resources within Kubernetes. The name of the folder is the name of the chart. A chart contains all the files for a Kubernetes deployment. Chart Repositories A chart repository is an HTTP server that serves the packaged charts. A packaged chart is a Helm chart that is processed into an archive (tar) by using the following command and a name. helm package chartname Note: A chart repository serves an index.yaml file. This file is the source of truth of packaged charts and where they are. (Example: https://charts.helm.sh/stable/index.yaml) The following section contains some commonly used commands. Add a repository: helm repo add stable https://charts.helm.sh/stable "stable" has been added to your repositories # Note: "stable" after the add argument is the name of the repository which can be changed. List the added repositories if the repositories are already configured: helm reop list NAME URL stable https://charts.helm.sh/stable Search and Download (or fetch) a chart: # search a chart helm search repo apache # download a chart helm fetch stable/spark ls spark-1.0.5.tgz Create a repository index from packaged charts: # We have a number of packaged helm charts in a directory ls sample kafka-manager-2.3.5.tgz spark-1.0.5.tgz # create an index and give the directory as the argument helm repo index ./sample/ # Notice that now we have an index file in the sample repository cat sample/index.yaml apiVersion: v1 entries: kafka-manager: - apiVersion: v1 appVersion: 1.3.3.22 created: "2021-05-23T17:36:13.257252559Z" dependencies: - condition: zookeeper.enabled name: zookeeper repository: https://charts.helm.sh/incubator version: 2.1.4 deprecated: true description: DEPRECATED - A tool for managing Apache Kafka. digest: 2570c27581a7b51e50e5857836aae0c1c86293cd68e36836cd7d91006520b84d home: https://github.com/yahoo/kafka-manager icon: https://kafka.apache.org/images/logo.png keywords: - kafka - zookeeper - kafka-manager kubeVersion: ^1.8.0-0 name: kafka-manager sources: - https://github.com/yahoo/kafka-manager urls: - kafka-manager-2.3.5.tgz version: 2.3.5 spark: - apiVersion: v1 appVersion: 1.5.1 created: "2021-05-23T17:36:13.258001094Z" deprecated: true description: DEPRECATED - Fast and general-purpose cluster computing system. digest: e0b4862812f85dcf7b92d71ed90e9f894f1aef35bb76f384fe322ba736cd10fc home: http://spark.apache.org icon: http://spark.apache.org/images/spark-logo-trademark.png name: spark sources: - https://github.com/kubernetes/kubernetes/tree/master/examples/spark - https://github.com/apache/spark urls: - spark-1.0.5.tgz version: 1.0.5 Update or remove repositories: helm repo update . helm repo remove name Install, get status or remove a chart: #search for an example chart, wordpress in our case. helm search repo wordpress NAME CHART VERSION APP VERSION DESCRIPTION stable/wordpress 9.0.3 5.3.2 DEPRECATED Web publishing platform for building... #install helm install wordpress stable/wordpress #You'll see the pods appear in your Kubernetes cluster kubectl get pods NAME READY STATUS RESTARTS AGE wordpress-6d8b97544f-jt7pc 0/1 Pending 0 60s wordpress-mariadb-0 0/1 Pending 0 59s #See the status of helm chart in helm helm ls NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION wordpress default 1 2021-05-23 18:07:33.476648857 +0000 UTC deployed wordpress-9.0.3 5.3.2 #Delete the installation helm delete wordpress release "wordpress" uninstalled Releases in Helm While a chart contains the structure of objects that you’ll need for a Kubernetes deployment, it will require values and other information for deployment. When we install a chart, the structure will be combined with Values to create objects in a Kubernetes cluster. This combination of chart and values is called a release. Each Chart has a version number and a REVISION number. We saw the following output in the wordpress installation after we installed the wordpress chart. Note the REVISION number. NAME: wordpress LAST DEPLOYED: Sun May 23 18:15:24 2021 NAMESPACE: default STATUS: deployed REVISION: 1 NOTES: This Helm chart is deprecated If you look at the Chart.yaml file in the wordpress chart, you’ll notice that the chart version number is 9.0.3. cat Chart.yaml apiVersion: v1 appVersion: 5.3.2 deprecated: true description: DEPRECATED Web publishing platform for building blogs and websites. engine: gotpl home: http://www.wordpress.com/ icon: https://bitnami.com/assets/stacks/wordpress/img/wordpress-stack-220x234.png keywords: - wordpress - cms - blog - http - web - application - php name: wordpress sources: - https://github.com/bitnami/bitnami-docker-wordpress version: 9.0.3 # This was the version that was deployed (You might have to install the wordpress chart again if you're following) helm ls NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION wordpress default 1 2021-05-23 18:15:24.991379171 +0000 UTC deployed wordpress-9.0.3 5.3.2 Let’s change the above installation. We can do that by using helm upgrade command. # Let's say in the example, we want to change the mariadb version. helm upgrade wordpress stable/wordpress --set image.tag='docker.io/bitnami/mariadb:10.3.21-debian-10-r27' # check the status of wordpress installation helm status wordpress NAME: wordpress LAST DEPLOYED: Sun May 23 18:27:00 2021 NAMESPACE: default STATUS: deployed REVISION: 2 NOTES: This Helm chart is deprecated #Note that the REVISION is changed to 2. Note that the Chart.yaml file which is fetched locally remains unchanged by an upgrade command. If you want to change the chart permanently, you need to change the value inside the chart file. Chart Structure The following folder structure shows a structure of a chart and how files are laid out. wordpress/ Chart.yaml # A YAML file containing information about the chart LICENSE # OPTIONAL: A plain text file containing the license for the chart README.md # OPTIONAL: A human-readable README file values.yaml # The default configuration values for this chart values.schema.json # OPTIONAL: A JSON Schema for imposing a structure on the values.yaml file charts/ # A directory containing any charts upon which this chart depends. crds/ # Custom Resource Definitions templates/ # A directory of templates that, when combined with values, # will generate valid Kubernetes manifest files. templates/NOTES.txt # OPTIONAL: A plain text file containing short usage notes If you want to create a demo chart, you can do that with a helm create command. Also, if you do a dry run on the newly created helm chart, helm will output the yaml file # Create a demo chart helm create demo # You can do a dry run to check how helm converts the chart to a Kubernetes manifest file helm install demo ./demo --dry-run NAME: demo LAST DEPLOYED: Sun May 23 20:45:34 2021 NAMESPACE: default STATUS: pending-install REVISION: 1 HOOKS: --- # Source: demo/templates/tests/test-connection.yaml apiVersion: v1 kind: Pod metadata: name: "demo-test-connection" labels: helm.sh/chart: demo-0.1.0 app.kubernetes.io/name: demo app.kubernetes.io/instance: demo app.kubernetes.io/version: "1.16.0" app.kubernetes.io/managed-by: Helm annotations: "helm.sh/hook": test spec: containers: - name: wget image: busybox command: ['wget'] args: ['demo:80'] restartPolicy: Never MANIFEST: --- # Source: demo/templates/serviceaccount.yaml apiVersion: v1 kind: ServiceAccount metadata: name: demo labels: helm.sh/chart: demo-0.1.0 app.kubernetes.io/name: demo app.kubernetes.io/instance: demo app.kubernetes.io/version: "1.16.0" app.kubernetes.io/managed-by: Helm --- # Source: demo/templates/service.yaml apiVersion: v1 kind: Service metadata: name: demo labels: helm.sh/chart: demo-0.1.0 app.kubernetes.io/name: demo app.kubernetes.io/instance: demo app.kubernetes.io/version: "1.16.0" app.kubernetes.io/managed-by: Helm spec: type: ClusterIP ports: - port: 80 targetPort: http protocol: TCP name: http selector: app.kubernetes.io/name: demo app.kubernetes.io/instance: demo --- # Source: demo/templates/deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: demo labels: helm.sh/chart: demo-0.1.0 app.kubernetes.io/name: demo app.kubernetes.io/instance: demo app.kubernetes.io/version: "1.16.0" app.kubernetes.io/managed-by: Helm spec: replicas: 1 selector: matchLabels: app.kubernetes.io/name: demo app.kubernetes.io/instance: demo template: metadata: labels: app.kubernetes.io/name: demo app.kubernetes.io/instance: demo spec: serviceAccountName: demo securityContext: {} containers: - name: demo securityContext: {} image: "nginx:1.16.0" imagePullPolicy: IfNotPresent ports: - name: http containerPort: 80 protocol: TCP livenessProbe: httpGet: path: / port: http readinessProbe: httpGet: path: / port: http resources: {} NOTES: 1. Get the application URL by running these commands: export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=demo,app.kubernetes.io/instance=demo" -o jsonpath="{.items[0].metadata.name}") export CONTAINER_PORT=$(kubectl get pod --namespace default $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}") echo "Visit http://127.0.0.1:8080 to use your application" kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT Read Values from an Existing Chart If you wanted to read just values from an existing chart, you can also use helm show values command. It will show values and strip out other information from a chart which makes it (a little bit) easier to read the values. helm show values demo # Default values for demo. # This is a YAML-formatted file. # Declare variables to be passed into your templates. replicaCount: 1 image: repository: nginx pullPolicy: IfNotPresent # Overrides the image tag whose default is the chart appVersion. tag: "" imagePullSecrets: [] nameOverride: "" fullnameOverride: "" serviceAccount: # Specifies whether a service account should be created create: true # Annotations to add to the service account annotations: {} # The name of the service account to use. # If not set and create is true, a name is generated using the fullname template name: "" podAnnotations: {} podSecurityContext: {} # fsGroup: 2000 securityContext: {} # capabilities: # drop: # - ALL # readOnlyRootFilesystem: true # runAsNonRoot: true # runAsUser: 1000 service: type: ClusterIP port: 80 ingress: enabled: false className: "" annotations: {} # kubernetes.io/ingress.class: nginx # kubernetes.io/tls-acme: "true" hosts: - host: chart-example.local paths: - path: / pathType: ImplementationSpecific tls: [] # - secretName: chart-example-tls # hosts: # - chart-example.local resources: {} # We usually recommend not to specify default resources and to leave this as a conscious # choice for the user. This also increases chances charts run on environments with little # resources, such as Minikube. If you do want to specify resources, uncomment the following # lines, adjust them as necessary, and remove the curly braces after 'resources:'. # limits: # cpu: 100m # memory: 128Mi # requests: # cpu: 100m # memory: 128Mi autoscaling: enabled: false minReplicas: 1 maxReplicas: 100 targetCPUUtilizationPercentage: 80 # targetMemoryUtilizationPercentage: 80 nodeSelector: {} tolerations: [] affinity: {} Chart Modifications There are three main ways to modify a helm charts. Inline modification using set If you want to just test a few values, you can use –set option with the install command. This will override the values from values.yaml file. This method is usually used in a proof of concept type environment. If you dry run the demo chart that we created earlier (helm install demo demo –dry-run), by default it uses nginx:1.16.0 image. <img width="854" height="223" data-attachment-id="485" data-permalink="https://hardik.info/image-2-2/" data-orig-file="https://hardik.info/wp-content/uploads/2021/05/image-2.png" data-orig-size="854,223" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="image-2" data-image-description="" data-image-caption="" data-large-file="https://hardik.info/wp-content/uploads/2021/05/image-2.png?w=854" src="https://hardik.info/wp-content/uploads/2021/05/image-2.png?w=854" alt="" class="wp-image-485" srcset="https://hardik.info/wp-content/uploads/2021/05/image-2.png 854w, https://hardik.info/wp-content/uploads/2021/05/image-2.png?w=150 150w, https://hardik.info/wp-content/uploads/2021/05/image-2.png?w=300 300w, https://hardik.info/wp-content/uploads/2021/05/image-2.png?w=768 768w" sizes="(max-width: 854px) 100vw, 854px" /> Now if you want to change this image, you can do that by –set option. helm install demo demo --set image.tag=latest --dry-run # Side note: the first "demo" argument is the name of the deployment and the second "demo" argument is the name of our chart folder. # you can do multiple sets together Passing a file You can also pass a file that contains the values that you want to override from the values.yaml. In the same example that we used above, we can create a yaml file with the image tag that we want to override. image: tag: latest # install the demo chart with input of the above yaml file to override values helm install demo demo -f override_values.yaml Change values.yaml Last but not least, you can modify the values.yaml instead of overriding values via passing a file or setting values inline. This effectively creates a new chart. Subcharts A parent chart can have a subchart located in charts directory in the main chart folder. Each chart has its own values file. The parent chart can override the value of a subchart by defining the value in its own (parent’s) values.yaml file. Continuing our earlier example of wordpress chart, you can see that wordpress chart has a subchart of mariadb. ├── charts │ └── mariadb │ ├── Chart.yaml │ ├── files │ │ └── docker-entrypoint-initdb.d │ │ └── README.md │ ├── OWNERS │ ├── README.md │ ├── templates │ │ ├── _helpers.tpl │ │ ├── initialization-configmap.yaml │ │ ├── master-configmap.yaml │ │ ├── master-pdb.yaml │ │ ├── master-statefulset.yaml │ │ ├── master-svc.yaml │ │ ├── NOTES.txt │ │ ├── rolebinding.yaml │ │ ├── role.yaml │ │ ├── secrets.yaml │ │ ├── serviceaccount.yaml │ │ ├── servicemonitor.yaml │ │ ├── slave-configmap.yaml │ │ ├── slave-pdb.yaml │ │ ├── slave-statefulset.yaml │ │ ├── slave-svc.yaml │ │ ├── test-runner.yaml │ │ └── tests.yaml │ ├── values-production.yaml │ ├── values.schema.json │ └── values.yaml ├── Chart.yaml ├── README.md ├── requirements.lock ├── requirements.yaml ├── templates │ ├── deployment.yaml │ ├── externaldb-secrets.yaml │ ├── _helpers.tpl │ ├── ingress.yaml │ ├── NOTES.txt │ ├── pvc.yaml │ ├── secrets.yaml │ ├── servicemonitor.yaml │ ├── svc.yaml │ ├── tests │ │ └── test-mariadb-connection.yaml │ └── tls-secrets.yaml ├── values.schema.json └── values.yaml A child chart cannot access values in the parent folders. If you want a child chart to access value from a parent chart, you need to create a global value. Global values can be accessed by both the parent all subcharts. Global values are defined using the global keyword. In our example, here are some global values in the main values.yaml file. Although commented out, but shows an example. ## # global: # imageRegistry: myRegistryName # imagePullSecrets: # - myRegistryKeySecretName # storageClass: myStorageClass Chart Hooks Chart hooks provide a way to allow chart developers to have control at certain points during a chart’s lifecycle. Hooks inject a job at certain points in chart lifecycles. For example, after we complete a release, we need to load some data in that release. We can use post install hook to run a pod that loads data. These are some of the hook types which are available at the time of the writing. Hook Type Description Pre-Install Executes after templates are rendered but before resource creation in k8s. Post-Install After all resources are loaded in k8s. Pre-Delete Before a resource is deleted. Post-Delete After a resource is deleted. Pre-Upgrade After templates are rendered but before resource updates in k8s. Post-Upgrade After all resources are updated. Pre-Rollback After templates are rendered but before resource roll backs. Post-Rollback After all resources are rolled back. Test Executes when helm test is invoked. Some pointers on hooks A hook is defined in its annotation. It is a template that has some special annotations to convert it into a hook. Hooks are part of a chart. A parent chart cannot override subchart’s hooks. Hooks have weights. If there are multiple hooks, you can provide it a priority by assigning weights to hooks. The hook with the lowest weight executes first. If hooks fail, the release will fail. There’s no limit on number of hooks. We make to make sure that hook jobs deletes or cleans itself because hook objects stand alone. When a release is deleted, hook resources can persist. Hook example Let’s create a demo chart for hooks. We’ll create a file named hook.yaml in templates directory. This example is of a post-install example which means that this pod will run after our release is complete. helm create demo-hook nano demo-hook/templates/hook.yaml #contents of hook.yaml apiVersion: v1 kind: Pod metadata: name: posthook annotations: "helm.sh/hook": "post-install" "helm.sh/hook-weight": "0" #"helm.sh/hook-delete-policy": hook-succeeded spec: containers: - name: democontainer image: busybox imagePullPolicy: IfNotPresent command: ['sh', '-c', 'echo post-install hook Pod is running && sleep 20'] restartPolicy: Never terminationGracePeriodSeconds: 0 helm install demo-hook ./demo-hook/ NAME: demo-hook LAST DEPLOYED: Sat Jun 19 12:39:35 2021 NAMESPACE: default STATUS: deployed REVISION: 1 NOTES: 1. Get the application URL by running these commands: export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=demo-hook,app.kubernetes.io/instance=demo-hook" -o jsonpath="{.items[0].metadata.name}") export CONTAINER_PORT=$(kubectl get pod --namespace default $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}") echo "Visit http://127.0.0.1:8080 to use your application" kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT kubectl get pods NAME READY STATUS RESTARTS AGE demo-hook-795cc96467-xdrwf 1/1 Running 0 30s posthook 0/1 Completed 0 30s Note the output of the last command where we can see the posthook pod as a complete pod. This is because hook-delete-policy line is commented out in the YAML file. If we uncomment that line, the posthook pod in the last command will not be part of the output. Test Hooks A test is a type of hook with a job definition that specifies a container with a command to run. If the command succeeds, then the container should exist with a success code. Helm will create an example test by default in In demo-hook/templates/tests/test-connection.yaml as part of chart creation process. Here are the contents of that test-connection.yaml file. You’ll see in annotations that this is a hook of type test. cat demo-hook/templates/tests/test-connection.yaml apiVersion: v1 kind: Pod metadata: name: "{{ include "demo-hook.fullname" . }}-test-connection" labels: {{- include "demo-hook.labels" . | nindent 4 }} annotations: "helm.sh/hook": test spec: containers: - name: wget image: busybox command: ['wget'] args: ['{{ include "demo-hook.fullname" . }}:{{ .Values.service.port }}'] restartPolicy: Never Now if you do run helm test with the deployment name, it will create a container that runs that test command for us. #run the test suite in the demo-hook release helm test demo-hook NAME: demo-hook LAST DEPLOYED: Sat Jun 19 18:10:58 2021 NAMESPACE: default STATUS: deployed REVISION: 1 TEST SUITE: demo-hook-test-connection Last Started: Sat Jun 19 18:11:55 2021 Last Completed: Sat Jun 19 18:13:56 2021 Phase: Succeeded Library Charts A library chart is a boilerplate helm chart that defines chart primitives and definitions for sharing templates with the goal of reducing repetition. It is important to note that in the templates directory, any file starting with an underscore (_) does not output to a manifest file. By convention, helper templates are placed in files with a naming pattern of _*.yaml or _*.tpl. Here’s an example: # Create a new chart helm create libdemo # remove all templates and the value file rm -rf libdemo/templates/* rm -f libdemo/values.yaml # You'll have two directories and one Chart.yaml file tree libdemo/ libdemo/ ├── charts ├── Chart.yaml └── templates 2 directories, 1 file We’ll create a common ConfigMap which creates a data resource with a defaultvalue config. This means that any chart that inherits this configmap will inherit this default value. nano libdemo/templates/_configmap.yaml {{- define "libdemo.configmap.tpl" -}} apiVersion: v1 kind: ConfigMap metadata: name: {{ .Release.Name | printf "%s-%s" .Chart.Name }} data: defaultvalue: defaultvalue {{- end -}} {{- define "libdemo.configmap" -}} {{- include "libdemo.util.merge" (append . "libdemo.configmap.tpl") -}} {{- end -}} # Note that the above file includes another named template (libdemo.configmap). Next, we’ll change the chart type to library in Chart.yaml. cat libdemo/Chart.yaml apiVersion: v2 name: libdemo description: A Helm chart for Kubernetes type: library version: 0.1.0 appVersion: "1.16.0" # We need to ensure that we add the utility function that we're using to merge two YAML templates. nano libdemo/templates/_util.yaml {{- /* libdemo.util.merge will merge two YAML templates and output the result. This takes an array of three values: - the top context - the template name of the overrides (destination) - the template name of the base (source) */}} {{- define "libdemo.util.merge" -}} {{- $top := first . -}} {{- $overrides := fromYaml (include (index . 1) $top) | default (dict ) -}} {{- $tpl := fromYaml (include (index . 2) $top) | default (dict ) -}} {{- toYaml (merge $overrides $tpl) -}} {{- end -}} # Now if you try to install this chart, you'll get an error confirming that library charts are not installable helm install libdemo libdemo/ Error: library charts are not installable Now if you want to use the above library chart, let’s create another chart, and include our common configmap in the new chart. This configmap will get all standard properties defined in the common configmap and just add myvalue in addition to that. helm create usinglibdemo #add a config map that will inherit configmap definition that we created earlier. nano usinglibdemo/templates/configmap.yaml {{- include "libdemo.configmap" (list . "mychart.configmap") -}} {{- define "mychart.configmap" -}} data: myvalue: "Hello World" {{- end -}} Before we try generating the above configmap, we will add the earlier libdemo chart as a dependency in usinglibdemo chart. nano usinglibdemo/Chart.yaml apiVersion: v2 name: usinglibdemo description: A Helm chart for Kubernetes dependencies: - name: libdemo version: 0.1.0 repository: file://../libdemo type: application version: 0.1.0 appVersion: "1.16.0" # update dependencies in usinglibdemo helm dependency update usinglibdemo Saving 1 charts Deleting outdated charts Now if you do a dry run, you’ll see your configmap rendered as part of the manifest and it includes the default value we defined in our libraryl. helm install usinglibdemo usinglibdemo/ --debug --dry-run install.go:173: [debug] Original chart version: "" install.go:190: [debug] CHART PATH: /home/user/usinglibdemo NAME: usinglibdemo LAST DEPLOYED: Sun Jun 20 14:51:02 2021 NAMESPACE: default STATUS: pending-install REVISION: 1 USER-SUPPLIED VALUES: {} COMPUTED VALUES: affinity: {} autoscaling: enabled: false maxReplicas: 100 minReplicas: 1 targetCPUUtilizationPercentage: 80 fullnameOverride: "" image: pullPolicy: IfNotPresent repository: nginx tag: "" imagePullSecrets: [] ingress: annotations: {} enabled: false hosts: - host: chart-example.local paths: [] tls: [] libdemo: global: {} nameOverride: "" nodeSelector: {} podAnnotations: {} podSecurityContext: {} replicaCount: 1 resources: {} securityContext: {} service: port: 80 type: ClusterIP serviceAccount: annotations: {} create: true name: "" tolerations: [] HOOKS: --- # Source: usinglibdemo/templates/tests/test-connection.yaml apiVersion: v1 kind: Pod metadata: name: "usinglibdemo-test-connection" labels: helm.sh/chart: usinglibdemo-0.1.0 app.kubernetes.io/name: usinglibdemo app.kubernetes.io/instance: usinglibdemo app.kubernetes.io/version: "1.16.0" app.kubernetes.io/managed-by: Helm annotations: "helm.sh/hook": test spec: containers: - name: wget image: busybox command: ['wget'] args: ['usinglibdemo:80'] restartPolicy: Never MANIFEST: --- # Source: usinglibdemo/templates/serviceaccount.yaml apiVersion: v1 kind: ServiceAccount metadata: name: usinglibdemo labels: helm.sh/chart: usinglibdemo-0.1.0 app.kubernetes.io/name: usinglibdemo app.kubernetes.io/instance: usinglibdemo app.kubernetes.io/version: "1.16.0" app.kubernetes.io/managed-by: Helm --- # Source: usinglibdemo/templates/configmap.yaml apiVersion: v1 data: defaultvalue: defaultvalue myvalue: Hello World kind: ConfigMap metadata: name: usinglibdemo-usinglibdemo --- # Source: usinglibdemo/templates/service.yaml apiVersion: v1 kind: Service metadata: name: usinglibdemo labels: helm.sh/chart: usinglibdemo-0.1.0 app.kubernetes.io/name: usinglibdemo app.kubernetes.io/instance: usinglibdemo app.kubernetes.io/version: "1.16.0" app.kubernetes.io/managed-by: Helm spec: type: ClusterIP ports: - port: 80 targetPort: http protocol: TCP name: http selector: app.kubernetes.io/name: usinglibdemo app.kubernetes.io/instance: usinglibdemo --- # Source: usinglibdemo/templates/deployment.yaml apiVersion: apps/v1 kind: Deployment metadata: name: usinglibdemo labels: helm.sh/chart: usinglibdemo-0.1.0 app.kubernetes.io/name: usinglibdemo app.kubernetes.io/instance: usinglibdemo app.kubernetes.io/version: "1.16.0" app.kubernetes.io/managed-by: Helm spec: replicas: 1 selector: matchLabels: app.kubernetes.io/name: usinglibdemo app.kubernetes.io/instance: usinglibdemo template: metadata: labels: app.kubernetes.io/name: usinglibdemo app.kubernetes.io/instance: usinglibdemo spec: serviceAccountName: usinglibdemo securityContext: {} containers: - name: usinglibdemo securityContext: {} image: "nginx:1.16.0" imagePullPolicy: IfNotPresent ports: - name: http containerPort: 80 protocol: TCP livenessProbe: httpGet: path: / port: http readinessProbe: httpGet: path: / port: http resources: {} NOTES: 1. Get the application URL by running these commands: export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=usinglibdemo,app.kubernetes.io/instance=usinglibdemo" -o jsonpath="{.items[0].metadata.name}") export CONTAINER_PORT=$(kubectl get pod --namespace default $POD_NAME -o jsonpath="{.spec.containers[0].ports[0].containerPort}") echo "Visit http://127.0.0.1:8080 to use your application" kubectl --namespace default port-forward $POD_NAME 8080:$CONTAINER_PORT Packaging and Validating Charts When we package a chart, we can sign it using a key by which helm will generate a signature of the chart – this will produce a chart in .tgz format. It also generates a provenance file which can be used to confirm that the chart has not been altered after it was signed. Here’s an example of how we can create a gpg key and sign a package. # Create a gpg key gpg --full-generate-key # Follow the prompts in the commandline. I've named the key as testkey gpg --list-keys /home/user/.gnupg/pubring.kbx ---------------------------------------------- pub rsa3072 2021-06-20 [SC] 9DC3BC5B27D3BC09DE6DE58474C435882DDA8D6F uid [ultimate] testkey (nocomments) <testkey@email.email> sub rsa3072 2021-06-20 [E] # Helm doesn't work with the latest .kbx file from gpg. So we need to export the private key in secring.gpg file. gpg --export-secret-keys >~/.gnupg/secring.gpg # Now we can use the testkey to sign our chart. helm package --sign --key testkey --keyring ~/.gnupg/secring.gpg usinglibdemo Successfully packaged chart and saved it to: /home/user/usinglibdemo-0.1.0.tgz # Note that this will give us two files. The packaged file usinglibdemo-0.1.0.tgz and a provenance file usinglibdemo-0.1.0.tgz.prov where the signature is stored. # now you can verify your packaged chart with helm verify usinglibdemo-0.1.0.tgz command
kubectl get pods #get pods from default namespace kubectl get pods -o wide #get pods from default namespace kubectl get nodes #get nodes kubectl get pods --all-namespaces #list all pods kubectl drain node_name --ignore-deamonsets --force #drain a node, cannot delete pods managed by replicationcontroller, replicaset, job, deamonset, statefulset kubectl uncordon node_name # allow pods to run on the node after draining # implement a manifest, create a pod kubectl apply -f pod.yaml # List of supported resources kubectl api-resources # get output in a different format or filter the results kubectl get pods -o json kubectl get pods -o yaml kubectl get pods -o wide --sort-by .spec.nodeName kubectl get pods -n kube-system --selector k8s-app=calico-node # info-gathering kubectl describe pod pod_name # execute commands in pods # -c is the containername kubectl exec my-pod -c busybox -- echo "test output" # delete a pod kubectl delete pod my-pod # describe a deployment kubectl describe deployment my-deployment # get a deamonset Kubectl get deamonset Those are two different approaches of managing resources Imperative Management kubectl create is what we call Imperative Management. On this approach you tell the Kubernetes API what you want to create, replace or delete, not how you want your K8s cluster world to look like. Declarative Management kubectl apply is part of the Declarative Management approach, where changes that you may have applied to a live object (i.e. through scale) are "maintained" even if you apply other changes to the object. # Run kubectl create to see a list of objects that can be created with imperative commands. kubectl create # Create a deployment imperatively. kubectl create deployment my-deployment --image=nginx # Do a dry run to get some sample yaml without creating the object. kubectl create deployment my-deployment --image=nginx --dry-run -o yaml # Save the yaml to a file. kubectl create deployment my-deployment --image=nginx --dry-run -o yaml > deployment.yml # Create the object using the file. kubectl create -f deployment.yml # Scale a deployment and record the command. # We can see the command in kubectl describe in annotations kubectl scale deployment my-deployment --replicas=5 --record --- # upgrade kubeadm kubectl drain <control plane node name> --ignore-daemonsets sudo apt-get update && \ sudo apt-get install -y --allow-change-held-packages kubeadm=1.20.2-00 kubeadm version Drain the control plane node. kubectl drain <control plane node name> --ignore-daemonsets Upgrade kubeadm. sudo apt-get update && \ sudo apt-get install -y --allow-change-held-packages kubeadm=1.20.2-00 kubeadm version Plan the upgrade. sudo kubeadm upgrade plan v1.20.2 Upgrade the control plane components. sudo kubeadm upgrade apply v1.20.2 Upgrade kubelet and kubectl on the control plane node. sudo apt-get update && \ sudo apt-get install -y --allow-change-held-packages kubelet=1.20.2-00 kubectl=1.20.2-00 Restart kubelet. sudo systemctl daemon-reload sudo systemctl restart kubelet Uncordon the control plane node. kubectl uncordon <control plane node name> Verify that the control plane is working. kubectl get nodes Upgrade the worker nodes. Note: In a real-world scenario, you should not perform upgrades on all worker nodes at the same time. Make sure enough nodes are available at any given time to provide uninterrupted service. Run the following on the control plane node to drain worker node 1: kubectl drain <worker 1 node name> --ignore-daemonsets --force Log in to the first worker node, then Upgrade kubeadm. sudo apt-get update && \ sudo apt-get install -y --allow-change-held-packages kubeadm=1.20.2-00 kubeadm version Upgrade the kubelet configuration on the worker node. sudo kubeadm upgrade node Upgrade kubelet and kubectl on the worker node. sudo apt-get update && \ sudo apt-get install -y --allow-change-held-packages kubelet=1.20.2-00 kubectl=1.20.2-00 Restart kubelet. sudo systemctl daemon-reload sudo systemctl restart kubelet From the control plane node, uncordon worker node 1. kubectl uncordon <worker 1 node name> Repeat the upgrade process for worker node 2. From the control plane node, drain worker node 2. kubectl drain <worker 2 node name> --ignore-daemonsets --force On the second worker node, upgrade kubeadm. sudo apt-get update && \ sudo apt-get install -y --allow-change-held-packages kubeadm=1.20.2-00 kubeadm version Perform the upgrade on worker node 2. sudo kubeadm upgrade node sudo apt-get update && \ sudo apt-get install -y --allow-change-held-packages kubelet=1.20.2-00 kubectl=1.20.2-00 sudo systemctl daemon-reload sudo systemctl restart kubelet From the control plane node, uncordon worker node 2. kubectl uncordon <worker 2 node name> Verify that the cluster is upgraded and working. kubectl get nodes Original documentation: Upgrading kubeadm clusters | Kubernetes # Backup etcD etcdctl --endpoitns $ENDPOINT snapshot save file_name etcdctl snapshot restore file_name # service accounts # Create a basic ServiceAccount. vi my-serviceaccount.yml apiVersion: v1 kind: ServiceAccount metadata: name: my-serviceaccount kubectl create -f my-serviceaccount.yml # Create a ServiceAccount with an imperative command. kubectl create sa my-serviceaccount2 -n default # View your ServiceAccount. kubectl get sa # Attach a Role to the ServiceAccount with a RoleBinding. vi sa-pod-reader.yml apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: sa-pod-reader namespace: default subjects: - kind: ServiceAccount name: my-serviceaccount namespace: default roleRef: kind: Role name: pod-reader apiGroup: rbac.authorization.k8s.io # kubectl create -f sa-pod-reader.yml # Get additional information for the ServiceAccount. kubectl describe sa my-serviceaccount # monitoring kubectl top pod kubectl top pod --sort-by cpu kubectl top pod --selector app=test-app kubectl top node # label a node Kubectl label nodes node_name node_lablel=somelabel Static pod = managed directly by a kubelet on a node (kubelet will create a mirror pod for each static API)
What are OAuth 2.0 and OpenID Connect? OAuth 2.0 is an authorization framework developed by IETF defined in RFC 6749. OpenID Connect (AKA OIDC) is an identity layer built on top of OAuth 2.0. OpenID connect specification is built by OpenID Foundation. The Problem Statement If you wanted a service or an application to access … Continue reading OAuth 2.0 and OpenID Connect Explained →
What are OAuth 2.0 and OpenID Connect? OAuth 2.0 is an authorization framework developed by IETF defined in RFC 6749. OpenID Connect (AKA OIDC) is an identity layer built on top of OAuth 2.0. OpenID connect specification is built by OpenID Foundation. The Problem Statement If you wanted a service or an application to access and process some resources that you own on your behalf, you’ll need to find a way to provide that access to that application or that service. For example, if you want a third-party application to read your tweets, you’ll need to give that third-party application the rights to access the Twitter API. This problem is called the access delegation problem. One way to provide that access is to share your Twitter login credentials – which is quite dangerous considering that not only the third-party application can read all tweets, it can also tweet on your behalf and also read your private messages. The Solution With OAuth 2.0 flow, the third-party application which is trying to access your tweets redirects the user to Twitter. Then the user authenticates to Twitter and provides consent that third-party is allowed to read your tweets. Twitter then provides a temporary token to the third-party application, by which the third-party application will get read access to the user’s tweets for a limited time. The subsequent requests from the third-party application send that token along with the request. <img width="1024" height="855" data-attachment-id="394" data-permalink="https://hardik.info/image-2/" data-orig-file="https://hardik.info/wp-content/uploads/2021/04/image-2.png" data-orig-size="1661,1387" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="image-2" data-image-description="" data-image-caption="" data-large-file="https://hardik.info/wp-content/uploads/2021/04/image-2.png?w=1024" src="https://hardik.info/wp-content/uploads/2021/04/image-2.png?w=1024" alt="" class="wp-image-394" srcset="https://hardik.info/wp-content/uploads/2021/04/image-2.png?w=1024 1024w, https://hardik.info/wp-content/uploads/2021/04/image-2.png?w=150 150w, https://hardik.info/wp-content/uploads/2021/04/image-2.png?w=300 300w, https://hardik.info/wp-content/uploads/2021/04/image-2.png?w=768 768w, https://hardik.info/wp-content/uploads/2021/04/image-2.png?w=1440 1440w, https://hardik.info/wp-content/uploads/2021/04/image-2.png 1661w" sizes="(max-width: 1024px) 100vw, 1024px" /> OAuth flow for a third-party application reading tweets. Per OAuth 2.0 terminology, the following are the flow participants. Resource Owner The Twitter use is called the resource owner. The owner should be the one deciding who has what kind of access to the resource. Authorization Server Twitter is the authorization server in our example. Resource Server The Twitter API server in our example is the resource server. Client The application which is tying to access our user’s tweets. Access Token The token issued by Twitter – the authorization server. Scope Access level of the access token is called scope. If the third-party application tries to tweet on behalf of the user, it will fail because that is out of scope. Grant Flow and Grant Type The flow of events is called grant flow and is defined by the grant type – discussed below. OAuth Participants Grant Types Depending on how a client application works, the client application selects a grant flow to get an access token from the authorization server. The OAuth 2.0 specification has five grant types. Each type specifies the steps for getting an access token. Client Credentials This flow is suitable for authentication between two applications or services with no end-user. Resource Owner Password This flow is suitable in cases where the resource owner has a trust relationship with the client, such as the device OS. Authorization Code Suitable for applications with an end-user. Implicit Not suitable for any cases, dangerous! Discussed below. Refresh Token Suitable when renewing expired access tokens. Grant Type and Use-Case Note that OAuth 2.0 framework doesn’t limit you to the above five grant types, you can add grant types as needed. For example, SAML profile for SSO applications or JWT profile for SSO using OpenID Connect. The following section describes the main five grant types. Client Credentials Grant Type In this grant type, only two participants are involved. In this type, the resource owner is not separate, the client application is itself is the resource owner. Each client has its own credentials, a client ID, and a client secret. In our example, the application which is trying to read tweets from the Twitter API server is the client. The client is responsible to store and protect the client’s secret. The client application has to send its client ID and client secret to the authorization server to get an access token. The authorization server is responsible to validate the client ID and the secret combination. Request POST /oauth2/token HTTP/1.1 Host: 192.168.0.114:9443 Authorization: Basic cVFCcnFlNTVZb1dvMElrTkdQaUFMc29hS1BZYTpDMGxlQW9KTlRsRThhYjVjeV9vMHduYkZtTEVh User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate Connection: close Upgrade-Insecure-Requests: 1 Content-Type: application/x-www-form-urlencoded Content-Length: 29 grant_type=client_credentials Response { "access_token":"da96597c-336d-4c54-ab26-ed29da4c0174", "token_type":"bearer", "expires_in":1530 } The above flow is suitable for applications that do not have to worry about end-users. The client application accesses an API by itself, not on behalf of anyone else. That’s the reason why this grant type is used when one system or service is authenticating to another system or service. Resource Owner Grant Type Resource owner grant type builds on top of the client credential grant type and adds support for resource owner authentication with username and password. In this flow, the client application prompts the end-user to provide a username and a password. The client application uses those credentials to request the authorization server for a token. Note the below request and response flow. The request by the client application to the authorization server. Request POST /oauth2/token HTTP/1.1 Host: 192.168.0.114:9443 Authorization: Basic cVFCcnFlNTVZb1dvMElrTkdQaUFMc29hS1BZYTpDMGxlQW9KTlRsRThhYjVjeV9vMHduYkZtTEVh User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate Connection: close Upgrade-Insecure-Requests: 1 Content-Type: application/x-www-form-urlencoded Content-Length: 62 grant_type=password&username=admin&password=admin&scope=openid Response { "access_token":"841e262c-9bcc-35eb-97b2-f3c1f8192b42", "refresh_token":"e28c8b38-e5fa-34e7-8861-b6080c545347", "scope":"openid", "id_token":"[binary].[binary].YxXUWApeWlc0wmUMiSb0R4_crAcAZ6Plw8LGw9s4KDdg3RtjY0chgSfYvw4nearK2hvZjkOlfxQ-DzkHQoD0e2jFiXJsoM4lJTwXYl1PEIiZ5DP6IUvl3k5SaRxMYyLuy1NmzfG_CGX58MvndxnD7aU0nIZwTzuhsVWjm7V9s7H-UJH463TsGfHV4A12wqLI3es-TV-8i4x3DMgEFu-14ycFlBGlSuPrKh_jVq98Xrlt3gXC6Mifm_MD0exzESr3kHX45Yc4whZjHZFww_4nzUoPvRX-38MQefPVH3gPicnwT73hjYMkaYeFECxn9UebIP2UhohyWGnPveumJMe9Tg", "token_type":"Bearer", "expires_in":1542 } In this grant type flow, it is important to note that the authorization server will validate both the client ID and the client secret and user’s credentials. The authorization server will issue the token only if all four fields are valid. In this flow, the authorization server also provides a refresh token which can be used to renew the access token before it expires. (Note: We didn’t get a refresh token in the client credentials grant type). This mode of delegating access is called access delegation with credential sharing. Note that this type of grant flow is considered legacy. Also, the client application is responsible to process the store credentials from the users correctly – ideally using it only to get the access token and then discard the credentials. Refresh Token Grant Type The refresh token grant type is used to renew an existing token before the access token is expired – so that the user of the application doesn’t have to enter credentials again. With this grant type, the application should receive a new access token and a new refresh token. Request POST /oauth2/token HTTP/1.1 Host: 192.168.0.114:9443 Authorization: Basic eDJmSWR1Y2FuRVF0RjU5Njg3VnhMU0dqQlJrYTpjREpkcHZ6NDhtbEZFVnFJTlRUV2M2UjViMmdh User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate Connection: close Upgrade-Insecure-Requests: 1 Content-Type: application/x-www-form-urlencoded Content-Length: 75 grant_type=refresh_token&refresh_token=e99f94aa-ba5b-3a42-9281-6eea5db98ced Response { "access_token":"18b9048d-2a9d-352a-82c2-f43abbb96d57", "refresh_token":"4b6cd6e0-ebd0-3675-8e57-dcdd42cd6a9a", "scope":"openid", "id_token":"[binary].[binary].ZwjZ0z_RZkYox8hgFtYaGelPBpHiaz3QCIaIDxCVsfGZviphc9hDoNRYdBQl8pGbW9XVy2Rp-fDByOpZ681hYeAT1DrhLncBd6nzuBAsfLzj3rQQbUxPtx17cw4D8x_-oVOg3m7Vnhqxg4y65iR33pO5elNuqS3JLpM9-zW6rBp4QjX7hPLKycJ60wU4Qj8PctDGurXkepDaN-my4CQfIv16a-mhqwcnRRYI_bSTRSQtedB5TmMxShNnT-avLAbXZElRG8wtIVXfsxlMqDjbJ-3n97-qhnF7iEMjlAY_H_XLfnRqrAJWsiw8kEMoA-sgU8YUZQjpNJlEmzzWGnlVww", "token_type":"Bearer", "expires_in":3600 } Note that in the request, we’re sending the current refresh token to the authorization server and in the response, the authorization server will send a new access token with a new refresh token. The expiry time of the refresh token is generally longer than the access token, this is intentional to make sure that the application can renew the access token after some time of inactivity. Authorization Code Grant Type Authorization code grant type is used when the client application is a web-browser based application or an application capable of processing HTTP redirects. The client application initiates the request with an authorization code request to the authorization server. The following are the steps associated with the authorization code grant type. The client application initiates the flow by redirecting the user to the authorization server. The authorization server will prompt the user to enter credentials in a browser prompt. The user submits credentials for authentication and the browser posts the credentials to the authorization server. The authorization server redirects the user back to the client application with an authorization code. The client application will use the authorization code to request the authorization server to get an access token. The authorization server sends an access token to the client application. Request GET /oauth2/authorize?client_id=x2fIducanEQtF59687VxLSGjBRka&scope=openid&redirect_uri=localhost:8080/callback&response_type=code HTTP/1.1 Host: 192.168.0.114:9443 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8 Accept-Language: en-US,en;q=0.5 Accept-Encoding: gzip, deflate Connection: close Cookie: JSESSIONID=D51E5AEF8F516B9674526F8CDC4E92EB; requestedURI=../../carbon/admin/index.jsp Upgrade-Insecure-Requests: 1 Response HTTP/1.1 302 X-Frame-Options: DENY X-Content-Type-Options: nosniff X-XSS-Protection: 1; mode=block Set-Cookie: JSESSIONID=730A05AA6C9EA098636761DC456BD13E38CC7B182602C6717D40473782945BA2CC617BE45F57661D494C7123D4F8EEB86ED6C89ED09D049BF0A2185D0C2C59329C92983CDF818407E1516492DE1A007F8F4848199AC438BE92B03C323F0E7186B946A77040DF7BCEE54FBE06333A155D661FE021897069E4E969062BED7D1919; Path=/oauth2; Secure; HttpOnly Location: https://192.168.0.114:9443/authenticationendpoint/login.do?client_id=x2fIducanEQtF59687VxLSGjBRka&commonAuthCallerPath=%2Foauth2%2Fauthorize&forceAuth=false&passiveAuth=false&redirect_uri=localhost%3A8080%2Fcallback&response_type=code&scope=openid&tenantDomain=carbon.super&sessionDataKey=de622420-0f8a-44dd-901d-1764ad6307d9&relyingParty=x2fIducanEQtF59687VxLSGjBRka&type=oidc&sp=oauth2&isSaaSApp=false&authenticators=BasicAuthenticator%3ALOCAL Content-Length: 0 Date: Sun, 02 Dec 2020 14:54:51 GMT Connection: close Server: WSO2 Carbon Server Note the Location header, the user is redirected to the authorization server which will give the user a prompt to login or to approve similar to the below screenshot. <img data-attachment-id="423" data-permalink="https://hardik.info/image-4/" data-orig-file="https://hardik.info/wp-content/uploads/2021/05/image.png" data-orig-size="915,738" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="image" data-image-description="" data-image-caption="" data-large-file="https://hardik.info/wp-content/uploads/2021/05/image.png?w=915" src="https://hardik.info/wp-content/uploads/2021/05/image.png?w=915" alt="" class="wp-image-423" width="654" height="527" srcset="https://hardik.info/wp-content/uploads/2021/05/image.png?w=654 654w, https://hardik.info/wp-content/uploads/2021/05/image.png?w=150 150w, https://hardik.info/wp-content/uploads/2021/05/image.png?w=300 300w, https://hardik.info/wp-content/uploads/2021/05/image.png?w=768 768w, https://hardik.info/wp-content/uploads/2021/05/image.png 915w" sizes="(max-width: 654px) 100vw, 654px" /> Once the user approves the prompt or enters the credentials, the authorization server will validate the credentials and if successful, the authorization server will redirect the user back to the client with an authorization code. With a redirect like this: https://localhost:8080/login?code=e840ff23-41b9-4ebb-add1-a2340f051bd7 The application then uses the above code to the authorization server for swapping it for an access token. A word of caution: This redirect from the authorization server to the client application happens via a URI which means that it will be logged in browser history and server logs. In a man in the middle scenario, an attacker can also use the same code to get an access token. That’s the reason why it is important to make sure that the flow occurs over an encrypted channel (HTTPS) and the authorization server invalidates the code as soon as it has been used to retrieve an access token. If the code is used more than once, the authorization server should revoke all tokens previously issued with the same code. Implicit Grant Type The implicit grant type is similar to the authorization gran type except for the intermediate step of getting an authorization code before getting an access token. Instead of redirecting the user to the client application with an authorization code, the authorization server redirects the user to the client application with an access token. Instead of redirecting the user with a code, the redirect in this grant type will work something like this: https://localhost:8080/login?access_token=e840ff23-41b9-4ebb-add1-a2340f051bd7&expires_in=3599 Again, because the access token is passed in the URL, the access token will be saved in browser history and server logs. This grant type is not recommended. More on Tokens & Scopes Scope for a token is the purpose of the token. A token can be associated with one or more scopes (more than one purpose). When a client application requests a token to the authorization server, the request contains the list of scopes. Although this doesn’t mean that the authorization server is required to grant a token for all requested scopes, the authorization server will grant a token for the scopes for which the user has given permissions (when prompted). There are two types of tokens. Reference tokens Self-contained tokens A reference token is a token that can only be understood and validated by the issuer of that token – the authorization server. This means that the resource server will have to talk to the authorization server continuously to validate the token. A self-contained token, however, can be validated without talking to the authorization server. This token is generally in the form of a signed JWT (RFC 7523). The resource server will need to pre-establish trust to verify the signatures of the JWT based self-contained tokens. OpenID Connect OpenID Connect (AKA OIDC) is an identity layer built on top of OAuth 2.0. OpenID Connect is based on a token called ID token. An ID token is also a JWT token (Similar to a JWT profile access token) that contains additional information about the user such as the email address or the username of the user. Whenever the authorization server sends an access token to a client application, the authorization server also sends an ID token with the access token in OpenID connect. The following is an example payload of an ID token from OpenID Connect Specification. { "iss":"http://demo.demo.com", "sub":"example@example.com", "aud":"s6BhdRkqt3", "nonce":"n-0S6_WzA2Mj", "exp":1311281970, "iat":1311280970, "auth_time":1311280969, "acr":"urn:mace:incommon:iap:silver", } Attribute details of the example payload: iss The identifier of the authorization server. sub The user information for which this ID token was generated. aud Information about the audience of the token who are supposed to use the ID token. This must contain the client ID of the client application. iat The time at which the ID token was issued exp The time at which the ID token will expire. Information about attributes of an ID token payload. The ID token is sent by an authorization server whenever OpenID is as the scope of the access token request from a client to the authorization server. If you check the example request and response in the above sections for the resource owner or the refresh token grant type, you’ll see that the responses contain id_token along with access_token (see the below screenshot). <img loading="lazy" width="1024" height="313" data-attachment-id="439" data-permalink="https://hardik.info/image-1-2/" data-orig-file="https://hardik.info/wp-content/uploads/2021/05/image-1.png" data-orig-size="1479,453" data-comments-opened="1" data-image-meta="{"aperture":"0","credit":"","camera":"","caption":"","created_timestamp":"0","copyright":"","focal_length":"0","iso":"0","shutter_speed":"0","title":"","orientation":"0"}" data-image-title="image-1" data-image-description="" data-image-caption="" data-large-file="https://hardik.info/wp-content/uploads/2021/05/image-1.png?w=1024" src="https://hardik.info/wp-content/uploads/2021/05/image-1.png?w=1024" alt="" class="wp-image-439" srcset="https://hardik.info/wp-content/uploads/2021/05/image-1.png?w=1024 1024w, https://hardik.info/wp-content/uploads/2021/05/image-1.png?w=150 150w, https://hardik.info/wp-content/uploads/2021/05/image-1.png?w=300 300w, https://hardik.info/wp-content/uploads/2021/05/image-1.png?w=768 768w, https://hardik.info/wp-content/uploads/2021/05/image-1.png?w=1440 1440w, https://hardik.info/wp-content/uploads/2021/05/image-1.png 1479w" sizes="(max-width: 1024px) 100vw, 1024px" /> References Demo server used in the request responses: Identity Server – On-Premise and in the Cloud (wso2.com)
What is Terraform? Terraform is an open-source IoC (infrastructure as code) software by HashiCorp. It allows us to define and provision infrastructure using a high-level configuration language. For example, the following piece of code will create a server on AWS. To deploy it, we can just run one command: Great right? Here are some Terraform … Continue reading Terraform quickstart →
What is Terraform? Terraform is an open-source IoC (infrastructure as code) software by HashiCorp. It allows us to define and provision infrastructure using a high-level configuration language. For example, the following piece of code will create a server on AWS. provider "aws" { region = "us-east-1" } resource "aws_instance" "example" { ami = "ami-12345678" instance_type = "t2.micro" } To deploy it, we can just run one command: > terraform apply Great right? Here are some Terraform quick-basics: We interact with Terraform with its CLI. Once we have created our .tf files (configuration files) in our Terraform project, we first need to run init command to initialize the directory as a starting point of our project. > terraform init The above command will download the provider and take all the actions required for project initialization. Now Let’s say we want to check what changes will be made in our environment before applying it to our environment. We can run the following command for that. > terraform plan After checking the configuration that will be applied, if we want to proceed with deployment, we can run the apply command. > terraform apply The above command will deploy our configuration. Secure VPC design using Terraform To deploy any kind of Infrastructure in AWS environment, first we need to create a VPC in AWS. We don’t want to use default VPC provided by AWS as the default VPC is a VPC with public subnets only where each EC2 instance will automatically receive a public IP Address. The following Terraform source creates a VPC with the following attributes: – Has public and private subnets – Has internet gateway – EC2 instances in private subnets do not get public IP addresses – Two route tables for public and private subnets – Association of public route table to public subnets – Association of private route table to private subnets – NAT gateway in case EC2 instances in private subnets needs to connect to the internet (For example, fetching updates) Create a main.tf file with the following configuration: provider "aws" { region = "us-west-2" shared_credentials_file = "/home/user/.aws/credentials" profile = "default" } resource "aws_vpc" "aws_vpc_terraform" { cidr_block = "10.0.0.0/16" tags { Name = "aws_vpc_terraform" } } resource "aws_internet_gateway" "aws_igw" { vpc_id = "${aws_vpc.aws_vpc_terraform.id}" tags { Name = "aws_igw" } } resource "aws_subnet" "aws_public_1" { vpc_id = "${aws_vpc.aws_vpc_terraform.id}" cidr_block = "10.0.1.0/24" map_public_ip_on_launch = true availability_zone = "us-west-2a" tags = { Name = "aws_public_1" } } resource "aws_subnet" "aws_public_3" { vpc_id = "${aws_vpc.aws_vpc_terraform.id}" cidr_block = "10.0.5.0/24" map_public_ip_on_launch = true availability_zone = "us-west-2c" tags = { Name = "aws_public_3" } } resource "aws_subnet" "aws_public_2" { vpc_id = "${aws_vpc.aws_vpc_terraform.id}" cidr_block = "10.0.3.0/24" map_public_ip_on_launch = true availability_zone = "us-west-2b" tags = { Name = "aws_public_2" } } resource "aws_subnet" "aws_private_1" { vpc_id = "${aws_vpc.aws_vpc_terraform.id}" cidr_block = "10.0.2.0/24" availability_zone = "us-west-2a" tags = { Name = "aws_private_1" } } resource "aws_subnet" "aws_private_2" { vpc_id = "${aws_vpc.aws_vpc_terraform.id}" cidr_block = "10.0.4.0/24" availability_zone = "us-west-2b" tags = { Name = "aws_private_2" } } resource "aws_subnet" "aws_private_3" { vpc_id = "${aws_vpc.aws_vpc_terraform.id}" cidr_block = "10.0.6.0/24" availability_zone = "us-west-2c" tags = { Name = "aws_private_3" } } resource "aws_eip" "aws_eip_nat" { vpc = true depends_on = ["aws_internet_gateway.aws_igw"] } resource "aws_nat_gateway" "aws_nat_gateway" { allocation_id = "${aws_eip.aws_eip_nat.id}" subnet_id = "${aws_subnet.aws_public_3.id}" depends_on = ["aws_internet_gateway.aws_igw"] } resource "aws_route_table" "aws_private_route_table" { vpc_id = "${aws_vpc.aws_vpc_terraform.id}" tags { Name = "aws-3 private route table" Owner = "aws-3" } } resource "aws_route" "private_route" { route_table_id = "${aws_route_table.aws_private_route_table.id}" destination_cidr_block = "0.0.0.0/0" nat_gateway_id = "${aws_nat_gateway.aws_nat_gateway.id}" } resource "aws_route_table" "aws_public_route_table" { vpc_id = "${aws_vpc.aws_vpc_terraform.id}" tags { Name = "aws-3 public route table" } } resource "aws_route" "public_route" { route_table_id = "${aws_route_table.aws_public_route_table.id}" destination_cidr_block = "0.0.0.0/0" gateway_id = "${aws_internet_gateway.aws_igw.id}" } resource "aws_route_table_association" "public_subnet_us_west_2a_association" { subnet_id = "${aws_subnet.aws_public_1.id}" route_table_id = "${aws_route_table.aws_public_route_table.id}" } resource "aws_route_table_association" "public_subnet_us_west_2b_association" { subnet_id = "${aws_subnet.aws_public_2.id}" route_table_id = "${aws_route_table.aws_public_route_table.id}" } resource "aws_route_table_association" "public_subnet_us_west_2c_association" { subnet_id = "${aws_subnet.aws_public_3.id}" route_table_id = "${aws_route_table.aws_public_route_table.id}" } resource "aws_route_table_association" "private_subnet_us_west_2a_association" { subnet_id = "${aws_subnet.aws_private_1.id}" route_table_id = "${aws_route_table.aws_private_route_table.id}" } resource "aws_route_table_association" "private_subnet_us_west_2b_association" { subnet_id = "${aws_subnet.aws_private_2.id}" route_table_id = "${aws_route_table.aws_private_route_table.id}" } resource "aws_route_table_association" "private_subnet_us_west_2c_association" { subnet_id = "${aws_subnet.aws_private_3.id}" route_table_id = "${aws_route_table.aws_private_route_table.id}" } We can now use terraform apply to create this VPC in our environment.
Arithmetic, Logical and Control Instructions in IA32 Assembly
This is not a blog post, per se, but rather an example program that contains basic usage of arithmetic, logical and control instructions. For this blog post, I thought that it is better to describe the example program in the program comments and not in separate paragraphs. We will write a simple program that will … Continue reading Arithmetic, Logical and Control Instructions in IA32 Assembly →
This is not a blog post, per se, but rather an example program that contains basic usage of arithmetic, logical and control instructions. For this blog post, I thought that it is better to describe the example program in the program comments and not in separate paragraphs. We will write a simple program that will have examples of the following: Arithmetic Addition Subtraction Increment/Decrement Multiplication Division Logical AND OR XOR NOT Control Unconditional Jump Conditional Jump Loop ; Author: HRDARJI ; This program was created to understand arithmetic, logical and control ; instructions of assembly language programming. global _start section .text _start: ; ----------- Arithmetic Instructions---------------- ; The below instructions are register based addition. &lt;span data-mce-type="bookmark" id="mce_SELREST_start" data-mce-style="overflow:hidden;line-height:0" style="overflow:hidden;line-height:0" &gt;&lt;/span&gt; mov eax, 0x0 add eax, 0x10 ; memory based addition mov eax, 0x0 add word [var2], 0x1111 clc ; clear carry flag stc ; set carry flag cmc ; complement carry flag ; subtract mov eax, 0x3 sub eax, 0x2 ; increment and decrement instructions inc eax dec eax ; multiplication mov eax, 0x0 mov ebx, 0x0 mov ax, 0x1111 mov bx, 0x0002 mul bx ; divistion mov dx, 0x0 mov ax, 0x1234 mov cx, 0x2 div cx ; ----------- Logical Instructions---------------- ; AND mov al, 0x10 and al, 0x01 ; AND can also be used for memory locations and byte [var1], 0x22 ; OR mov al, 0x10 or al, 0x01 ; XOR mov al, 0x10 xor al, 0x01 ; NOT mov eax, 0xFFFFFFFF not eax ; ----------- Control Instructions---------------- jmp LabelExample2 ; instructions under LabelExample1 will never be executed. LabelExample1: mov eax, 0x11 add eax, 0x22 LabelExample2: ; we are updating eax to 5 because we will print hello world 5 times mov eax, 0x5 PrintString: push eax ; print hello world 5 times mov eax, 0x4 mov ebx, 1 mov ecx, message mov edx, mlen int 0x80 pop eax dec eax jnz PrintString ;-------------- loop instruction example --------------- mov ecx, 0x5 PrintStringLoop: push ecx mov eax, 0x4 mov ebx, 1 mov ecx, message mov edx, mlen int 0x80 pop ecx loop PrintStringLoop ; exit program mov eax, 1 mov ebx, 1 int 0x80 section .data var1 db 0x00 var2 dw 0x0000 var3 dd 0x00000000 message: db "Hello World" mlen equ $-message &lt;span data-mce-type="bookmark" id="mce_SELREST_start" data-mce-style="overflow:hidden;line-height:0" style="overflow:hidden;line-height:0" &gt;&lt;/span&gt;
This post is a part of my attempts at learning Assembly with NASM as our assembler with explanation of a simple, helloworld program. ; helloworld.nasm ; Author: HRDARJI global _start section .text _start: ; printing hello world mov eax, 0x4 mov ebx, 0x1 mov ecx, message mov edx, mlen int 0x80 ; exiting properly mov … Continue reading Hello World in Assembly →
This post is a part of my attempts at learning Assembly with NASM as our assembler with explanation of a simple, helloworld program. ; helloworld.nasm ; Author: HRDARJI global _start section .text _start: ; printing hello world mov eax, 0x4 mov ebx, 0x1 mov ecx, message mov edx, mlen int 0x80 ; exiting properly mov eax, 0x1 mov ebx, 0x5 int 0x80 section .data message: db "Hello World" mlen equ $-message ; in the above line mlen will store length of the message. First, to let the linker know where my program starts, we need to add “_start:” as an identifier to the start of our program (line 8). To add comments in an assembly program, we can use “;” at the start of the line. All the code (instructions) for our program will remain in text section. For our program, the text section declaration syntax can be seen on line 6. All the initialized data will be in our data section. The declaration of data section is on line 24. The message that we want to print, “Hello World” is initialized under label “message”. db is “define bytes”; It will store our string in consecutive memory location (thanks to nasm). To print our string, we will use a system call that can print our string. In IA-32, we will use interrupt 0x80 (line 16)to enter a system call. For more information on interrupt and int 0x80, I will recommend reading this stackoverflow post and wiki page on interrupt vector table. How a system call with interrupt instruction works? When the interrupt handler catches an interrupt, the program will execute as per the current state or values of the registers in the processor. In our case, when the system receives an interrupt on line 18, the value of eax register is 0x4. Which means, that OS will know that write syscall needs to be executed. The mapping of the values to specific syscall can be fonund in “/usr/include/i386-linux-gnu/asm/unistd_32.h” in Ubuntu 21 bit. There are many system calls defined in “/usr/include/i386-linux-gnu/asm/unistd_32.h” (in Ubuntu 32 OS). Example system calls are printing to the screen, make directory, link files, fork exit. Here is a small preview of the file in which system calls are defined. #ifndef _ASM_X86_UNISTD_32_H #define _ASM_X86_UNISTD_32_H 1 #define __NR_restart_syscall 0 #define __NR_exit 1 #define __NR_fork 2 #define __NR_read 3 #define __NR_write 4 #define __NR_open 5 #define __NR_close 6 #define __NR_waitpid 7 #define __NR_creat 8 #define __NR_link 9 #define __NR_unlink 10 #define __NR_execve 11 #define __NR_chdir 12 #define __NR_time 13 #define __NR_mknod 14 #define __NR_chmod 15 #define __NR_lchown 16 #define __NR_break 17 #define __NR_oldstat 18 #define __NR_lseek 19 #define __NR_getpid 20 #define __NR_mount 21 #define __NR_umount 22 #define __NR_setuid 23 #define __NR_getuid 24 #define __NR_stime 25 For our example, we will be using wring system call (line 8 in unistd_32.h). As per Linux manual page for write system call, we need 3 arguments. ssize_t write(int fd, const void *buf, size_t count); fd (file descriptor) is stdout in our case. For any program in Linux, by default, stdin (input) is mapped to fd 0, stdout to fd 1 and stderr fd 2. This will be in ebx register. buf will be pointing to our hello world string. This will be in ecx register. count will be the length of our string. This will be in edx register. Almost every system will have a return value and will be stored in eax register. Write syscall’s return value is the length of the string. Similarly, when the next interrupt is received, the OS will find value 1 in the eax register and will know that this is an exit syscall. The return value is 0x5 as per the argument passed in ebx register. To summarize, we created two interrupts in our program. One for printing a string to stdout and the second one for exiting with return value 0x5.