Terraform in Ubuntu

I have utilized the following steps to install terraform in Ubuntu 16.04

Download Terraform for Linux 64 bit version.

Save the downloaded version in a specific folder. (Example: /home/user/terra)

Within the folder where the downloaded version is saved, use “unzip” command:

unzip terraform

Set the path:

export PATH=$PATH:/home/user/terraform

Reference Link.

Ansible Components

What?

Ansible is a simple IT automation tool. Ansible exists as CLI & GUI. GUI is called the Ansible Tower and Ansible, Inc., which is owned by RedHat, officially supports this.

Controlling Nodes:

The Network infrastructure is managed from these Controlling Nodes. In an Enterprise environment, Controlling Nodes are typically Linux bastion servers.

Managed Nodes:

Managed Nodes are the Network Devices that is being managed by the Controlling Nodes. Managed Nodes are typically of Cisco, Juniper, and Arista make and can be classified as Switches, Routers, Firewalls and Load Balancers based on their function from a Network Engineer’s perspective.

Why?

There are many automation tools like Chef, Puppet, and CFEngine but in my opinion, Ansible is suited for Network Automation for the following reasons:

  1. Ansible does not require an agent to be installed in the Managed Node (Network Device).
  2. Ansible requires Python on the Managed Node and most Network Devices support Python.
  3. Ansible relies on YAML as the descriptive language and Jinja2 for templates.

Among the points mentioned above, most Network Vendors do not support the installation of agents and even if they did support the installation, it would be tough to get the relevant permissions within an organization to install the agents in an Enterprise environment that has different Network Teams managing different aspects of the infrastructure.

Fortunately, most network vendors provide native support for Python and Ansible rely on this to execute automation tasks on the “Managed Nodes”.

As a Network Engineer working in an environment with significant scale (1,000s of Network Devices across multiple datacenters), Ansible has been quite useful in obtaining data and deploying configuration. Ansible seems to have widespread support among the Network Engineers seeking automation to manage at scale and there are resources online that can be leveraged to implement Network Automation Solutions.

Ansible Components

Ansible requires the following components in order to automate Network Infrastructure:

  • Controlling Nodes
  • Managed Nodes
  • Ansible Playbook

As noted earlier, Controlling Nodes are usually Linux Bastion Servers that are used to access the switches/routers and other Network Devices. These Network Devices are referred to as the Managed Nodes. Managed Nodes are stored in the hosts file for Ansible automation.

Ansible Playbook:

Ansible Playbooks are expressed in YAML format and serve as the repository for the various tasks that will be executed on the Managed Nodes (hosts). Playbooks are a collection of tasks that will be run on one or more hosts.

Setting up Ansible:

After installing Ansible, I recommend creating a separate directory from which Ansible is executed. For this process, let’s create a directory named “AnsiblePlay”. Within the “AnsiblePlay” directory, I will have the following files:

  • ansible.cfg
  • hosts

and the following directories ./AnsiblePlay/

  • templates
  • hosts_var

Ansible Configuration File:

An Ansible Playbook utilizes the Ansible Configuration File in order to access resources required for the Ansible Playbook. For example, the configuration file stores location information for the hosts file that contains the Managed Nodes (hosts) on which the playbook is executed.

Ansible Configuration File can exist in the following locations and is utilized by the Ansible playbook in the following order.

* ANSIBLE_CONFIG (an environment variable)
* ansible.cfg (in the current directory)
* .ansible.cfg (in the home directory)
* /etc/ansible/ansible.cfg

I would recommend creating your own Ansible configuration file in the Ansible directory. I use the following:

$ cat ansible.cfg

[defaults]
hostfile = ./hosts
host_key_checking=False
timeout = 5

Inventory File or Hosts File:

Inventory File or Hosts File is a text file that contains the Managed Nodes (Hosts) that will be subjected to automation tasks defined in the playbook.

Inventory File can be static or dynamic. For now, the examples use static inventory files.

This is an inventory/host file example:

$ cat hosts
[ios]
Switch-1
Switch-2
Switch-3
Switch-4

[distro]
Distro1
Distro2

[aggr]
Aggr1
Aggr2

Lists: [ ]

Dictionaries: { }. Dictionary has “Key: Value” pair.

YAML – Anything is a string. Quoting strings is optional most of the times.

FACTS:

FACTS are data about the Managed Nodes. Example: Code Version running on Managed Node.

 TASK:

Ansible playbook contains one or more tasks. A task makes sure that the hosts exist in a specific state. When there are multiple tasks and if any task fails, subsequent tasks will not be executed.

 Idempotent:

Running a task once or multiple times is the same in terms of the final output. For example, a task that involves creating a user in the Managed Node will create the user only once no matter how many times the task is executed.

 

F5 Logs

F5 logs are available under /var/log/ directory.

If you are looking for LTM traffic related logs, look under  /var/log/ltm

If you are looking for GTM traffic related logs, look under /var/log/gtm

GTM has been renamed as DNS from 12.x code version.

The logs are rotated every 24 hours. There will be one ltm file that contains the latest logs and multiple ltm.gz files like: ltm.1.gz, ltm.2.gz etc. The ltm.1.gz files can be opened by using the following command: zcat ltm.1.gz

    (tmos)# run util bash
    ~# cd /var/log
    ~# cat ltm

Other such logs are outlined in this article – K16197

Kubernetes Intro

Kubernetes is Greek for Pilot or Helmsman. It is primarily used as an orchestration tool for containers.

Kubernetes Deployment (KD):

Deployment is responsible for creating and updating instances of application.

Kubernetes Master (KM):

KM schedule the application instances that was created by the KD onto individual nodes in the cluster.

Kubernetes Deployment Controller (KDC):

KDC monitors application instances and replaces an instance if the node hosting the instance goes down.

Kubernetes Pod (KP):

KP is a subset of KD. KP consists of one or more application containers and some shared resource for those containers. Shared resource can be shared storage or networking resource like IP address.

Kubernetes Node (KN):

KN is a physical or virtual machine that is managed by KM. Multiple Pods can exist in a Node. KN has the following:

  • kubelet: A process that ensures communication between KM and KN.
  • Container runtime like docker/rkt

k8s_node

 

Kubernetes Service (KS):

KS is an abstraction layer which defines a logical set of pods and enables external traffic exposure like load balancing. The set of pods can exist in one or more KN.

k8s_scaling

Labels:

Labels are key-value pairs that are attached to objects such as pods. They are similar to hash-tags.

k8s_labels

Scaling:

Integrated load balancing in KS enables scaling.

k8s_scaling

Rolling Updates:

Rolling updates allows incremental updates by replacing existing pods with new ones within a KS.

Reference: Kubernetes Basics. Images were taken from the link provided.

Thoughts on F5 Deployment

This is a simplified check list for GTM & LTM deployment based on my experience.

Don’t deploy GTM in HA pair in a single DC:

GTM devices work in a synchronization group across geographic regions. If you deploy GTM-1 in DC-1 and GTM-2 in DC-2, these 2 GTM devices will serve as Active-Active HA Pair for most deployments.

There is no reason to have GTM-1A & GTM-1B in DC-1 and GTM-2A & GTM-2B in DC-2, where the A & B devices are part of an HA pair. Option-B design is just an overkill.

F5_GTM_Deployment

GTM to LTM VS Health Monitors:

Let the LTM monitor its pool members. The GTM will obtain the status of the LTM pool members from the LTM. GTM directly monitoring the LTM pool members or LTM VS is not required in most cases.

Don’t deploy LTM in Active-Active state:

There is a lot of room for flexibility with this statement. I haven’t found the Active-Active pair to be reliable in production environment. Such deployments are rare in the industry and hence, if there are any bugs with Active-Active pair, there is a greater chance that your deployment could be affected by the software bug than the Active-Standby setting which is widely deployed and stable.

Code Upgrade Checklist:

  1. Check license reactivation date before code upgrade.
  2. Check to make sure that the code you are utilizing is not End of Support.
  3. Save the configuration on the box and outside the box whenever upgrading code.
  4. Save the license number on the box in a /var/tmp file and off the box.

Code Version:

You are better off picking one of the Long Term Stable code version releases as provided in this K5903.

Route Domains:

Route domains are F5’s version of VRF. Although they are easy to deploy, they can be a pain to troubleshoot with the “%” addition to IP address.

iRule Vs LTM Policy:

My personal preference is always iRule as it provides greater granular control and flexibility. However, LTM Policy may be better optimized and can provide lower latency for the same function. LTM Policy can be an effective substitute for simple iRules.

Pool & Pool Members:

As a rule of thumb, all the members in a pool should have the same port. For example: POOL_WEB_80. This naming convention tells you that this is a pool of web servers that are listening on port 80 even without looking into the individual pool members. Having multiple members listening on a common port as part of a single pool helps in the long run to scale the configuration without adding significant operational complexity.

Naming Convention:

While creating custom VS, Pool, Monitor etc, I recommend starting with capital letters. For example:

POOL_Web_80
VS_domain.com_80
VS_domain.com_443
MON_HTTP_80

The default configuration elements like tcp, http monitor tends to be lower case in F5. The capital letters helps to distinguish between user-created (custom profiles) and default profiles which are all lower-case.

Custom & Default profiles:

NEVER use default configuration elements. Create custom configuration elements with the default configuration elements as parent profile.Whenever configuration change is required, always change only the custom profiles. This will prevent someone from accidentally changing the default profiles while working on the F5.

F5 TMM Crash

We were using a DNS VS listening on port 53 but configured to handle TCP protocol as shown here:

ltm virtual /Common/VS_DNS {
 destination /Common/10.10.10.10:53
 ip-protocol tcp
 mask 255.255.255.255
 pool /Common/pool_dns
 profiles {
 /Common/tcp { }
 }
 source 0.0.0.0/0
 source-address-translation {
 pool /Common/SNAT-10.10.10.10
 type snat
 }
 translate-address enabled
 translate-port enabled
 }

An iRule was using RESOLV::lookup against the configured TCP VS. RESOLV::lookup uses UDP requests and since the VS was configured to handle only TCP, the F5 crashed with core file generated.

According to F5 Engineers, bug alias 570575 is associated with this condition where RESOLV::lookup against a TCP Virtual Server causes the F5 to crash generating a core file in /var/core/

The workaround involved using “ip-protocol” as “any” and “profiles” as “fastL4” for a configuration that looks like this:

ltm virtual /Common/VS_DNS {
 destination /Common/10.10.10.10:53
 ip-protocol any
 mask 255.255.255.255
 pool /Common/pool_dns
 profiles {
 /Common/fastL4 { }
 }
 source 0.0.0.0/0
 source-address-translation {
 pool /Common/SNAT-10.10.10.10
 type snat
 }
 translate-address enabled
 translate-port enabled
 }

 

Cisco Nexus – Ping Sweep

The following script can be utilized to execute a ping sweep of a /24 network on Cisco Nexus switch. This has been tested and verified on Cisco Nexus 7K running 6.2.8a code version.

Actual script:

tclsh
for {set i 1} {$i < 255} {incr i} {
cli "ping 192.168.1.$i count 2 time 1 vrf VRF_A"
}

Script execution on a Cisco Nexus 7000 switch:

N7K_SW1# tclsh
N7K_SW1-tcl# for {set i 1} {$i < 255} {incr i} {  
> cli "ping 192.168.1.$i count 2 time 1 vrf VRF_A"
> }

Similar script but copy/pasting specific IP addresses:

tclsh
foreach address {
> 10.90.55.97
> 10.90.55.98
> } { cli ping $address count 2 time 1 vrf VRF_A }

IPSec VPN – Key Terms

The following article is a brief introduction to IPSec VPN that is utilized to provide a logical connection between 2 sites (Site to Site) or a client and a site (Client to Site). The article is written to provide the key terms behind IPSec VPN implementation in a Cisco ASA Firewall or any other similar device.

IPSec Virtual Private Network (VPN) provides the following services:

  1. Authentication
  2. Confidentiality
  3. Integrity
  4. Anti-Replay

ISAKMP: Internet Security Association and Key Management Protocol

IKE: Internet Key Exchange

IPSec: Internet Protocol Security

ISAKMP is the framework for secure communication channel establishment. IKE is the actual implementation of ISAKMP framework. There are 2 main versions of IKE – IKEv1 and IKEv2. In this article, we will stick with IKEv1.

IKE runs over UDP 500 and consists of 2 Phases.IKE Phase I can utilize either one of 2 modes – Main or Aggressive mode in order to establish a ISAKMP SA. IKE Phase II utilizes Quick Mode in order to establish IPSec SA. ISAKMP SA established in IKE Phase I is bidirectional. Two unidirectional IPSec SAs are established in IKE Phase II.

F5 Certification – Concepts

F5 certification bridges the gap between Networking and Advanced Application Layer Stack. It takes about 8-12 months to develop a test. I was fortunate to be part of the Item Development Workshop (IDW) for F5 201v2 exam and wanted to share some of the information I learned during the IDW.

Key Development Concepts utilized during the IDW:

Reliability: Consistent and precise questions.

Fairness: Does not put any group under disadvantage.

Validity: Accurately and appropriately measures what is relevant.

Reliability is not just related to the individual items but the exam as a whole. In essence, reliability of an exam is measured by the consistency of an individual’s score over multiple attempts, assuming the individual’s ability hasn’t changed substantially over the many attempts.

Validity is similarly not just about an item but the overall exam. Validity is how well the proposed purpose of the exam meets the outcomes of the exam.

Minimal Competency:

Minimally Qualified Candidate (MQC) is someone who meets the minimum requirements defined by the syllabus. A rough definition of MQC is that of an MQC Lawyer who may or may not have the skills to become a Supreme Court Judge but society is comfortable with them practicing law as the MQC Lawyer satisfies widely accepted qualification standards.

Cognitive Complexity and Difficulty:

The difference between low and high difficulty is knowing pi is 3.14 or 3.1415926535. Cognitive Complexity is using pi to find the area of circle with specific radius. Based on the blueprint of the exam, up to the 3xx level exams, only Remember and U/A were used extensively in the topics. A/E shows up more in the 4xx level exam. The difference in cognitive complexity for multiple topics are provided in the blueprint of the exam.

Cognitive Complexity:

  1. Remember (R)
  2. Understand / Apply (U/A)
  3. Analyze / Evaluate (A/E)
  4. Create (C)

Remember:

The Remember cognitive complexity generally tests rote memorization and information retrieval. There is a general preference against Remember (R) questions. So, instead of asking a question to list the TCP flags, a question that requires understanding of TCP flags in order to answer the question is preferred.

Understand/Apply:

U/A is utilized to test application of concepts within standard operations. U/A requires an understanding of processes and the ability to pick the right process to solve a problem while being able to compare multiple processes.

Analyze/Evaluate:

A/E tests the ability integrate new information with existing information to provide answers. Diagnose a problem and understand the relationship between the concepts and how one concept influences other concepts.

Create:

Create (C) tests the ability to create new products/solutions by utilizing new or existing concepts.

Items:

Items are the questions and possible options that show up in the exam. An item consists of a Stem and Options. Stem is a combination of Problem statement and Question Statement. Options can be Distractor options or Key option(s). The Key option(s) is the right answer in the item.

Remember requires only question statement. Problem statement is required for U/A, A/E and C. An ideal MQC should be able to determine the Key without having to read the options. This is one of the reasons why time is an important aspect to differentiate the competence of an exam taker. An ideal MQC should know the answer without looking at the options. Others may have to check all the options which means they will end up spending more time per question and may run short on time.

The Stems were constructed with positive words (Positive Construction). Almost all the stems eschew negative words like NOT, NEVER, EXCEPT that could potentially lead to a wrong answer as the candidate may miss the key negative word while reading the Stem.

Each item is intended to focus on a single trait that is being tested instead of multiple traits as much as possible. A trait is a subtopic that is utilized within the blueprint. Higher complexity question could have multiple traits utilized. The item is intended to be congruent to cognitive complexity & content identified in the exam blueprint without introducing any irrelevant variance that is not required to answer the question.

Response options (Distractor options and Key options) should be similar in terms of length and logic in order to prevent the option from being an obvious wrong/right answer.