Universal or UIE Persistence

Universal or UIE Persistence is one of the great features that is available with F5 devices. UIE stands for Universal Inspection Engine. Universal Persistence provides us with the ability to load balance traffic based on the information available in the HEADER or the CONTENT of the incoming packet.

When to use it:

  • When you require persistence based on a session identifier that can’t be utilized using the pre-configured persistence profiles like Source IP or Cookie Insertion.

Usage Example:

Sometimes, you may want to have persistence to exist based on a “Session Cookie” that is inserted by the Server. This “Session Cookie” could be utilized within the Header or URI of incoming packet and this may also be utilized within the content, when you are using the HTTP POST method.

Configuration Steps:

  • Identify the Session Identifier & its location – Example: JSessionID present in URI
  • Write a simple iRule utilizing “persist UIE”
  • Create a Custom UIE profile with default UIE profile as the parent profile. Attach the iRule to this Custom UIE profile

A simple example based on X-Forwarded-For header:

when HTTP_REQUEST { 
    if {[HTTP::header X-Forwarded-For] != ""}{  
       persist uie [HTTP::header X-Forwarded-For] 600 
    } 
 } 

Use the above iRule within a UIE persistence profile for persistence based on the incoming X-Forwarded-For header value. “600” denotes the timeout in seconds and this can also be set in the UIE persistence profile.

Reference:

SOL7392

ADX Health Check

By default, when a real-server is created, the ADX will perform the following checks:

  • Layer 2 ARP
  • Layer 3 ICMP Ping

After the real-server is attached to the virtual-server using the bind command, the ADX will perform Layer 4 or Layer 7 checks based on the configuration.

It is essential to make sure that there is nothing blocking L2  or L3 checks between the ADX and the Server like a Firewall.

As there are multiple health checks for Brocade ADX, it is better to limit these checks to specific health checks to prevent any issues.

For example, when a real-server is created and “port http” configured,

port http url “HEAD /” is automatically added. Using “port http l4-check-only” will limit the health-check to Layer 4 health check instead of the Layer 7 checks. By default, for L7 “HEAD /” will mark the real-server as UP, if the response status code is 200-299, 401. This can be limited by using the command:

port http status-code 200 200

The above command will mark the real-server’s HTTP port UP, only if the response status-code is 200.

It is also essential to remember that the ADX will perform the L4 & L7 checks only once – when the real-server is bound to the virtual-server. If you require periodic checks, you would have to utilize the following command:

port http keepalive

The above command tells the ADX to keep checking the real-server’s port periodically instead of checking it only once.

A simple real-server would look like this for L4 Health-Check:

server real-server RS1 10.10.10.1

port http

port http keepalive

port http l4-check-only

port http url “HEAD /”

Reference:

ADX Health Check

 

F5 iRule – Secure & HTTPOnly Cookie

The following iRule taken from devcentral.f5.com was utilized to insert the “Secure” tag to all the cookies within the Response Header. Note that some part of the iRule has been “deactivated” as this part involves adding the “HTTPOnly” cookie tag which isn’t required for this customer. I was able to verify its functionality in 10.x code version (also works in 11.x):

when HTTP_RESPONSE {
if { [catch { set CK_VALUE [HTTP::header values "Set-Cookie"]
HTTP::header remove "Set-Cookie"

foreach value $CK_VALUE {
if { "" != $value } {
set TEST_VALUE [string tolower $value]
set LENGTH_VALUE [string length $value]

switch -glob $TEST_VALUE {
"*;secure*" -
"*; secure*" { }
default { set value "$value; Secure"; }
}

switch -glob $TEST_VALUE {
 "*;httponly*" -
 "*; httponly*" { }
 default { set value "$value; HttpOnly"; }
 }

HTTP::header insert "Set-Cookie" $value

}
}
} ] } {
log local0. "Exception thrown, client_addr=[client_addr] HttpOnly and/or Secure cookie attributes may not have been set"
}
}

For 11.x code version, there is a simpler way of achieving this functionality as noted here: https://devcentral.f5.com/wiki/iRules.HTTP__cookie.ashx

HTTP::cookie httponly <name> [enable|disable]
HTTP::cookie secure <name> [enable|disable]

When the “HTTP::cookie” rules were tried out in a lab environment using the following iRule:

when HTTP_RESPONSE {
   HTTP::cookie version BigIP_Cookie 1
   HTTP::cookie httponly BigIP_Cookie enable 
}

I got the following error log:
TCL error: /Common/iRule-HTTPOnly-Secure <HTTP_RESPONSE> – Illegal argument (line 1) invoked from within “HTTP::cookie version BigIP_Cookie 1”

So, I removed the “cookie version” command:

when HTTP_RESPONSE {
   HTTP::cookie httponly BigIP_Cookie enable 
}

and got the following error:
TCL error: /Common/iRule-HTTPOnly-Secure – Improper version (line 1) invoked from within “HTTP::cookie httponly BigIP_Cookie enable”

It looks like the “HTTP::coookie version” function isn’t working as expected in 11.x code version. F5 Bug-id for this issue 338981.

This is an alternate iRule that seems to be working in 11.x code version

when HTTP_RESPONSE {
set COOKIE_VAL [HTTP::header values "Set-Cookie"]
HTTP::header remove "Set-Cookie"

foreach COOKIE_NAME $COOKIE_VAL {
HTTP::header insert "Set-Cookie" "${COOKIE_NAME}; Secure; HttpOnly"
}
}

Reference: Devcentral.

CSW PHPSESSID Persistence

csw-rule “CSW_RESPONSE_Cookie” response-header “Set-Cookie” pattern “PHPSESSID”
csw-rule “CSW_URI” url pattern “PHPSESSID”
csw-rule “CSW_PHPSESS” header “Cookie” pattern “PHPSESSID”

csw-policy “POLICY_PHPSESSID”
match “CSW_RESPONSE_Cookie” passive-persist offset 0 length 7
match “CSW_PHPSESS” persist offset 0 length 7 passive-persist
match “CSW_URI” persist offset 0 length 7 passive-persist

The above CSW Rule & Policy will provide persistence based on the PHPSESSID cookie that is presented by the server.

F5 Error – bcm56xxd

Oct 20 10:08:16 local/lb01 err bcm56xxd[3506]: 012c0011:3: I2C SFP operation failed: dev:0xee at :228, errmsg(Operation timed out)

This specific error is benign. bcm56xxd has a thread that checks every 5 seconds to see if there is any units plugged or unplugged on all the SFP and XFP ports by accessing the GPIO pins attached to these SFP/XFP ports via the I2C bus.

The plug check does not disrupt the traffic flow on an existing port.

If the system is busy some I2C operation may time out as the I2C bus becomes overloaded.

bcm56xxd – The switch daemon configures and controls the Broadcom 56xx switch chips.

Reference:

SOL13444

Questions to consider – Buying an ADC

These are some of the questions that you would want to analyze and answer before thinking about buying an ADC:

What is the load requirement ?

This is defined in terms of maximum number of concurrent or simultaneous connections/requests that an ADC can handle.

Another factor to consider is the rate of connections/requests. If your application requires short bursts of traffic, the ADC should be able to handle it.

What kind of protocol do you intend to load balance ? 

Most customers tend to load balance HTTP traffic. If you intend to perform load balancing of specific applications like Citrix/Xen related apps, it will be better to buy a Citrix ADC like NetScaler.

Does your application require “persistence” and if so, what kind of “persistence” do you require ?

Persistence is the ability of the load balancer to send a client connection request to the same server that handled the previous request based on information presented by the client connection. This information can be Source IP address or Cookie or any information available in the incoming packet like JSession ID.

Persistence information is generally required for applications like a checkout cart. For example, after a client adds merchandise to the checkout cart, subsequent HTTP request/TCP connection should be sent to the server with the checkout cart information in order to complete the transaction. If the connection/request is sent to a different server, the checkout cart may not have the relevant information.

Do you require Layer7 load balancing like redirect or load balancing based on HTTP header/content ?

One of the differentiating factor between the newer generation ADC and the older Load Balancer is the difference in their ability to handle L7 function. Load Balancers don’t provide as much L7 functionality as the ADC.

Certain functions like redirects can be implemented on the ADC instead of the servers. This will reduce the round trip time and the latency involved in serving the application and at the same time make it easier from an administrative perspective as the redirect configuration is implemented at one point instead of multiple servers.

Are you planning to terminate the SSL certificate/key on the ADC and send the unencrypted traffic to the servers ?

Implementing SSL termination on the load balancer will reduce the load on the servers as SSL processing can be resource intensive. From a management perspective, it is easier to replace the certificate/key on a single device (ADC) than a multitude of servers. With a flood of SSL vulnerabilities, any changes required to the SSL ciphers or versions can be done at one location.

If load balancing requires L7 functionality, the SSL cert/key has to be terminated on the ADC as the SSL encrypted traffic has to be decrypted at the ADC before any L7 functionality can be implemented by the ADC.

Does your business require specific SSL ciphers/versions for regulatory or security reasons ?

SSL processing is done on hardware or software. For any ADC, some ciphers are handled at the hardware level and some ciphers are handled at the software level. Hardware SSL generally tends to be efficient than software SSL processing.

SSL Keys – Newer ADCs are optimized for the 2K keys and the older ones can only handle 1K key efficiently. Newer F5 platforms are better optimized for SSL 2K keys than the older F5 LTM 1600 & 3600

Do you require High Availability ?

ADC can be a single point of failure. Using ADC in high availability setting would provide the redundancy.

Do you require any specific performance features ?

These performance features can be like caching, compression or newer protocols like SPDY.

Do you require any other functionality ?

There are functions like application acceleration, application firewall, IPv6 Gateway that can be implemented on the ADC in addition to the normal load balancing.

Major ADC Vendors:

Top 3 based on Gartner 2013:

BigIP F5

Citrix NetScaler

Radware

Others:

Riverbed, A10, Brocade, Barracuda etc.

ADC Functions:

Scalability:

This provides the ability to add/remove servers with minimal disruption to ongoing traffic processing.

High Availability:

Do you require 2 (or more) Load Balancers that can be set up such that one of the “standby” load balancers take over the active load balancers function, if one of them fails ?

Performance:

This is not just about the values like connections/s or throughput. You would have to consider the feature set that is available to you to maximize the application delivery. These features can be caching, compression, newer protocol support like SPDY.

Security:

The newer load balancers provide greater support against certain Denial of Service (DoS) and security like a single authentication portal, web application firewall (WAF)

RackConnect Versions

RackConnect:

RackConnect is a Rackspace product that provides connectivity between Dedicated, Single Tenant Environment and an Openstack based Public Cloud Environment.

RackConnect_How

*Image taken from Rackconnect page.

RackConnect Versions:

RackConnect v1.0 is the very 1st version of RackConnect and it provides simple connectivity between the Dedicated, Single Tenant Environment and the Cloud Servers via ServiceNet.

After a cloud server is spun up, manual changes would have to be made to the cloud server’s route table, interface and software firewall/iptable in order to force the traffic to flow to dedicated servers.

RackConnect v2.0 is the next iteration of the RC product series. The major advantage of RC v2 compared to RC v1 is Automation. Changes to route table, cloud server interface, software firewall/iptable and IP assignment were taken care by the automation. RC v2 was also essential for utilizing the next generation “performance cloud servers” offered by Rackspace.

RackConnect v2.1 is similar to RC v2 but consists of RC v1 configurations that were migrated to RC v2.

RackConnect v3.0 is the latest in the RackConnect series. RC v3 utilizes SDN & Cloud Networks while the previous versions utilized the ServiceNet network offered for access to the different “Services” provided within the Datacenter.

Comparison of RC v3 & RC v2 is seen here.

As of Q4 2014, new customers will be provided with RC v3 features. There are quite a few customers on RC v2 and RC v2.1. RC v1 is almost non-existent.

Reference:

RackConnect

BigIP F5 – Poodle Vulnerability

Poodle was initially targeted against SSLv3. Quite a few websites fixed this issue at the server and client side by disabling SSLv3. There is a variation of Poodle for TLS with the following CVE ID: CVE-2014-8730. For a brief description of the issue: Poodle on TLS

This is known to affect load balancers like F5. F5 recommends a code upgrade. As of now (Dec 09, 2014), it is recommended that the code is upgraded to at least 10.2.4 with HotFix 10 if you are running 10.x code version and one of the 11.x code versions that is not vulnerable as per F5 documentation: SOL15882

F5 has stated that the code upgrade is the best possible option available. 

BigIP F5:

If you are running F5 LTM on 11.x code version, using 11.5.1 with HF6 and above is recommended.

If you are running F5 LTM on 10.x code version and want to stay in 10.x code version, you would have to perform the following tasks in order to have a “A-” rating in Qualys SSL test:

  • Upgrade code to 10.2.4 HF10
  • After code upgrade to 10.2.4 HF10, complete the following steps in order to remove RC4.
(tmos.ltm)# create profile client-ssl PARENT-SSL-SECURE ciphers 'HIGH:MEDIUM:!SSLv3:!RC4'

(tmos.ltm)# modify profile client-ssl CUSTOM-CLIENT-SSL defaults-from PARENT-SSL-PROFILE-SECURE

Qualys Rating is “B” after code upgrade and “A-” after performing code upgrade and removing RC4.

After making the code upgrade & removing RC4 cipher, it is recommended that you test your site for any vulnerabilities at the Qualys Site.

The above commands will create a “Parent” SSL Client profile – “PARENT-SSL-SECURE” that will disable SSLv3, RC4 and order the ciphers from High to Medium strength. Please, note that any client initiating connection on SSLv3 will be dropped. Usually clients running Windows XP and IE6 will initiate SSLv3 connections.

In order to know the total SSLv3 connections, you can run this command:

(tmos.ltm)# show profile client-ssl CUSTOM-CLIENT-SSL | grep Protocol
 Protocol
 SSL Protocol Version 2                       0
 SSL Protocol Version 3                      156
 TLS Protocol Version 1.0                   16.1K
 TLS Protocol Version 1.2                   11.6K
 DTLS Protocol Version 1                      0

Based on the above output, it can be seen that the client SSL profile configured on the Virtual Server handling SSL traffic has received 156 SSLv3 connection requests.

The stats can be cleared using this command:

(tmos.ltm)# reset-stats profile client-ssl

(tmos.ltm)# show profile client-ssl CUSTOM-CLIENT-SSL | grep Protocol
 Protocol
 SSL Protocol Version 2                       0
 SSL Protocol Version 3                       0
 TLS Protocol Version 1.0                     0
 TLS Protocol Version 1.2                   11.6K
 DTLS Protocol Version 1                      0

Note: The “reset-stats” command will NOT clear the stats for TLS1.2  in 10.2.3 code version. F5 is aware of this bug. I was able to clear the stats for 10.2.4 code version. I am not sure if this bug exists for 10.2.3 and lower code versions.

The cipher suite that is being utilized, after the change can be identified by running the following command in bash:

[root@lbal1:Active] config # tmm --clientciphers 'HIGH:MEDIUM:!SSLv3:!SSLv2:!RC4:!COMPAT:!EXP'

Cipher-Suite-F5-TMM

NOTE:

After making the above changes, some of the older browsers may not be able to connect to your website on HTTPS as the older browsers don’t support TLS1.2. For a list of browsers and supported protocols, see here.

Having dealt with SSL/TLS vulnerabilities, it looks like TLS1.2 with Ephemeral Diffie-Hellman is the most secure current version. Anything before TLS1.1 would be insecure and would have to be avoided, if possible. The main problem is that there are quite a few client browsers that are just incompatible with TLS1.2 and this has to be factored before making any changes.

GHOST Vulnerability:

This is a recent vulnerability – CVE-2015-0235

According to SOL16057, if you want to stay with 10.x code version on F5 LTM, it is better to use 10.2.4 + HF11. For 11.x code version, use at least 11.5.1 + HF8.

Update:

For a great Qualys grade see the following link.

!SSLv2:!EXPORT:!DHE+AES-GCM:!DHE+AES:!DHE+3DES:ECDHE+AES-GCM:ECDHE+AES:RSA+AES-GCM:RSA+AES:ECDHE+3DES:RSA+3DES:-MD5:-SSLv3:-RC4:@STRENGTH