Tag: NetScaler Page 2 of 3

Citrix ADC, Developer monitor

Again, by a customer I was asked to deliver a solution to there Citrix ADC. The customer has a web server farm which is controlled and maintained by their web developers. There needed their web developers to control witch servers where active in the farm and witch there where in maintains mode.

The easy way is to create a login for the web developers for the Citrix ADC, but there where other system load balanced through the Citrix ADC, and the web developers may not change the configuration for the other systems. We could have created a partition on the Citrix ADC, but I came up with a custom monitor solution. The web developers agreed to put a file on the web servers, and I created a monitor the checked for the content of that file.

GUI:

CLI:

add lb monitor lb_mon_www HTTP-ECV -send "GET https://www.domain.dk/nodestatus.txt" -recv online -LRTM DISABLED -secure YES

If the “nodstatus.txt” return online, the monitor detects the web server as “up” and the load balancing will send traffic to the web server. If anything, else than online is received the monitor will set the web server as “down” and no traffic will be sent to it.

The web developers control the content of the nodestatus.txt file, and in that way, they can control which web servers receive request and which is not, and the developers have no login/access to the Citrix ADC.

Citrix ADC 12.1 (NetScaler ADC), New licensing

Expiration:

 

With the new version 12.1 Build 48.13, Citrix added information about license expiration date. This is nice when running a Trial or Demo license:

With that they added another change, if care reading their documentation:

From https://docs.citrix.com/en-us/netscaler/12-1/licensing/netscaler-licensing-overview.html:

“Upon license expiration, the Citrix ADC appliance automatically restarts to revoke the license. If Citrix ADC appliance uses Citrix service provider (CSP) licenses, the appliance does not restart automatically to revoke the license. However, if the user restarts the appliance, it restarts as unlicensed.”

And trust me it will restart the NetScaler. As a consultant I offend use Trial versions for PoC. There where no problems in running beyond the expiration date if you did not restart the NetScaler. This is over now.

 

Express vs. Freemium:

 

Some time a go the Express version was replaced with the Freemium license. The Express license need to be updated every year because of the 1-year expiration. The Freemium have no expiration date, but it has not the Access Gateway feature.

This is a problem as the Express version was for very small customers with a replace for their old Citrix Secure Gateway (Yes, I am that old and have done a lot of installations of the Citrix Secure Gateway). So, what to customers running the Express version then do?

If we look at https://support.citrix.com/article/CTX121291 we find the answer:

Just update the NetScaler, and you will end up with an Express version with no expiration:

This must be done before the expiration of the old Express license, because if you restart the NetScaler with an expired license all features are disabled.

 

Conclusion:

 

Changes are made, so read the documentation. Get out there and upgrade the Express installations to version 12.1.

 

NetScaler, you are in control

As a consultant I often debug on NetScaler related issues. When I do this debugging I use different methods like nstcpdump, nstrace.

Nstcpdump I normally use just to clarify where we are sending data and if we are getting any response – this can tell if there are routing or Firewall issues.

SNIP = 10.11.12.152, Web Server = 192.168.1.152 port 8443

 

Routing problem, only sending data no answers:

Firewall Block, only sending data no answers:

And when it works we see monitoring data every 5 sec. (default):

As you see we can not tell from the nstcpdump if it is a routing problem or a Firewall block. We can only see that we are sending monitor data but never get any answer back.

 

The above is just for quick debugging on network related issues. When we want to do some deeper debugging nstrace is the tool to use.

As I do this it is important to inform the customer that we can see everything, and I mean EVERYTHING. I think this is important because we can see data that was not intended for us to see.

Let me show you.

I just do a simple nstrace with this command:

start nstrace -size 0

and stop it again with this command:

stop nstrace

The trace file will be in the /var/nstrace catalog on the NetScaler. Opening the trace that I just made looks like this (filtered with “frame contains “nsroot””:

As the trace show I just did a logon to my NetScaler while the trace was running, and I can se what username and password was provided. No surprise, I use the default nsroot, nsroot for my test NetScaler which is not in production.

But what if a user at the same time did a logon to the VIP = 10.11.12.156 for the Web Server 192.168.1.152?

Yes, I can see that too:

But what if we are using SSL connections? Well that is not a problem as we can get the NetScaler to decrypt the trace by doing this command (Not on SSL_BRIGDE sessions):

start nstrace -size 0 -mode SSLPLAIN

With that the login for an SSL session will be in clear text within the trace. I have not made screenshots of that as It will look just like the above screenshots.

 

But what do we then do to keep our back safe. Well we can do some filters on the traces, so we only see the traffic we are interested in.

Here are some examples.

To see traffic to and from a specified host:

start nstrace -size 0 -filter CONNECTION.DSTIP.EQ(192.168.1.152).OR(CONNECTION.SRCIP.EQ(192.168.1.152))

To see traffic for a specified port:

start nstrace -size 0 -filter CONNECTION.PORT.EQ(514)

By using filters on that nstrace command we only get the traffic we need for debugging on a specified system.

And again, remember to get permission to do the trace and tell the customer that when we do this we can see all traffic to and from a system.

 

Conclusion:

Well, this is only best practice for us NetScaler consultanst. We need to inform the owner of the system that when we do debugging on their system, we can get data that was not intended for us to see. Always keep a good and open dialog with the system owners so that we don’t get into trouble.

NetScaler CLI, Watch Out

A lot of my NetScaler configurations are done at NetScaler Command Line Interface (CLI). Bye doing this I am sure that I remember to set all the parameters of a service or other stuff.

But do you know about the “history” command?

This will display the lasted command used in NetScaler CLI. Here is an example:

Be aware that when you use CLI, that you never type password in the CLI command as other users can see this.

Here I created a local NetScaler user at CLI, and this is what are displayed in that history:

A better way is to use the -password when creating or changing password for a user. Like this:

When using the -password option, this is what is displayed in the history:

I think that Citrix is wrong when writing the CLI commands in some of their articles, like this one:

https://support.citrix.com/article/CTX224027

The god thing is that the history is pr. user list. With the nsread user I can’t see the CLI commands done by the nsroot:

Unforcedly it is not possible to clear the history list in NetScaler kernel mode, as discussed here:

https://discussions.citrix.com/topic/386456-clearing-cli-history-security-concerns/

 

My college Allan has blogged about history issue with PowerShell, see his blog here:

https://www.virtual-allan.com/powershell-command-history/

 

Conclution

Well the NetScaler CLI is powerful and is god to use. Another benefit of using the NetScaler CLI is that you will soon be abled to read the full NetScaler configuration file (ns.conf)

As I wrote in this blog just be aware of the history and avoided using password in CLI commands.

Maybe Citrix will consider making a command to clear the history list.

Citrix NetScaler, Custom logging

We, my college and I, where doing a XenMobile installation at a customer, which had a strict logging policy. The users where allow internet access through Secure Web App in XenMobile.

All internet traffic had to go out of the company Firewall, for security reasons. When doing this the NetScaler SNIP is logged in the Firewall, but the customer wants the user’s devices IP address logged.

 

We came up with a custom logging solution with a syslog server. The syslog server configuration on the NetScaler was done, se more here:

 

How to Configure Syslog on a NetScaler Appliance

https://support.citrix.com/article/CTX121728

 

When that was done, I configured an Auditing Message Actions:

With this I log the device IP and the requested URL.

The I made an Rewrite policy, with the flowing settings:

This will not do any rewrite, as action is set to NOREWRITE, but it will use the Auditing Message Actions and send the information to the syslog server.

 

Finally, I bound the rewrite policy to the XenMobile Access Gateway configuration:

For the test I tried to access http://dr.dk with in the Secure Web App, and I found the flowing logging on the syslog server:

Conclusion:

We ended up with a logging of the device IP and the access URL. With the many expressions available on the NetScaler you would be able to log almost everything in the syslog server.

Remember to bound the rewrite policy with NEXT as Goto Expression, or you could end up with others rewrite policies not being processed.

 

CLI commands:

add audit messageaction log_act_custom ALERT “\”Client:\”+CLIENT.IP.SRC+\” accessed \”+HTTP.REQ.URL”

add rewrite policy log_rw_pol true NOREWRITE -logAction log_act_custom

bind vpn vserver < Gateway virtual server name> -policy log_rw_pol -priority 100 -gotoPriorityExpression NEXT -type REQUEST

 

Page 2 of 3

Powered by WordPress & Theme by Anders Norén