Data Domain – System Tuning Guide for Solaris 10 / Solaris 11

by

How to fine tune your Solaris backup servers:

Recommended mount option

For Solaris 10

#mount -F nfs -o hard,intr,llock,vers=3,proto=tcp,sec=sys, timeo=1200,rsize=1048576,wsize=1048576 HOSTNAME:/backup /ddr/backup

# mount -F nfs -o hard,intr,llock,vers=3,proto=tcp,sec=sys, timeo=1200,rsize=1048576,wsize=1048576 HOSTNAME:/data/col1/<mtree> /ddr/<mountpoint>

For Solaris 11

#mount -F nfs -o hard,intr,llock,vers=3,sec=sys,timeo=1200,rsize=1048576,wsize=1048576 HOSTNAME:/backup /ddr/backup

# mount -F nfs -o hard,intr,llock,vers=3,sec=sys,timeo=1200,rsize=1048576,wsize=1048576 HOSTNAME:/data/col1/<mtree> /ddr/<mountpoint>

Recommended Network Settings:

Data Domain recommends you to set the following IP and NFS parameters on the Solaris server:

Below nfs tunable applies for both Solaris 10 and Solaris 11 OS version

In the file /etc/system, add the following lines:

  • set nfs:nfs3_max_threads=16
  • set nfs:nfs3_async_clusters=4
  • set nfs:nfs3_nra=8

For NFS throughput

To maximize parallelism between an NFS client and its servers: /etc/system

  • set rpcmod:clnt_max_conns = 8

For higher network throughput

For Solaris 10

Create a file /etc/rc3.d/S##. Enter the following lines in the file:

  • ndd -set /dev/tcp tcp_max_buf 16777216
  • ndd -set /dev/tcp tcp_cwnd_max 8388608
  • ndd -set /dev/tcp tcp_xmit_hiwat 2097152
  • ndd -set /dev/tcp tcp_recv_hiwat 2097152

For Solaris 11

  • ipadm set-prop –p send_buf =2097152 tcp
  • ipadm set-prop –p recv_buf=2097152 tcp
  • ipadm set-prop –p max_buf=16777216 tcp
  • ipadm set-prop –p _cwnd_max=8388608 tcp

Note: The value used here is in bytes. You can use the same buffer values on a 1GIG and 10GIG interface as well and see if it helps in the performance improvement.

Jumbo Frames

  • The one factor that significantly improves NFS performance is the use of Jumbo Frames.  But one thing to take into consideration is that enabling Jumbo Frames will not guarantee improved performance.
  • It is important to understand that when you want to use Jumbo Frames, all devices in the communications path have to support it. This means that the SUN server interface(s), the DDR interface(s) and any network switches in between need to be configured to use/allow Jumbo Frames. Refer to the network switch manufacturers documentation for enabling Jumbo Frames on Ethernet switches.
  • Please make sure the MTU size used is same throughout the data path. Verify that the MTU size is consistent between Solaris client and DD.  An inconsistent MTU will cause fragmentation which can lead to slow backups

How to enable Jumbo Frames

For Solaris 10

To set the MTU size temporarily use the following command for igb0

  • # ifconfig igb0 plumb mtu 9000 up

For Solaris 11

To set the MTU size temporarily use the following command

  • # dladm set-linkprop -t -p mtu=9000 igb0

For Solaris 10 and Solaris 11 its same procedure to set the MTU size permanently

Add the following line in the /kernel/drv/igb.conf file:

default_mtu = <desired-frame-size>

The desired-frame-size value can range from 60 to 9216.

For example default_mtu=9014

And Reboot the Solaris server.

How to enable Jumbo Frames on Data Domain

To change the MTU size for an Ethernet interface, use the net config ifname mtu command. Supported values are from 256 to 9014.

For 100 Base-T and gigabit networks, 1500 is the standard default. The default option returns the setting to the default value.

Make sure that all of your network components support the size set with this option.

#net config ifname mtu {size | default}

For example, to set a maximum transfer unit size of 9014 for the interface eth2:

# net config eth2 mtu 9014

Note: Be aware that some LAN switches do not support MTU sizes over 9000.

Solaris 10 systems with soft rings enabled may have performance issues.

Solaris Clients Tuning for DDBOOST Backups

Solaris 10 Media Server tcp level tuning:

  • ndd /dev/tcp [parameter]  and set with:
  • ndd -set /dev/tcp [parameter] [value]
  • ndd -set /dev/tcp tcp_time_wait_interval 2000
  • ndd -set /dev/tcp tcp_wscale_always 1
  • ndd -set /dev/tcp tcp_tstamp_if_wscale 1
  • ndd -set /dev/tcp tcp_sack_permitted 2
  • ndd -set /dev/tcp tcp_max_buf 16777216
  • ndd -set /dev/tcp tcp_cwnd_max 8388608
  • ndd -set /dev/tcp tcp_xmit_hiwat 2097152
  • ndd -set /dev/tcp tcp_recv_hiwat 2097152

Solaris 11 Media Server tcp level tuning:

  • # ipadm set-prop -p [parameter] =[value] tcp
  • # ipadm set-prop -p _time_wait_interval=2000 tcp
  • # ipadm set-prop –p _wscale_always=1 tcp
  • # ipadm set-prop –p _tstamp_if_wscale=1 tcp
  • # ipadm set-prop –p sack=2 tcp
  • # ipadm set-prop –p send_buf =2097152 tcp
  • # ipadm set-prop –p recv_buf=2097152 tcp
  • # ipadm set-prop –p max_buf=16777216 tcp
  • # ipadm set-prop –p _cwnd_max=8388608 tcp

Timestamps, window scaling and SACK should be enabled by default.

Outgoing Connections

Some systems allow you to configure how long a system waits for an outgoing connection to be established. When set too high, establishing outgoing connections to destination servers such as replicas not responding quickly can cause long delays. For Intranet deployments on fast, reliable networks, you can set this parameter to a value of 10 seconds to improve performance. Do not, however, use such a low value on networks with slow, unreliable, or WAN connections.

On Solaris systems, this time interval is configured through the tcp_ip_abort_cinterval parameter.

#ndd -set /dev/tcp tcp_ip_abort_cinterval 1000 (** default is 3min (180000))

Disable naggle algorithm for better performance of small packets such as RPCs.

For Solaris 10 server

  • ndd -set /dev/tcp tcp_naglim_def 1
  • ndd -set /dev/tcp tcp_deferred_acks_max 0

For Solaris 11 server

  • ipadm set-prop -p _deferred_acks_max=0 tcp

Note: The tcp_naglim_def value is not available in Solaris 11

Solaris TCP_KEEPALIVE_INTERVAL

The keepAlive packet ensures that a connection stays in an active and established state.

How to view or set the value

For Solaris 10

Use the ndd command to determine the current value or to set the value.

  • ndd -set /dev/tcp tcp_keepalive_interval 15000

For Solaris 11

  • ipadm set-prop -p _keepalive_interval=15000 tcp
  • Default value for tcpkeepalive: 7200000 milliseconds
  • Recommended value is: 15000 milliseconds

Solaris Connection Failure Using DD Boost and 10G Ethernet

  • DDBOOST plugins running on Solaris 10 x86_64 (starting in 2.4) may fail to connect to the Data Domain system over a 10 Gigabit ethernet connection due to a failure to communicate using UDP with the portmapper service on the Data Domain system.

Leave a Comment