Performance Tuning

Discussion in 'General Discussion' started by tomthefighter, Jul 6, 2014.

Remove all ads!
  1. tomthefighter

    tomthefighter Errr......

    Joined:
    Jun 20, 2009
    Messages:
    102
    Likes Received:
    4
    Good Morning Co8 :)

    I was wondering if a thread on performance tuning for Windows / Linux / "insert OS here" would be of any value. A ( really ) quick forum search turned up this:

    http://www.co8.org/forum/showthread.php?t=2443&page=4&highlight=performance+tuning

    ( last post 2006 - win 98/XP )


    If y'all think it would be worthwhile, please let me know and I'll put a document together & post it.


    Tomtheponderingfighter
     
  2. edmortimer

    edmortimer Occupy Wall Street

    Joined:
    Jan 16, 2014
    Messages:
    97
    Likes Received:
    0

    I think so. Any info on graphic cards and drivers would be helpful too. It seems my ATI Radeon x300 with the latest drivers (I know it's an old card, so is the 'puter) contribute to my jerkstop issues. But any and all info to help performance is always a Good Thing (tm), I think.
     
  3. Gaear

    Gaear Bastard Maestro Administrator

    Joined:
    Apr 27, 2004
    Messages:
    11,029
    Likes Received:
    42
    Feel free. But just a general word of caution: times and conditions change, and most importantly, this is ToEE. What works for one guy may not work for the next, even on seemingly identical systems. :tired:
     
  4. tomthefighter

    tomthefighter Errr......

    Joined:
    Jun 20, 2009
    Messages:
    102
    Likes Received:
    4
    My thoughts were to keep the information as general as possible. There are a number of specifics, however the stuff I'm thinking of will stand the the test of time a bit better than the " nvidia vs ATI/AMD " sort of thing.

    Tomthefighter
     
  5. tomthefighter

    tomthefighter Errr......

    Joined:
    Jun 20, 2009
    Messages:
    102
    Likes Received:
    4
    There are hundreds ( dare I say thousands ) of articles and websites dedicate to the practice & theory of performance tuning. In the course of this thread, I'd like us to get away from the typical Seagate VS W. Digital type religious wars and focus on topics that will be better able to stand the test of time.

    The initial topic I'd like to start out with is the slowest part of any computer system. Thankfully, we've come along ways away from cassette tapes ( http://en.wikipedia.org/wiki/Commodore_PET <ah the memories> ). Today's hard drives are light years removed from what they were even 10 – 15 years ago. Unfortunately trying to figure out exactly what we're buying is just as confusing today as it was in 1994 when I bought my first hard drive.


    When considering how fast a given drive is, we must think about how data is moved between it and the computer:


    Computer <--> HD Interface <–> HD buffer <– > Disc Platter(s)

    When a request for a chunk of data is made of a hard drive, the first place looked is the buffer. If this specific data was recently requested and the buffer is large enough, it may still be there. If so, the query goes no further and the data is pulled from the buffer. If not, the search proceeds on to the much slower physical disc platters.

    As the discs spin, a physical read/write head actually moves across the disc platters searching for the requested information. Once found, the data is read into the buffer and then on through the interface ( PATA, SATA, SCSI, SAS, etc ) and processed by the system.

    The key characteristics of this process are:

    Interface Speed, Buffer Size, Spindle Speed, and Internal Transfer Rate/Sustained Transfer Rate.

    Typically, the only performance information found on a store's web page / product packaging are characteristics such as:


    Interface: Serial ATA-300
    Buffer: 64MB
    Form Factor: 3.5"
    Spindle Speed: 7200 rpm
    Data Transfer Rate: 300 MBps

    You notice what's missing? Internal Transfer Rate. The Internal Transfer Rate is rarely listed in consumer marketing material and it is the key bit of info when considering the raw ability of a given drive to pump data off the drive and into your computer. The place to find this information is the manufacture's website. Once there, search for the technical document for the drive in question ( frequently, "googling the Model Number is sufficient ) - for example:

    http://www.wdc.com/wdproducts/library/SpecSheet/ENG/2879-701338.pdf

    Is the tech doc for a WD RE4 WD1003FBYX - hard drive - 1 TB – SATA-300.


    Western Digital provides the following performance info for the 1TB version of the drive:

    Data transfer rate (max)
    Buffer to host 3 Gb/s
    Host to/from drive (sustained ) 128 MB/s
    Cache (MB) 64
    Rotational speed (RPM) 7200

    The Internal Transfer Rate for this drive is listed as " Host to/from drive (sustained ) 128 MB/s ".

    128 MB/s is the "real" speed of this hard drive. Which is actually quite fast for a modern "spinning disc' hard drive. As you research various drives you'll quickly run across drives sold as "3Gbps" drives that have internal speeds far far slower ( recently ran across one for sale that had an internal rate of 75 Mbps! ).

    Another characteristic is "Average Latency". This is how long it takes ( in milliseconds ) for the read/write head to move from its current position to the location of the requested data on the disc platter. The lower the latency, the more quickly it's able to respond to the command received.

    You'll note that this metric is not listed in the first spec sheet, I had to dig around a bit more to find it here:

    http://www.wdc.com/global/products/specs/?driveID=799&language=1

    As a side note – vibration will degrade your drive's performance, using rubber washers with your drive's mounting screws is recommended. Some of these "screwless" OEM cases that let hard drives rattle around in loose fitting plastic brackets bother me....

    Next time we'll take a look at partioning strategies for Windows & FreeBSD / Linux and a note or two about linux software RAIDS.
     
  6. tomthefighter

    tomthefighter Errr......

    Joined:
    Jun 20, 2009
    Messages:
    102
    Likes Received:
    4
    The installment of the Performance Tuning Series will focus on planning one's disc layout and RAID arrays.


    We shall approach this first by asking what our overall goals should be from a performance perspective.

    These goals arguably should be:

    1 – To reduce contention for disc IO between the operating system and what ever is the application we wish maximum performance from.

    2 – To control fragmentation.

    3 – To maintain some sort of logical organization of our data.


    The first of these, disc contention, refers to the fact that minus any interference from the operating system accessing data on any type of data storage device is really a "first come first served" phenomenon ( this is not strictly accurate, but will suffice for our purposes, a more detailed examination is beyond this document's scope ). If the OS is querying drive, your application's ability to access that resource is degraded and vis a versa. A classic example of this may be observed by comparing two systems – one connecting a hard drive and cd rom drive to the same PATA cable and the other moving the cdrom drive to the second UDMA controller. Next throw an cd containing your favorite OS and run the install on both systems and see which is faster. The system with drives on dedicated cables will win every time – by a significant margin.

    With this in mind, there are three major sources of disc contention to plan for – the OS making various system calls, the swap file, and the application ( in our case TOEE! ). In a perfect world we would place the OS on disc one, the swap file on disc two and your applications on disc three. Most folks will have only one or two hard drives however. In a two disc scenario, placing your OS and swap file on disc one and your applications on disc two will yield optimal results.


    For those who compile their own source code, especially those of you who build your own custom *BSD/Linux kernels & compile its apps from source, dividing your "source files" and "object files" file system slices/partitions between separate physical discs will significantly decrease build times. For FreeBSD this would mean putting these file systems on different discs:

    /usr/src

    and

    /usr/obj

    Linux users also have a /usr/src directory that could have a dedicated disc slice.

    Next we come to controlling fragmentation. Fragmentation, as we all know, is the natural result of data being moved to/from a hard drive. If the data on a drive becomes severely fragmented, disc IO performance will be noticeably degraded. Some file systems, such as UFS and EXT3/4 manage this on their own while MS file systems ( FAT32/NTFS ) require a bit of assistance via defragmentation software. One common problem encountered with MS OSs is that files which the OS has flagged as "open" ( such as system libraries and the swap file(s) ) cannot be defragmented. To consolidate these files we'll need a "boot time" or "off line" defragmentation program, which will execute before the OS fully loads. In addition to several commercial applications that are available, there is a free program from Microsoft's Sysinternals website PageDefrag http://technet.microsoft.com/en-us/sysinternals/bb897426 that will at least address fragmentation of the swap file and registry.

    Another technique we can employ to control fragmentation ties into our third goal, maintaining logical organization of our data. This refers to idea that one should plan the partition scheme of their system's hard drive(s) based on what the role of the system is going to be and what environment will it be running in. If there are applications / OS functions that will be conducting large numbers of reads/writes we may wish to segregate them to their own partition or physical disc. A prime candidate for this is the system's swap file. FreeBSD and Linux will dedicate partitions ( aka "slices" ) for swap use as part of the OS's initial install. Windows, however, creates its swap file on the "C:" drive by default. This, and the fact that their swap files allocate disc space dynamically, leads to fragmentation of the swap file. We can address this by moving the swap file to its own dedicated partition and setting it to a fixed size.

    On the *BSDs and Unix like systems, the /var directory is one which will see large amounts of typically small files created / manipulated. The /var directory contains all of the main system logs, mail / print spoolers, package databases ( for the MS folks out there think "add/remove programs" ), etc. Therefore it is recommended to give /var its own disc partition ( if the system is going to be a dedicated mail server one might consider devoting an entire physical disc to spooling directory of your e-mail daemon if maximum performance is required ).


    RAID

    What is RAID and why do I care? Modern motherboards frequently support a special type of hard drive controller called a RAID Controller. Additionally, Windows 7 Pro/Enterprise/Ultimate, *BSD Unix , and Linux also support "Software RAID" which emulates the functions of a dedicated hardware raid controller. A RAID or "Redundant Array of Independent Discs" is a set of technologies that are used to aggregate individual hard drives for a specific goal. Normally, this is to provide a measure of "Redundancy" to protect the data stored on a computer from damage in the event of a disc failure. However, RAID can also be implemented to increase the performance of system as well. When a collection of discs is dedicated to a RAID set, it is presented to the OS as a single drive ( aka "Volume" or “Container” ). RAID Arrays are available in several "levels" depending on the system's hardware and/or operating system. While there are many RAID levels we will focus on RAID levels 0,1,5, and 6.

    RAID 0 splits a system's data between two physical discs which results in much greater data transfer rates. The down side with RAID 0 is that there is no data redundancy. WHEN one of the discs fails, you WILL loose ALL of your data. Be sure that you maintain regular backups when using this RAID level. Requires minimum of two drives.

    RAID 1 "mirrors" the data from disc 1 to disc 2. This provides a "real time" backup of all of the data stored on the volume. If one of the drives fails, no data is lost because an exact copy is on the remaining disc. Performance wise, writing data tends to be noticeably slower when compared to writing to a single disc. Reading data is slightly faster. Requires minimum of two drives.

    RAID 5 uses three or more drives to provide an aggregated pool of disc space and the ability to tolerate the loss of one drive. If a second drive fails the RAID is broken and all of the data is lost. Typically, a "hot spare" disc is designated when the RAID set is created. This spare drive will be automatically used to replace a failed drive and "rebuild" the array ( which could take days if the array is large enough to hold someone's Blue Ray collection for example ). If the RAID array should experience another drive failure during the rebuild it will fail and all of the data will be lost.

    RAID 6 requires at least four drives and is able to withstand the loss of two discs before the RAID set fails. RAID 6 is a little slower than RAID 5 because the extra data needed to protect the array from drive failure is duplicated to allow for the extra redundancy.

    Both RAID 5 and RAID 6 will perform better if more discs are dedicated to them ( you really need four discs for RAID 5 and five discs for RAID 6 otherwise they'll be a bit on the slow side ).

    Hardware VS Software RAID

    Generally speaking hardware RAID is faster than software raid. Note though that typically, a motherboard's built-in RAID Controller ( or lower end add-on card ) accelerates only RAID 0 & 1 in hardware and rely on software ( drivers ) to provide RAID 5/6 functionality.

    Linux software RAID has a very low default data transfer rate ( the actual range is 1MBps – 100 Mbps but my experience with CentOS5 has been that the system auto tunes at ~10 MBps ). This can be adjusted via this sysctl ( a “sysctl” is anologus to Windows environmental variables and Registry settings under HKLM ):


    sysctl -w dev.raid.speed_limit_min=xxxxxx

    sysctl -w dev.raid.speed_limit_max=xxxxxx

    Don't set them both to the same value and don't set the MIN value to anything ridiculous ( try 40000 as a good starting point ). You'll have to experiment a bit to settle on values that make sense for your system.

    For those interested in more information about RAID I would encourage you to take a look at the following links:

    http://en.wikipedia.org/wiki/RAID

    https://raid.wiki.kernel.org/index.php/RAID_setup

    http://www.freebsd.org/doc/handbook/geom-graid.html

    http://msdn.microsoft.com/en-us/library/aa363785(VS.85).aspx
     
  7. tomthefighter

    tomthefighter Errr......

    Joined:
    Jun 20, 2009
    Messages:
    102
    Likes Received:
    4
    The next installment of this series shall cover SSD drives.

    If there are specific aspects of performance tuning you'd like discussed, please post them. If anyone has a question or two about what we've covered, post those as well and I'll do the best I can to find an answer - or someone else reading the thread may come up with a solution.

    -Tomthefighter
     
  8. tomthefighter

    tomthefighter Errr......

    Joined:
    Jun 20, 2009
    Messages:
    102
    Likes Received:
    4
    Many apologies for not following up sooner, personal problems have forced me to set this thread aside for longer than I would've liked. I hope to have the next posting covering SSD drives up within the next week.

    I would appreciate any feedback y'all choose to provide. I started this thread by taking a look a the slowest part of any computer, that being its mass storage. For the next install- ment after the SSD segment, I'd find it helpful if I knew what aspects of Performance Tuning folks are interested in.


    I hope those who've taken the time to read this thread have found it helpful.

    Tomthefighter
     
Our Host!