Sorting

Quadratic Sorting Algorithms

    Original version by Dr. Dee Parks. Modifications were made by Rahman Tashakkori in July 2004.

  1. A quadratic sorting algorithm is one that takes approximately n2 comparisons to sort n items. All of the algorithms we will study sort items that are stored in an array.

  2. Selection Sort. This algorithm repeatedly finds the next smallest item in the unsorted portion of the array and swaps it with the item that is currently in the next smallest one's rightful place at the beginning of the unsorted portion. An alternative version repeatedly finds the next largest item and swaps it into its rightful place at the end of the unsorted portion. Here's the pseudocode for the latter:
    i = n-1;

    while (i > 0)

    {

    find the index, j, of the largest value in the cells from 0 to i;

    swap(array[i], array[j]);

    --i;

    }

  3. Use selection sort to sort the following:
    6   3  10  15   2   4   8  12  5







  4. Selection Sort is a quadratic algorithm because for each of the n items, the algorithm looks at the entire unsorted portion. So the total number of comparisons is n + (n-1) + (n-2) + ... + 2 = n(n+1)/2 - 1 = (n2 + n)/2 - 1 = (1/2)n2 + (1/2)n - 1. We call any algorithm whose complexity function contains an n2 term a quadratic algorithm.

  5. Insertion Sort. This algorithm repeatedly inserts the first element in the unsorted portion into its rightful place in the sorted portion. In order to perform the insertion it shifts necessary elements rightward to make room for the one being inserted. Here's the pseudocode:
    i = 1;

    while (i < n)

    {

    j = i;

    temp = array[i];

    while (j > 0 && temp < array[j-1])

    {

    array[j] = array[j-1];

    j--;

    }

    array[j] = temp;

    ++i;

    }

  6. Use Insertion Sort to sort the following array:
    11  9   4  22  12   3   2







  7. Insertion Sort in the worst case performs the same number of comparisons as Selection Sort.

  8. Bubble Sort. This algorithm works by repeatedly swapping adjacent pairs of items if that pair is out of order. At each step the algorithm puts any given item one cell closer to its final resting place.

  9. Here is the pseudocode:
    end = n - 1;

    while (end > 0)

    {

    i = 0;

    while (i < end)

    {

    if (array[i] > array[i+1])

    swap(array[i], array[i+1]);

    ++i;

    }

    --end;

    }

  10. Use Bubble Sort to sort the following list:
    9   2   8   1  10  11   5   6







Faster Sorting Algorithms

  1. Shell Sort. Shell Sort is our first example of a non-quadratic sorting algorithm. Its speed-up is due to the fact that elements in the array move larger distances toward their rightful places at each step.

  2. Shell Sort is a variation of Insertion Sort in which we only look at elements that are a particular increment apart each time insertion sort runs. When elements are shifted, they move a distance equal to that increment.

  3. Shell Sort uses a set of increments that decrease in size with the final increment equal to 1. By the time the increment of 1 is used, the array is almost already sorted. Suppose that we use the increments {40, 13, 4, 1}. After we use the increment of 40 we say that the array is 40-sorted, and after the next pass we say that the array is 13-sorted, etc. The array is completely sorted after we finish the pass with an increment of 1.

  4. Here is one version of Shell Sort pseudocode:
    d = n;

    while (d > 1)

    {

    if ( d < 5) d = 1;

    else d = (5*d-1)/11;

    /*** Do linear insertion sort in steps size d ***/

    for ( i = n - 1; i >= n-d; i-- )

    {

    temp = array[i];

    for ( j = i-d; j >= 0 && temp < array[j]; j -= d )

    array[j+d] = array[j];

    array[j+d] = temp;

    }

    }

  5. Use Shell Sort to sort the following:
    3   2   9   1   3   2   9   8   2   6   4   3   7   2   1





  6. Merge Sort. Merge Sort is our first example of a recursive sorting algorithm that uses
    the divide-and-conquer approach. It repeatedly divides the array in half, calls Merge Sort on each half and then calls a merging function to put the two halves back together into a sorted whole.

  7. Any divide-and-conquer algorithm that divides the job in half and takes no more than n steps to either divide or glue the halves together takes approximately n log2(n) steps. This describes Merge Sort because it takes only n steps to carry out the merge of two sorted array pieces.

  8. Here is the code for Merge Sort:
    void sort(int numarray[], int low, int high)

    {

    int bottom = low, top = high;

    if (bottom >= top) return;

    int mid = (bottom + top)/2;

    sort(numarray, bottom, mid);

    sort(numarray, mid+1, top);

    int endlow = mid, starthigh = mid+1;

    while (bottom <= endlow && starthigh <= top)

    {

    if (numarray[bottom] <= numarray[starthigh])

    bottom++;

    else

    {

    int temp = numarray[starthigh];

    for (int counter = starthigh-1; counter >= bottom; counter--)

    numarray[counter+1] = numarray[counter];

    }

    numarray[bottom] = temp;

    bottom++;

    endlow++;

    starthigh++;

    }

    }

  9. Use Merge Sort and make a few passes on the following data:
    3   2   8  10  14   1   2   1   0   15   3   2   5   7   11























  10. Quicksort. On average, Quicksort is the fastest sorting algorithm known for sorting random data. It, too, is a divide-and-conquer algorithm but unlike Merge Sort, Quicksort does the majority of its work to accomplish a divide. If Quicksort gets lucky, it divides the array nearly in half. If not, one piece will be substantially larger than the other. If this latter size inequality occurs at every recursive step, Quicksort will not perform well. It will be no better than a quadratic sorting algorithm.

  11. In order to divide the array, Quicksort selects one element of the array and uses it to compare to all the others. This one particular element is called the pivot. Quicksort swaps elements around in the array to divide it into two portions. All the elements less than the pivot are placed in the leftmost portion, all elements greater than the pivot are placed in the rightmost portion, and the pivot is in between. There may be elements equal to the pivot in either portion. The process of dividing the array into these two pieces is called partitioning the array.

  12. Once Quicksort partitions the array, it recursively partitions each of the two pieces and then partitions each piece of those, etc. Generally, when a piece reaches a certain minimum size, Quicksort stops recursing and calls a simpler sorting algorithm like insertion sort to complete the work on that piece.

  13. Here is one implementation of Quicksort:
    void sort(int numaray[], int low, int high)

    {

    int bottom, top, pivot, temp;

    if (high >= low)

    {

    pivot = numarray[high];

    bottom = low - 1;

    top = high;

    while (true)

    {

    while (numarray[++bottom] <= pivot &&

    bottom < numarray.length);

    while (numarray[--top] >= pivot && top > 0);

    if (bottom >= top) break;

    temp = numarray[bottom];

    numarray[bottom] = numarray[top];

    numarray[top] = temp;

    }

    temp = numarray[bottom];

    numarray[bottom] = numarray[high];

    numarray[high] = temp;

    sort(numarray, low, bottom-1);

    sort(numarray, bottom+1, high);

    }

    }

  14. Use Quicksort to partition the following array a few times:
    15  3   5   5   7   1   2   11   3   4   3   9   7  10  20  11  3   2































The Fastest Algorithm for Sorting Fixed-Length Integers

  1. It can be proven that any sorting algorithm that accomplishes its task by comparing one element of the array to another must make at least n log2(n) comparisons to sort the array. Both Merge Sort and Quicksort reach this lower bound on average cases. There is a sorting algorithm for fixed-length integers, however, that can do its job more quickly because it never compares one element to another. The algorithm is called Radix Sort. Radix Sort is the only one of the algorithms we are studying that works best with a linked list of elements instead of an array. Each node of the linked list contains one integer of some fixed length.

  2. In addition to the linked list to be sorted, Radix Sort maintains 10 buckets, one for each decimal digit. A bucket is a linked list with both a head and a tail pointer. Suppose the integers to be sorted each have k digits. Radix Sort makes one pass down the linked list to be sorted for each of the k digits beginning with the rightmost digit and progressing leftward. It looks at the digit in question of every integer in the linked list. For each integer x, if the digit of integer x is a 0, x is inserted at the tail of the 0 bucket's linked list. This holds for every possible value of the digit--whatever the digit is, integer x is inserted at the tail of the appropriate list. Once all the integers have been placed in the right buckets for that digit, the linked lists in the buckets are concatenated together to form a new version of the original linked list. The head of the 0 bucket's linked list becomes the head of the new list. The tail of the 0 bucket's list is made to point to the head of the 1 bucket's list, the tail of the 1 bucket's list is made to point to the head of the 2 bucket's list, and so forth. The tail of the 9 bucket's list should be left with a NULL link as it marks the end of the new version of the linked list.

  3. After Radix Sort makes a pass for the leftmost digit in the fixed-length integers and recompiles the list, the integers are sorted. Since Radix Sort makes k passes, one for each digit in the integers being sorted, and in each pass it looks at all n integers, Radix Sort takes a multiple of n steps. This is the fastest algorithm of all.

  4. Use Radix Sort to sort the following list:
    673

    012

    112

    954

    771

    325

    397

    211

    798

    432

    190

    300