Development Kit

Installation

  1. Download the Caltech-256 Dataset and put it somewhere convenient. The below examples assume that the images are located in ~/256/images.
  2. Download the Caltech-256 Toolkit which has been tested on MATLAB 7.3 under the Linux and OS X operating systems. Please report problems .
  3. Unpack the toolkit somewhere on your local hard drive, then add this directory to your path.

Picking and Viewing Images

figure 1
 
>> cd ~/256/images
>> rand256(3);
>> cats= imcats256(1:8);
>> strview(cats)
    1  001.ak47
    2  002.american-flag
    3  003.backpack
    4  004.baseball-bat
    5  005.baseball-glove
    6  006.basketball-hoop
    7  007.bat
    8  008.bathtub
>> images= impick256(cats,8);
>> figure(1);
>> mosaic256(images,8);
>> figure(2);
>> mosaic256(images,4,4,'page',
3,'refit',true,'sideways',false)
figure 2

The command rand256 resets the random number generator seed value, making results reproducable. Next, the code fetches the first 8 categories and samples 8 images from each. Finally, mosaic256 displays the results over 8 columns (figure 1), or allows you to look more closely at a category (figure 2). Help pages (or type name in MATLAB):
help imcats256 help impick256 help mosaic256 help rand256 help strview
The following code produced the figure at the top of this page: 
         >> rand256(4);
         >> files=impick256(imcats256(1:256),1); 
         >> mosaic256(files,2,10,'page',6,'aspect',true,'labels',false,'rescale',1.3);
The imname256 function quickly accesses images by category and image number, it differs from imread:
  1. Accepts a filename or Caltech-256 category/image numbers
  2. Loads multiple images
  3. Automatically converts images to grayscale
The mosaic256 command is similar to imshow in that you can pass it either a filename or an image matrix. It differs in that it:
  1. Shows multiple images at once
  2. Accepts
    1. a single filename
    2. a single image
    3. a cell with many filenames
    4. a cell with many images

Examples

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Get 12 backpack image file names
% and load images from the first three

>> cd ~/256/images
>> filenames= imname256(3,50:61);
>> im= imread256(filenames{1:3})

im =

  [480x640 uint8] [450x600 uint8] [395x233 uint8]




%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Pass all 12 filenames to mosaic256

>> figure(3);
>> mosaic256(filenames,'sideways',false)

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Load 16 images from 100th category


>> cd ~/256/images
>> im= imread256(100,20:35);

im =

  [199x300 uint8]  [204x300 uint8]  [207x300 uint8]  [200x300 uint8]  
  [237x300 uint8]  [198x300 uint8]  [193x300 uint8]  [215x300 uint8]  
  [159x300 uint8]  [208x300 uint8]  [245x300 uint8]  [221x300 uint8]
  [255x300 uint8]  [202x300 uint8]  [218x300 uint8]  [243x300 uint8]


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Pass all 16 images to mosaic256

>> figure(4);
>> mosaic256(im);
figure 3            
figure 4            

help imname256 help imread256 help mosaic256

Training and Testing

This example picks two sets of images and generates simple feature vectors for classification. The routine features256 applies a function to each image. Here the function is regularizer, which normalizes and resizes an image: 
>> cd ~/256/images
>> ntrain= 10;
>> ntest = 25;
>> nseed= 10;
>> guesses= cell(nseed,1);
>> truths = cell(nseed,1);
>> for seed=1:nseed,

   % use consistant seeds for comparison to others
   rand256(seed);

   % generate training and test sets
   [ftrain,ftest]= impick256(imcats256(1:256),ntrain,ntest);

   % generate ground truth class labels for training and test data
   [gtrain, gtest]= groundtruth256(ntrain,ntest);

   % generate feature vectors that are our inputs
   xtrain= features256(ftrain, 'regularizer', 8);
   xtest = features256(ftest , 'regularizer', 8);

   % train using xtrain
   % 
   % [ INSERT YOUR CODE HERE ]    
   % 
  
   % test using xtest  
   %
   % [ INSERT YOUR CODE HERE]
   %
   % guess= [put your guess here]

   % at the end all we need to keep are our classification
   % guesses, for comparison with the actual ground truth   
   truths{seed} = gtest;
   guesses{seed}= guess;

end
You do not have to use our toolkit. However, you must use the same lists of files as generated above with seeds 1-10.
help impick256 help groundtruth256 help features256 help regularizer

Performance

confusion256 generates confusion matrices.

Example

>> guess= [1 2 3 4 3 3 2 2]; 
>> truth= [1 2 3 4 1 2 3 4] )
>> confusion(guess,truth)

ans = 
    1/2             0              0              0
    0               1/2            1/2            1/2
    1/2             1/2            1/2            0
    0               0              0              1/2
In the first column half the class 1 examples were labelled correctly, while the other half were mislabelled as class 3. Each of the column must sum to 1, since all test samples are classified into an available class. The overall performance is 1/2 correct or 50% (the diagonal's mean).

Overall performance (for above 10 seed values)
>> conf10= confusion256(cell2mat(guesses), cell2mat(truths) );
>> perf10= mean(diag(conf))*100

ans =
    
    4.9437
Our Ntrain=10 example yields a mean (of 10 classification trials) confusion matrix, with a 4.94% overall performance. For the contest you repeat this over 4 Ntrain values.
help confusion256
Greg Griffin
Last modified: Wed Sep 5 16:14:04 PDT 2007