AI: PBL3

In PBL 1 & 2, we’ve used MNIST images after vectorizing them. This transformation is not ideal since an image may contain shapes in the directions of width or height, and therefore we can lose important spatial information between pixels that are critical to identify such shapes.

In PBL3, we will try to create new features from the pixel values of the MNIST images, so that such spatial information will be preserved and can be used for better classification in SVMs.

We learned that SVMs can use kernels for feature transformation which implicitly creates new features from the original pixel values. In the unconstrained formulation using SGD, we cannot use kernels for the purpose. In this case, we try to create the feature transformation function $\phi(x)$ for ourselves.

Goals

- Define and use new features from the original pixel values of the MNIST images

- Refine the implementation to use GPU acceleration

- Use the GPU cluster machine

- Better accuracy on the D3 dataset.

Other conditions are the same in PBL2

• The algorithm should be based on stochastic gradient descent (incl. mini-batch)
• D1: Training set: 60,000 [train-images-idx3-ubyte.gz ] [train-labels-idx1-ubyte.gz ]
• D2: Test set: 10,000 [test-images-idx3-ubyte.gz ] [test-labels-idx1-ubyte.gz ]
• D3: Test set (D2 + new 50,000 images, unlabeled): 60,000 [testall-images-idx3-ubyte.gz ]

• Your team's python code should be <team#>.py
  • Execution arguments: <team#>.py <test_image>
  • Your code must have all turned hyperparameter values within the code.
  • Submit the trained weights, and make your code to use the stored weights for testing.
• You need to submit your code, and the prediction outcome on the D3 dataset as a text file, prediction.txt. In the file, each line should be the predicted digit. For example,

1|

0|

3|

4|

....

where | stands for '\n' as in C/C++.