AI: PBL Problem 1

In US post office, many paper mails and post cards are coming in especially in holiday seasons. To deliver them, the first thing to do is to categorize them by their zipcode. A US zipcode is a five-digit number, as in South Korea (e.g. the zipcode of Hanyang University ERICA is 15588).

Since it is very time consuming for workers to do the categorization, the US post office came up with an idea with the help from the National Institute of Standards and Technology (NIST), that they will make an automated zipcode recognition system, using technologies of artificial intelligence. As a first step, they collected handwritten digits from the envelopes and postcards, making a dataset known as the NIST dataset.

Later, scientists found the dataset is quite interesting for testing AI systems and algorithms, and a collective effort by Yann LeCun, Corrinna Cortest, Christopher Burges had transformed the NIST database into the well-known MNIST dataset, which has 60,000 digit images of 28x28 pixels, labeled as digits 0, 1, ..., 9.

A company in Ansan manufacturing parts of automobile, has been interested making a system to read the serial numbers of the car parts produced in their factory, so that they can automatically collect the same or related parts to the same packages. The company has asked the students in the AI class that if they can create an accurate digit classification system as the first trial, based on the MNIST dataset. The MNIST dataset has 60,000 training images and 10,000 test images, and the company like to see how well your system performs on the 10,000 test images.

Main objective: find the best model out of the following settings:

• Classifiers: logistic regression and SVM
  • Hyperparameters:
    • C in SVM
    • lambda, ||w||_2^2, ||w||_1 in Logistic regression
• Data: 60k training images of MNIST
• Criterion:
  • Model selection: f1-score
  • Test performance: f1-score of the final model on the test images