Init Imagenet dataset using tensorflow_datasets

[leondgarse]

• Download Imagenet dataset from Kaggle imagenet object localization patched 2019.
• Processing imagenet-object-localization-challenge.zip to traditional ILSVRC2019_img_train.tar and ILSVRC2019_img_val.tar format.

  DATASET_PATH=$PWD
  unzip imagenet-object-localization-challenge.zip -d imagenet-object-localization-challenge
  cd imagenet-object-localization-challenge
  tar xvf imagenet_object_localization_patched2019.tar.gz
  
  cd ILSVRC/Data/CLS-LOC/train/
  ls -1 | xargs -I '{}' tar cvf {}.tar {}
  tar cvf ILSVRC2019_img_train.tar ./*.tar
  
  cd ../test
  ls > foo && tar cvf ILSVRC2019_img_test.tar -T foo  # exclude the directory `./`
  cd ../val
  tar cvf ILSVRC2019_img_val.tar $(ls *.JPEG)  # exclude the directory `./`
  cd ../
  
  echo ">>>> Data saving path: $DATASET_PATH"
  mv train/ILSVRC2019_img_train.tar test/ILSVRC2019_img_test.tar val/ILSVRC2019_img_val.tar $DATASET_PATH

• Create softlinks in ~/tensorflow_datasets/downloads/manual.

  mkdir -p ~/tensorflow_datasets/downloads/manual
  rm -f ~/tensorflow_datasets/downloads/manual/ILSVRC2012_img_*.tar  # Remove any existing one
  ln -s $DATASET_PATH/ILSVRC2019_img_train.tar ~/tensorflow_datasets/downloads/manual/ILSVRC2012_img_train.tar
  ln -s $DATASET_PATH/ILSVRC2019_img_val.tar ~/tensorflow_datasets/downloads/manual/ILSVRC2012_img_val.tar
  # ln -s $DATASET_PATH/ILSVRC2019_img_test.tar ~/tensorflow_datasets/downloads/manual/ILSVRC2012_img_test.tar

• tensorflow_datasets download and prepare.

  import tensorflow_datasets as tfds
  aa = tfds.image_classification.Imagenet2012()
  aa.download_and_prepare()
  train_ds, val_ds = aa.as_dataset(split='train'), aa.as_dataset(split='validation')

• Show test.

  import tensorflow_datasets as tfds
  train_ds, val_ds = tfds.load('imagenet2012', split='train'), tfds.load('imagenet2012', split='validation')
  
  data = val_ds.as_numpy_iterator().next()
  print(data.keys())
  # dict_keys(['file_name', 'image', 'label'])
  plt.imshow(data['image'])