Interaction_CNN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Interaction Convolution [상호작용을 반영한 convolution]\n",
    "\n",
    "convolution 에서 filter [W] 는 학습될 수 있는 가중치이고,\n",
    "image patch[X] 와 곱해져 feature map 을 생성한다.\n",
    "\n",
    "이것은 내가 이해하기로는 image_patch 는 data 고, \n",
    "filter는 beta 값인 채로 linear regression 혹은 linear transform, affine transform 과 같다.\n",
    "\n",
    "그렇다면 선형 회귀에서 사용하듯 interaction term(상호작용 항)을 고려하면 주변 픽셀들 간에 interaction 에 대한 weight 를 학습할 수 있지 않을까?\n",
    "\n",
    "내가 단순하게 구현해보기론 X.T * X 로 feature map 혹은 input 에 대한 interaction 을 구하고 그 interaction 에 대한 convolution을 다시 함으로써 feature map 을 구할 수 있고, \n",
    "이것을 원래의 feature map과 더함으로써\n",
    "상호작용 항이 고려된 feature map 을 구할 수 있을 거라 생각한다.\n",
    "\n",
    "cifar10 학습해본 결과\n",
    "\n",
    "일반적인 cnn 보다 학습이 조금 더 빨리 되는 것을 알 수 있었다.\n",
    "\n",
    "------------------------------------------------------------------------------------------------------------------\n",
    "foot translation\n",
    "\n",
    "In convolution, filter[W] is learnable weight, and then feature map produced by that image patch multiply filter \n",
    "\n",
    "I think image_patch is data, filter is coefficient, and then it is linear regression or linear transform, affine transform.\n",
    "\n",
    "If we take the interaction term as in linear regression, can we learn the weight of the interaction between the neighboring pixels?\n",
    "\n",
    "I simply implement it, input or feature map is X.T * X, convolution it produce feature map. \n",
    "\n",
    "It add original feature map, produce feature map with interacion.\n",
    "\n",
    "Cifar10 results\n",
    "\n",
    "It faster than original cnn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "import numpy as np\n",
    "tf.set_random_seed(777) \n",
    "import cifar10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data has apparently already been downloaded and unpacked.\n"
     ]
    }
   ],
   "source": [
    "cifar10.maybe_download_and_extract()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading data: data/CIFAR-10/cifar-10-batches-py/data_batch_1\n",
      "Loading data: data/CIFAR-10/cifar-10-batches-py/data_batch_2\n",
      "Loading data: data/CIFAR-10/cifar-10-batches-py/data_batch_3\n",
      "Loading data: data/CIFAR-10/cifar-10-batches-py/data_batch_4\n",
      "Loading data: data/CIFAR-10/cifar-10-batches-py/data_batch_5\n"
     ]
    }
   ],
   "source": [
    "images_train, cls_train, labels_train = cifar10.load_training_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading data: data/CIFAR-10/cifar-10-batches-py/test_batch\n"
     ]
    }
   ],
   "source": [
    "images_test, cls_test, labels_test = cifar10.load_test_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def random_batch(train_batch_size):\n",
    "    # 학습 데이터셋 안에 이미지 수\n",
    "    num_images = len(images_train)\n",
    "\n",
    "    # 무작위 인덱스 만들기\n",
    "    idx = np.random.choice(num_images,\n",
    "                           size=train_batch_size,\n",
    "                           replace=False)\n",
    "\n",
    "    # 무작위 인덱스를 이용해 이미지와 라벨을 선택함\n",
    "    x_batch = images_train[idx, :, :, :]\n",
    "    y_batch = labels_train[idx, :]\n",
    "\n",
    "    return x_batch, y_batch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 상호작용이 반영된 CNN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0001 cost = 1.525525022\n",
      "Epoch: 0002 cost = 1.257694640\n",
      "Epoch: 0003 cost = 1.166033215\n",
      "Epoch: 0004 cost = 1.101056453\n",
      "Epoch: 0005 cost = 1.059084578\n",
      "Epoch: 0006 cost = 1.011964905\n",
      "Epoch: 0007 cost = 0.968527456\n",
      "Epoch: 0008 cost = 0.956996769\n",
      "Epoch: 0009 cost = 0.932130665\n",
      "Epoch: 0010 cost = 0.913818698\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH6tJREFUeJzt3Xl8lOW99/HPb7KSkA0SEkImCSg7ypYAgqit+1KtVgVc\nulk5Hlu7P6fLeZ4u9unp0z6trV1sReqx1Qqu1K3a1g2psiQsIiggAkkIZMFAEgghJHOdP2ZEREMC\nmeTOzHzfrxcvk5nbzNd5yXeuXPd1X7c55xARkeji8zqAiIiEn8pdRCQKqdxFRKKQyl1EJAqp3EVE\nopDKXUQkCqncRUSikMpdRCQKqdxFRKJQvFcvnJ2d7YqLi716eRGRiLR69eo9zrmcro7zrNyLi4sp\nLy/36uVFRCKSmVV05zhNy4iIRCGVu4hIFFK5i4hEIZW7iEgUUrmLiEQhlbuISBRSuYuIRKGIK/et\ndfu5/ak3aWsPeB1FRKTfirhyr2po4d5Xt/Piplqvo4iI9FsRV+5njcohLz2ZxWVVXkcREem3Iq7c\n43zGtSUFLN1Sz659B72OIyLSL0VcuQNcU+IH4JHynR4nERHpnyKy3P2DUjjz1GweLq+iI+C8jiMi\n0u90We5mdq+Z1ZnZhk6eP8fMGs1sXejP98If88PmlPqp3neQV7fu6YuXExGJKN0Zud8HXNTFMcuc\nc5NCf27veayunT8ul6yUBB7SiVURkQ/pstydc68ADX2Q5YQkxcdx1ZQC/vFmDe/uP+R1HBGRfiVc\nc+5nmNnrZvasmY3v7CAzm29m5WZWXl9f3+MXnVPq53CHY8na6h7/LBGRaBKOcl8DFDnnJgK/Af7a\n2YHOuQXOuRLnXElOTpd3ierSqNw0phRmsrisCud0YlVE5D09LnfnXJNzbn/o678BCWaW3eNk3TS3\ntJCtdftZXbG3r15SRKTf63G5m1memVno62mhn/luT39ud116+lBSE+N0xaqIyFG6sxRyEbAcGG1m\nO83sJjO7xcxuCR1yNbDBzF4Hfg3MdX04R5KaFM/lk/J5Zv1umloP99XLioj0a/FdHeCcm9fF878F\nfhu2RCdhTmkhi1ZV8dTru7h+epGXUURE+oWIvEL1WBMLMhiTl6Y17yIiIVFR7mbGnFI/63c2snFX\no9dxREQ8FxXlDnDl5GEkxvt4WKN3EZHoKffMlEQuGp/HkrXVtB7u8DqOiIinoqbcAeaW+mlqbee5\nDTVeRxER8VRUlfuMEYMpGpzC4rJKr6OIiHgqqsrd5zOuLfGzYlsD2/cc8DqOiIhnoqrcAa6eWkCc\nz3i4XCdWRSR2RV2556Yn87HRQ3h09U4OdwS8jiMi4omoK3cInlitbz7ES5vqvI4iIuKJqCz3c0bn\nMCQtSVesikjMispyj4/zcU1JAS9trqOmsdXrOCIifS4qyx3g2hI/AQePrtboXURiT9SWe9HgVGae\nMpiHyqsIBHSXJhGJLVFb7hC8x2pVw0GWb+uze4eIiPQLUV3uF47PI2NAgu7SJCIxJ6rLPTkhjisn\nD+PvG2rYe6DN6zgiIn0mqssdglMzbR0Blqyt9jqKiEififpyHzs0nYn+TB4qq6IPb+0qIuKpqC93\nCF6xurm2mXVV+7yOIiLSJ2Ki3D8xMZ+UxDhdsSoiMSMmyn1gUjyXnT6UJ1/fxf5D7V7HERHpdTFR\n7gBzSgtpaevgmfW7vI4iItLruix3M7vXzOrMbEMXx5WaWbuZXR2+eOEzpTCTkUMGas27iMSE7ozc\n7wMuOt4BZhYH/BT4Rxgy9QozY06pn7WV+9hc0+x1HBGRXtVluTvnXgEaujjsNuAxoF9voH7VlAIS\n4kwnVkUk6vV4zt3MhgFXAr/veZzeNSg1kQvG5/H42p0cau/wOo6ISK8JxwnVXwHfcs51eU87M5tv\nZuVmVl5fXx+Glz5xc0v97Gs5zD821nry+iIifSEc5V4CLDazHcDVwF1m9smPOtA5t8A5V+KcK8nJ\nyQnDS5+4WadkMyxzgKZmRCSq9bjcnXPDnXPFzrli4FHgVufcX3ucrJf4fMETq//auoeqhhav44iI\n9IruLIVcBCwHRpvZTjO7ycxuMbNbej9e77h6agE+Q6N3EYla8V0d4Jyb190f5pz7bI/S9JH8zAGc\nPSqHR1ZX8dXzRhIfFzPXcolIjIjZVptTWkht0yGWbvHmxK6ISG+K2XI/d+wQsgcm6YpVEYlKMVvu\nCXE+rp5awIub6qhravU6johIWMVsuUPwLk0dAceja3Z6HUVEJKxiutyHZ6cyffgg3aVJRKJOTJc7\nwNxpfirebWHFtq62zxERiRwxX+4XTxhKWnI8D5VVeh1FRCRsYr7ckxPiuHLyMP62oYbGlsNexxER\nCYuYL3cInlhtaw/w13XVXkcREQkLlTswPj+D04ZlsGhVpU6sikhUULmHzCn1s6mmmTeqG72OIiLS\nYyr3kMsn5ZOc4NMVqyISFVTuIenJCVx6Wj5PrttFS1u713FERHpE5X6UudP87D/UzjPrd3sdRUSk\nR1TuRykpymJETqr2eReRiKdyP4qZMbfUT3nFXrbWNXsdR0TkpKncj3HVlALifabRu4hENJX7MbIH\nJnH+uFweW1NNW3vA6zgiIidF5f4R5pT6aTjQxvNv1XodRUTkpKjcP8LskTnkZyRrzbuIRCyV+0eI\n8xnXlPhZ9nY9O/e2eB1HROSEqdw7cU1JAQCPlOsuTSISeVTunSjISmH2yBweKa+iI6DNxEQksnRZ\n7mZ2r5nVmdmGTp6/wszWm9k6Mys3szPDH9Mbc0v97GpsZdnb9V5HERE5Id0Zud8HXHSc518AJjrn\nJgGfBxaGIVe/cN7YXAalJmrNu4hEnC7L3Tn3CtDpDUadc/vd+5ugpwJRM4eRGO/jU1OG8c83a9mz\n/5DXcUREui0sc+5mdqWZbQKeITh6jxpzSv20BxyPr9GJVRGJHGEpd+fcEufcGOCTwI86O87M5ofm\n5cvr6yNjHvvUIWmUFGWxuKxKd2kSkYgR1tUyoSmcEWaW3cnzC5xzJc65kpycnHC+dK+aU+pnW/0B\nyiv2eh1FRKRbelzuZnaqmVno6ylAEvBuT39uf3Lp6UMZmBTP4lU6sSoikaE7SyEXAcuB0Wa208xu\nMrNbzOyW0CGfAjaY2Trgd8AcF2XzFymJ8Vw+KZ9n3thFU+thr+OIiHQpvqsDnHPzunj+p8BPw5ao\nn5pXWsiDKyt5ct0ubphR5HUcEZHj0hWq3TRhWDrjhqazuKzS6ygiIl1SuXeTmTF3mp8N1U1sqG70\nOo6IyHGp3E/AFROHkRTv0xWrItLvqdxPQEZKApecNpS/rqvmYFuH13FERDqlcj9Bc0r9NLe28+yG\n3V5HERHplMr9BE0fPojiwSm6S5OI9Gsq9xNkZswpLWTV9ga21e/3Oo6IyEdSuZ+ET00dRpzPeKhc\no3cR6Z9U7idhSFoy544ZwkNlVWypbfY6jojIh6jcT9I3LxxNQpyPT/3+NZa/E1Vb6YhIFFC5n6RR\nuWksuXUmuenJfPrelTyxrtrrSCIiR6jce6AgK4XHbpnJlMIsvrJ4Hb97aav2fBeRfkHl3kMZKQn8\n+aZpXDEpn///983851830N4R8DqWiMS4LneFlK4lxcfxy2snMSxzAHe9/A679x3kt9dNITVJb6+I\neEMj9zDx+Yz/uGgMP75yAku31DNnwXLqmlu9jiUiMUrlHmbXTy9i4WdKeKfuAFf+7jW21mmppIj0\nPZV7L/j4mFwe+rcZHGoPcNVdr7Fym5ZKikjfUrn3ktMLMlly60xy0pK48Y+rtFRSRPqUyr0X+Qel\n8Pi/z2JSYSZfWbyO37/8jpZKikifULn3soyUBO6/aRqfmJjPT5/bxP95QkslRaT3aa1eH0iKj+PO\nOcGlkn9Y+g6797Xym+smk5Kot19EeodG7n3E5zO+ffEYfvTJCby0uY45d6/QUkkR6TUq9z5244wi\n7vl0CVvr9nPVXa+xtU57wotI+KncPXDu2FwWz59B6+EOPvX711i1vcHrSCISZbosdzO718zqzGxD\nJ89fb2brzewNM3vNzCaGP2b0mejPZMmtsxg8MJEbFq7kqdd3eR1JRKJId0bu9wEXHef57cDZzrnT\ngB8BC8KQKyYEl0rOZKI/g9sWreXupVoqKSLh0WW5O+deATqdN3DOveac2xv6dgVQEKZsMSEzJZH7\nb5rOpacP5SfPbuJ7T2ykI6CCF5GeCfdavJuAZzt70szmA/MBCgsLw/zSkSs5IY7fzJ1MQeYA7n5l\nG7sbD/LreVoqKSInL2wnVM3sYwTL/VudHeOcW+CcK3HOleTk5ITrpaOCz2d855Kx3H7FeF7cVMe8\nBSuobz7kdSwRiVBhKXczOx1YCFzhnNMuWT3w6TOKufvGEjbXNnPV71/lnXotlRSRE9fjcjezQuBx\n4Ebn3JaeR5Lzx+WyeP4ZtBwKLpUs26GlkiJyYrqzFHIRsBwYbWY7zewmM7vFzG4JHfI9YDBwl5mt\nM7PyXswbMyaFlkoOSknk+oUreWb9bq8jiUgEMa+W3pWUlLjycn0OdGXvgTZu/nM55RV7+e4lY7h5\n9gjMzOtYIuIRM1vtnCvp6jhdodrPZaUm8sAXpnPpaUP5r79t4gdPaqmkiHRNa+0iQHJCHL+ZN5n8\nzGTuWbadXY2t/HruZAYkxnkdTUT6KY3cI4TPZ/znpeP4wSfG8fxbtcy9ZwV79muppIh8NJV7hPns\nrOHcfcNUNtc0cdVdr7FNSyVF5COo3CPQBePzWHTzDA4caucq7SopIh9B5R6hJhdm8fitM8lKSWTO\nguV8/4kNNLUe9jqWiPQTKvcIVjQ4lSe+NItPzyjizysqOO8XS3lm/W7tLCkiKvdIl56cwA+vmMBf\nb51FTloSX3xwDZ+7r4yqhhavo4mIh1TuUWKiP5MnvjiL7102jrLtDZz/y6Xc9fJW2toDXkcTEQ+o\n3KNIfJyPz585nOe/cTbnjBrCz57bzGW/Waa9aURikMo9Cg3NGMAfbpzKHz9TwoFDHVzzh+V869H1\n7Gtp8zqaiPQRlXsUO3dsLv/8+lnMP2sEj67Zycd/sZTHVu/UCVeRGKByj3IpifF895KxPH3bmRQN\nTuEbj7zOdfes1D7xIlFO5R4jxg5N57FbZvLjKyewcVcjF/9qGXf8cwuthzu8jiYivUDlHkN8PuP6\n6UW88I1zuPi0PH79wttcfOcyXt26x+toIhJmKvcYlJOWxJ1zJ3P/TdNwznH9wpV8dfFa3bNVJIqo\n3GPY7JE5PPfVs/jyx0/lmTd2c+4vXubBlZUEtF+8SMRTuce45IQ4vn7BaJ79ylmMHZrOd5e8wTV3\nL2dTTZPX0USkB1TuAsCpQwayeP4Mfn7NRLbvOcBlv/4XP3n2LVra2r2OJiInQeUuR5gZV08t4IWv\nn81VU4Zx99JtnH/HK7y4qdbraCJyglTu8iFZqYn87OqJPPxvZzAgMY7P31fOvz+wmprGVq+jiUg3\nqdylU9OGD+JvX57N/7pwNC9uquO8O5by369u1w26RSKAyl2OKzHexxc/dir//NrZTC3K4odPvckn\nf/cq63fu8zqaiBxHl+VuZveaWZ2Zbejk+TFmttzMDpnZN8MfUfqDwsEp3Pe5Un573WRqmlr55O9e\n5QdPbqRZd38S6Ze6M3K/D7joOM83AF8Gfh6OQNJ/mRmXnZ7PC984mxtmFPGn5Ts4746l/O0N3f1J\npL/pstydc68QLPDOnq9zzpUBGsLFiPTkBG6/YgJLbp3F4NQkbv3LGj5/Xxlb67QZmUh/oTl3OWmT\n/Jk8+aVZ/O9Lx7JyewPn3bGUuQuW88S6ag61a0MyES/F9+WLmdl8YD5AYWFhX7609JL4OB9fmD2C\nKyYN45HVVSxaVclXFq9jUGoiV08tYG6pnxE5A72OKRJzrDtzpWZWDDztnJtwnGN+AOx3znVr7r2k\npMSVl5d3L6VEjEDA8eo7e3hwZSX/fLOW9oDjjBGDuW56IReOzyMxXr8sivSEma12zpV0dVyfjtwl\n+vl8xuyROcwemUNdcyuPlO9k0apKblu0lsGh0fy8aYUUZ6d6HVUkqnU5cjezRcA5QDZQC3wfSABw\nzv3BzPKAciAdCAD7gXHOuePuPKWRe+wIBBzLtu7hwZUVPP9WHR0Bx6xTBzNvWiEXjNNoXuREdHfk\n3q1pmd6gco9NdU2tPFxexaJVVVTvO0j2wESunupn3jQ/RYM1mhfpispd+rWOgGPZ2/U8uLKSFzYF\nR/NnnprNddMLOW9srkbzIp1QuUvEqG1q5eGyKhaXvTeaT+KakgLmlRZSODjF63gi/YrKXSJOR8Dx\nypZ6HlxVyQtv1RJwMHtkNtdNK+S8cbkkxGk0L6Jyl4hW09jKQ2VVPFRWya7GVrIHJnFtSXCljX+Q\nRvMSu1TuEhU6Ao6lW+p4cGUlL26qwxG89+t10/ycO1ajeYk9KneJOrsbD4ZG81XsbmxlSFoS15b4\nmVPq12heYobKXaJWe0eAlzfXs2hVJS9tDo7mzxqZw3XTCzl3zBDiNZqXKKZyl5hQve8gD4dG8zVN\nreSmJ3HD9CJumFFEVmqi1/FEwk7lLjGlvSPAS5vreWBFBUu31JOc4OPqqQXcdOYIhmurA4kiKneJ\nWVtqm/njsu0sWVvN4UCA88fmcvNZIygpysLMvI4n0iMqd4l5dc2t3L+8gvtXVLCv5TAT/ZnMnz2C\nC8fnal5eIpbKXSSkpa2dx1bv5I//2s6Od1soyBrA52cN59pSPwOTtDGqRBaVu8gxOgKO59+qZeGy\nbZTt2EtacjzXTy/iszOLyctI9jqeSLeo3EWOY23lXhYu286zG3bjM+Pyifl8YfYIxuWnex1N5LhU\n7iLdUNXQwr2vbuehsipa2jqYdepgbp49grNH5ejkq/RLKneRE9DYcpgHV1Vy32vbqW06xKjcgXzh\nzBFcMTmfpPg4r+OJHKFyFzkJbe0Bnnp9F/cs28ammmZy0pL4zBlFXD9dF0VJ/6ByF+kB5xyvbn2X\ne5ZtO3JR1DVT/dx05nDd/1U8pRtki/SAmXHmyGzOHJnN5ppmFi7bxkNlVTywsoILxuVy8+wRTNVF\nUdKPaeQu0k11Ta38afkOHlhRSePBw0zyZ3KzLoqSPqZpGZFe0tLWzqOhi6Iq3m3BPyh0UVSJn1Rd\nFCW9TOUu0ss6Ao5/vlnDPcu2s7piL+nJ8Vyni6Kkl6ncRfrQ6oq9LFy2jb9vrCHOZ3xiYj6fnVnM\n6QWZXkeTKBO2E6pmdi9wGVDnnJvwEc8bcCdwCdACfNY5t+bEI4tErqlFWUwtmkrlu8GLoh4ur+Lx\nNdVMLMjgxjOKuez0oSQnaL289J0uR+5mdhawH/hzJ+V+CXAbwXKfDtzpnJve1Qtr5C7RrKn1MI+v\n3sn9Kyp4p/4AmSkJXFvi5/rphRQN1lJKOXlhnZYxs2Lg6U7K/W7gZefcotD3m4FznHO7j/czVe4S\nC5xzLN/2Lg+sqODvG2sJOMfZo3K4cUYR54weQpxPSynlxPTlOvdhQNVR3+8MPXbccheJBWbGzFOy\nmXlKNjWNrSxaVcmiVZXc9KdyhmUO4PoZhcwp8TN4YJLXUSXK9OniXDObb2blZlZeX1/fly8t4rm8\njGS+dv4oXv32x7nr+in4Bw3gZ89t5oyfvMjXHlrH6oq9eLXAQaJPOEbu1YD/qO8LQo99iHNuAbAA\ngtMyYXhtkYiTEOfjktOGcslpQ3m7tpkHVlTw2JpqlqytZnx+OjfOKOLySfmkJGrNvJy8cIzcnwQ+\nbUEzgMau5ttFJGhkbho/vGICK757Lv/3kxPoCDi+/fgbTP+vF/jhUxt5p36/1xElQnVntcwi4Bwg\nG6gFvg8kADjn/hBaCvlb4CKCSyE/55zr8kypTqiKfJhzjvKKvdy/vIJnN+zmcIfjzFOzuWFGEeeN\nHaJtDkQXMYlEurrmVh4uq+LBlZXsamxlaEYy100rZM40P0PSdAVsrFK5i0SJ9o4AL26q4/4VFSx7\new/xPuOiCXncOKOIacMHaWfKGKMtf0WiRHycjwvG53HB+Dy21e/nLysreaS8iqfX72Z0bho3nFHE\nlZOHMVCblslRNHIXiUAH2zp48vVq/ry8go27mhiYFM9VU4Zxw4wiRuWmeR1PepGmZURigHOOdVX7\nuH9FBU+v301be4Dpwwdx4xlFXDg+jwSdgI06KneRGNNwoI2Hy6t4YEUFO/ceJCctiSsnD2NEdip5\nGcnkZw4gLyOZtKR4zdNHMJW7SIzqCDhe2VLP/SsqeHlzHYFj/oqnJsaRl5HM0IwBDM1IZmhGMnnv\nfZ2ZzND0AaQP0AdAf6UTqiIxKs5nfGzMED42Zght7QHqmlupaWxld2MruxsPsrvx/e+Xvb2HuubW\nD30ADEiIC5X++x8CeUc+CJLJzxhAZkqCPgD6MZW7SBRLjPdRkJVCQVZKp8e0dwSoaz50VOkf/MCH\nwfJ39lDbfIiOYz4BkuJ9Hyj7vGN/C8hIZlBqoj4APKJyF4lx8XE+8jMHkJ85oNNjOgKOPfsPsWvf\n+8Vf0xT6ANh3kJXbG6htaqX9mA+AxDgfeRnJFGenUlKURWnxICYXZurGJX1A5S4iXYrzGbnpyeSm\nd35lbCDg2HPgEDWNreza10pN40F2NwV/G9hc08wvn9+Cc5AQZ5w2LIPS4kGUFg+ipDiLzJTEPvyv\niQ06oSoifaKx5TCrKxtYtX0vZTsaWL9zH4c7gv0zOjeN0uFZRwr/eL9FxDqtlhGRfq31cAevV+2j\nbEcDq3bsZU3FXvYfagdgWOYApg0PFv204VmckjNQc/chWi0jIv1ackIc00cMZvqIwUDwxO6mmmZW\nbW+gbEcDy96uZ8na4K0hslISKCkexLTiQZQOH8T4/HRdoNUFjdxFpF9yzrHj3RbKtjewakew8Cve\nbQGCSzWnFGUemcaZXJgZMzc30bSMiESduqZWynYE5+xXbW/grZomnIN4nzF+WAbTit+ft89Kjc6T\ntCp3EYl6Ta2HWV2xl7LtDZTv2Mu6nftoaw8AMHLIwOBUTuhE7fHW+kcSlbuIxJzWwx28Ud14ZN5+\n9Y69NIdO0uZnJDPRn8novDTG5KUxOi+dwkEpxPki60StTqiKSMxJTog7Mi0DwYuvNtc0h1bkNLCx\nupHnNtbw3pg2OcHHyCFpRxV+8E/OwKSIX52jkbuIxJSWtna21u1nU00zm0N/NtU0s2f/oSPHZKUk\nhAo//Ujhj8pN6xc3RNHIXUTkI6QkxnN6QSanF2R+4PF39x9ic+0HC//h8ipa2jqOHFOQNeCoEX46\no3PTGJGT2i+XZarcRUSAwQOTmDkwiZmnZB95LBBwVO87GBrlN7Gpppkttc28vLn+yD46CXHGKTkD\nj4zwR+cG/zksc4CnUzsqdxGRTvh8hn9QCv5BKZw/LvfI44faO9hWf+DICH9LbTPlO/byxLpdR45J\nS4pn1DGFPyYvrc/20VG5i4icoKT4OMYOTWfs0PQPPN7UepgtNc1Hpnc21TTzzPrdPHiw8sgxuelJ\n3Dx7BF+YPaJXM6rcRUTCJD05uE1CSWi1DgSvtK1tem8+Pzi1k5OW1OtZulXuZnYRcCcQByx0zv2/\nY54vAu4FcoAG4Abn3M4wZxURiThmRl7opiZnj8rps9ft8hSvmcUBvwMuBsYB88xs3DGH/Rz4s3Pu\ndOB24CfhDioiIt3XnfU704Ctzrltzrk2YDFwxTHHjANeDH390kc8LyIifag75T4MqDrq+52hx472\nOnBV6OsrgTQzG3zsDzKz+WZWbmbl9fX1J5NXRES6IVwr778JnG1ma4GzgWqg49iDnHMLnHMlzrmS\nnJy+m3sSEYk13TmhWg34j/q+IPTYEc65XYRG7mY2EPiUc25fuEKKiMiJ6c7IvQwYaWbDzSwRmAs8\nefQBZpZtZu/9rO8QXDkjIiIe6bLcnXPtwJeAvwNvAQ875zaa2e1mdnnosHOAzWa2BcgFftxLeUVE\npBu0K6SISATp9zfrMLN6oOIk//VsYE8Y40Q6vR8fpPfjfXovPiga3o8i51yXK1I8K/eeMLPy7nxy\nxQq9Hx+k9+N9ei8+KJbej/63CbGIiPSYyl1EJApFarkv8DpAP6P344P0frxP78UHxcz7EZFz7iIi\ncnyROnIXEZHjiLhyN7OLzGyzmW01s297ncdLZuY3s5fM7E0z22hmX/E6k9fMLM7M1prZ015n8ZqZ\nZZrZo2a2yczeMrMzvM7kFTP7WujvyAYzW2RmyV5n6m0RVe7d3Fs+lrQD33DOjQNmAF+M8fcD4CsE\nr6SW4A12nnPOjQEmEqPvi5kNA74MlDjnJhC86dBcb1P1vogqd7q3t3zMcM7tds6tCX3dTPAv77Hb\nMccMMysALgUWep3Fa2aWAZwF/BHAOdcW45v5xQMDzCweSAF2dXF8xIu0cu/O3vIxycyKgcnASm+T\neOpXwH8AAa+D9APDgXrgv0PTVAvNLNXrUF5wzlUTvFtcJbAbaHTO/cPbVL0v0spdPkJom+XHgK86\n55q8zuMFM7sMqHPOrfY6Sz8RD0wBfu+cmwwcAGLyHJWZZRH8DX84kA+kmtkN3qbqfZFW7l3uLR9r\nzCyBYLH/xTn3uNd5PDQLuNzMdhCcrvu4mT3gbSRP7QR2Oufe+03uUYJlH4vOA7Y75+qdc4eBx4GZ\nHmfqdZFW7l3uLR9LzMwIzqm+5Zy7w+s8XnLOfcc5V+CcKyb4/8WLzrmoH511xjlXA1SZ2ejQQ+cC\nb3oYyUuVwAwzSwn9nTmXGDi53J07MfUbzrl2M3tvb/k44F7n3EaPY3lpFnAj8IaZrQs99l3n3N88\nzCT9x23AX0IDoW3A5zzO4wnn3EozexRYQ3CF2Vpi4EpVXaEqIhKFIm1aRkREukHlLiIShVTuIiJR\nSOUuIhKFVO4iIlFI5S4iEoVU7iIiUUjlLiIShf4H7GbW/ibGMXEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1fb49852ac8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.5764\n"
     ]
    }
   ],
   "source": [
    "tf.reset_default_graph()\n",
    "X_img = tf.placeholder(tf.float32, shape=[None, 32,32,3])\n",
    "X = tf.reshape(X_img, [-1,32,32,3])\n",
    "Y = tf.placeholder(tf.uint8 , shape = [None , 10])\n",
    "conv1 = tf.layers.conv2d(X,32,3,activation=tf.nn.elu)\n",
    "conv11_interaction = tf.layers.conv2d(tf.multiply(X, tf.transpose(X,[0,2,1,3])),32,3,activation=tf.nn.elu)\n",
    "conv11_add = tf.add(conv1,conv11_interaction)\n",
    "conv11_pooling = tf.layers.max_pooling2d(conv11_add,2,2)\n",
    "conv11_flatten = tf.reshape(conv11_pooling,[-1,conv11_pooling.get_shape()[1:4].num_elements()])\n",
    "dense1 = tf.layers.dense(conv11_flatten,128)\n",
    "dense2 = tf.layers.dense(dense1,10)\n",
    "cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=Y, logits=dense2))\n",
    "optimizer = tf.train.AdamOptimizer().minimize(cost)\n",
    "costs = []\n",
    "batch_size = 32\n",
    "training_epochs = 10\n",
    "sess = tf.Session()\n",
    "sess.run(tf.global_variables_initializer())\n",
    "for epoch in range(training_epochs):\n",
    "    avg_cost = 0\n",
    "    total_batch = int(images_train.shape[0] / batch_size)\n",
    "\n",
    "    for i in range(total_batch):\n",
    "        batch_xs, batch_ys = random_batch(batch_size)\n",
    "        feed_dict = {X_img: batch_xs, Y: batch_ys}\n",
    "        c, _ = sess.run([cost, optimizer], feed_dict=feed_dict)\n",
    "        avg_cost += c / total_batch\n",
    "    costs.append(avg_cost)\n",
    "    print('Epoch:', '%04d' % (epoch + 1), 'cost =', '{:.9f}'.format(avg_cost))\n",
    "plt.plot(costs)\n",
    "plt.show()\n",
    "correct_prediction = tf.equal(tf.argmax(dense2, 1), tf.argmax(Y, 1))\n",
    "accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\n",
    "print('Accuracy:', sess.run(accuracy, feed_dict={\n",
    "      X_img: images_test, Y: labels_test}))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 일반적인 CNN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 0001 cost = 1.502513721\n",
      "Epoch: 0002 cost = 1.260068155\n",
      "Epoch: 0003 cost = 1.169101817\n",
      "Epoch: 0004 cost = 1.109899973\n",
      "Epoch: 0005 cost = 1.067770822\n",
      "Epoch: 0006 cost = 1.027605854\n",
      "Epoch: 0007 cost = 0.998852198\n",
      "Epoch: 0008 cost = 0.978262731\n",
      "Epoch: 0009 cost = 0.948536852\n",
      "Epoch: 0010 cost = 0.939814833\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHnxJREFUeJzt3Xl8VfWd//HXJyvZSIAsLFkRJAGUxRBAXEBErdW6VFF0\n8OfSWrpNF2em7fzm1/7aTjvTqd1ta61F68bi0talKgqKC2tANtlkTwJkARJIIPt3/rgRowUS4CYn\n99z38/HgYXLv4Z4392He9+T7/Z5zzDmHiIj4S4TXAUREJPhU7iIiPqRyFxHxIZW7iIgPqdxFRHxI\n5S4i4kMqdxERH1K5i4j4kMpdRMSHorzacWpqqsvNzfVq9yIiIWnVqlVVzrm0jrbzrNxzc3MpLi72\navciIiHJzHZ3ZjsNy4iI+JDKXUTEh1TuIiI+pHIXEfEhlbuIiA91WO5mNtvMKsxsw0men2xmNWa2\npu3Pd4MfU0RETkdnlkI+CjwAPHaKbd52zl0TlEQiInLWOjxyd869BRzshiydsq2ilu+/8D6Nza1e\nRxER6bGCNeY+0czWmtnLZjYiSK95QiUHj/LIu7t4fVN5V+5GRCSkBaPcVwM5zrlRwG+Av55sQzO7\n18yKzay4srLyjHZ2yblpDEzuxZwVe84srYhIGDjrcnfOHXbO1bZ9/Xcg2sxST7LtQ865QudcYVpa\nh5dGOKHICGP6uCze/qCKkoNHzzy4iIiPnXW5m1l/M7O2r4vaXvPA2b7uqUwvzCLCYO5KHb2LiJxI\nZ5ZCzgGWAsPMrNTM7jGzWWY2q22Tm4ANZrYW+DVwq3POdV1kGJgSx5Rh6TxdXEpTiyZWRUQ+qcOl\nkM65GR08/wCBpZLd6taibBY+VsyizRVcOaJ/d+9eRKRHC9kzVKcMSyOjd6wmVkVETiBkyz0qMoJb\nCrNYvLWSsupjXscREelRQrbcAaaPywJg3soSj5OIiPQsIV3umX3iuWRoGvNXltCsiVURkeNCutwB\nZhRlsf9wPYu3ntlJUSIifhTy5T61IIPURE2sioi0F/LlHh0ZwfTCTBZtrmB/Tb3XcUREeoSQL3eA\nW8Zl0epgfrEmVkVEwCflntMvgYuGpDJvZQktrV16cqyISEjwRbkDzCjKpqz6GG9/oIlVERHflPu0\n4Rn0S4jRxKqICD4q95ioCG66IJPXN1VQcVgTqyIS3nxT7hCYWG1pdTy9qtTrKCIinvJVuQ9OS2TC\n4L7MXbmHVk2sikgY81W5Q2BiteTgMd7dXuV1FBERz/iu3K8c0Z8+8dHMXaE17yISvnxX7r2iI7lx\nbCavvr+fyiMNXscREfGE78odAhcTa251PLtaE6siEp58We5D0pMoyu3L3BV76OLbuYqI9Ei+LHeA\nW4uy2HXgKEt3HPA6iohIt/NtuV993gB694rSxKqIhCXflvuHE6uvbNjPwbpGr+OIiHQr35Y7BIZm\nGltaeU4TqyISZnxd7vn9ezM2O4U5mlgVkTDj63IHuLUom+2VdazcdcjrKCIi3cb35X7N+QNIio3S\npYBFJKz4vtzjY6K4fswgXlq/j+qjmlgVkfDQYbmb2WwzqzCzDR1sN87Mms3spuDFC45bi7JobG7l\nL++VeR1FRKRbdObI/VHgqlNtYGaRwE+ABUHIFHQjBiYzKjNZE6siEjY6LHfn3FvAwQ42+yrwLFAR\njFBdYUZRNlvLa1m9p9rrKCIiXe6sx9zNbBBwA/D7Tmx7r5kVm1lxZWX33sj62lEDSYiJ1MSqiISF\nYEyo/hL4lnOutaMNnXMPOecKnXOFaWlpQdh15yXERvGZ0YN4cd1eao41deu+RUS6WzDKvRCYa2a7\ngJuA35nZ9UF43aC7rSib+qZWnl+jiVUR8bezLnfnXJ5zLtc5lws8A3zJOffXs07WBc7LTGbEwN48\nuVwTqyLib51ZCjkHWAoMM7NSM7vHzGaZ2ayujxd8M4qy2bz/CGtLa7yOIiLSZaI62sA5N6OzL+ac\nu/Os0nSD60YP5EcvbWLuij2MzkrxOo6ISJfw/Rmqn5TUK5prRw3g+bV7qW1o9jqOiEiXCLtyh8DQ\nzNHGFp5fs9frKCIiXSIsy310Vgr5/ZO05l1EfCssy93MmFGUzfqyGjaUaWJVRPwnLMsd4Poxg4iN\nitDRu4j4UtiWe3JcNJ8+fwB/W7OXOk2siojPhG25Q+CM1dqGZl5cp4lVEfGXsC73C3L6MDQ9kTkr\nSryOIiISVGFd7mbGrUXZrCmpZuPew17HEREJmrAud4AbxwwiJiqCuSs1sSoi/hH25d4nIYarR/bn\nL++Vcayxxes4IiJBEfblDnBrUTZH6pt5af0+r6OIiASFyh0Yn9eXwakJWvMuIr6hcuejM1ZX7T7E\n1vIjXscRETlrKvc2N44dRHSk6ehdRHxB5d6mX2IsV47oz3Ory6hv0sSqiIQ2lXs7M4qyqTnWxCsb\n9nsdRUTkrKjc25k4uB85/eJ5SkMzIhLiVO7tREQYt47LZsXOg2yvrPU6jojIGVO5f8JNF2QSFWHM\n1dG7iIQwlfsnpCXFMm14Bs+sKqWhWROrIhKaVO4nMKMom0NHm1jwfrnXUUREzojK/QQuGpJKZp84\nrXkXkZClcj+BwMRqFku2H2BXVZ3XcURETpvK/SRuLswiMsKYu1I38hCR0KNyP4mM3r24LD+dZ1aV\n0Njc6nUcEZHT0mG5m9lsM6swsw0nef46M1tnZmvMrNjMLgp+TG/cVpRNVW0jr2/SxKqIhJbOHLk/\nClx1iucXAqOcc6OBu4GHg5CrR7jk3DQGJvfSxKqIhJwOy9059xZw8BTP1zrnXNu3CYA72bahJjLC\nmD4ui7c/qKLk4FGv44iIdFpQxtzN7AYz2wy8RODo/WTb3ds2dFNcWVkZjF13uemFWUQYzNPEqoiE\nkKCUu3PuL865fOB64Ien2O4h51yhc64wLS0tGLvucgNT4pg8LJ35xSU0tWhiVURCQ1BXy7QN4Qw2\ns9Rgvq7XZhRlU3GkgUWbK7yOIiLSKWdd7mY2xMys7euxQCxw4GxftyeZMiyNjN6xupiYiISMqI42\nMLM5wGQg1cxKge8B0QDOuQeBzwJ3mFkTcAy4pd0Eqy9ERUYwvTCLB97YRln1MQalxHkdSUTklDos\nd+fcjA6e/wnwk6Al6qE+LPd5K0v45rRzvY4jInJKOkO1k7L6xnPJ0DSeLi6hWROrItLDqdxPw4yi\nLPbV1LN4a2gs4xSR8KVyPw1TCzJITYzVGasi0uOp3E9DdGQE0wszWbS5gv019V7HERE5KZX7abpl\nXBatDuYX64xVEem5VO6nKadfAhcNSWXeyhJaWn214lNEfETlfgZmFGVTVn2Mtz/QxKqI9Ewq9zMw\nbXgG/RJieGq5JlZFpGdSuZ+BmKgIpo/LYsHGch55d6fXcURE/kGHZ6jKiX398qHsqKzl+y9spK6h\nmS9PGULbJXZERDynI/czFBsVyW9vG8uNYwZx/4Kt/PfLm/HZJXVEJITpyP0sREVGcP/No0iIjeIP\nb+2gtqGZH143kogIHcGLiLdU7mcpIsL4wXUjSOwVxe/f3M7RxhZ+etP5REXqlyIR8Y7KPQjMjG9d\nlU9ibBQ/fXULdQ3N/Oa2McRGRXodTUTClA4vg+jLU4bwg+tGsGBjOZ/7czFHG5u9jiQiYUrlHmR3\nTMzl/ptH8e62Kmb+aQU1x5q8jiQiYUjl3gVuuiCT3942lnWl1cx4aBkHahu8jiQiYUbl3kU+dd4A\n/nhHITuqapn+h6XsqznmdSQRCSMq9y40eVg6j909nvLDDdz84FJ2H6jzOpKIhAmVexcryuvLU58f\nT11DMzc/uJSt5Ue8jiQiYUDl3g3Oz0xh3hcmAnDLH5ayvrTG40Qi4ncq925ybkYST8+aSEJsFDP+\nuIwVOw96HUlEfEzl3o1y+iXw9KyJZPSO5Y7Zy3WjbRHpMir3bjYgOY55X5jI4NREPvfnlbyyYZ/X\nkUTEh1TuHkhNjGXOvRM4b1AyX3pyNc+uKvU6koj4jMrdI8lx0Tx+z3gmntOP+55ey2NLd3kdSUR8\npMNyN7PZZlZhZhtO8vztZrbOzNab2RIzGxX8mP6UEBvFn/7POKYNz+C7f3uf376xzetIIuITnTly\nfxS46hTP7wQudc6dB/wQeCgIucJGr+hIfnf7WK4bPZCfvrqFn7yim36IyNnr8JK/zrm3zCz3FM8v\nafftMiDz7GOFl+jICH4xfTQJsYFrwtfWN/P9z4zQTT9E5IwF+3ru9wAvB/k1w0JEhPGj60eS1HZX\np7qGZv5HN/0QkTMUtHI3sykEyv2iU2xzL3AvQHZ2drB27Rtmxrc/lU9SryjuX7CVusZmfj1DN/0Q\nkdMXlMNCMzsfeBi4zjl34GTbOececs4VOucK09LSgrFr3zEzvnLZUL537XBefV83/RCRM3PW5W5m\n2cBzwEzn3NazjyQAd03K439uOp93t1Vxx59WcLheN/0Qkc7rzFLIOcBSYJiZlZrZPWY2y8xmtW3y\nXaAf8DszW2NmxV2YN6xML8ziNzPGsra0mtv+uIyDdY1eRxKREGFeLbsrLCx0xcX6HOiMNzZXMOuJ\nVWT3jeeJz40no3cvryOJiEfMbJVzrrCj7bQUIwRMyU/nz3cXsbf6GDc/uJSSg0e9jiQiPZzKPURM\nGNyPJz8/gZpjTdz04BK2VeimHyJycir3EDI6K4V5X5hASytM/8MyNpTpph8icmIq9xCT3783T8+a\nSFx0JDMeWkbxLt30Q0T+kco9BOWlBm76kZYUy8w/reCNLRVeRxKRHkblHqIGpgRu+pGbmsBdj6zk\nvvlrOVDb4HUsEekhVO4hLC0plue+eCFfmnwOz68t47KfLWbOij20tuqqkiLhTuUe4uJiIvm3q/L5\n+z9fzLD+SXznufXc9OASNu077HU0EfGQyt0nhmYkMe/eCdx/8yh2HTjKNb95h/98cSO1DboujUg4\nUrn7iJlx0wWZLLrvUqYXZvLwOzuZ9vPFvLJhn24AIhJmVO4+lBIfw3/deD7PfvFCkuOimfXEau75\nc7HObBUJIyp3H7sgpw8vfvUi/uPTBSzbcYBpv1jMb9/YRmNzq9fRRKSLqdx9Lioygs9dPJjXv3kp\nk89N56evbuHqX7/N0u0nvey+iPiAyj1MDEyJ48GZF/DIneNoaG5hxh+X8c35a6jS2ngRX1K5h5kp\n+eks+PqlfHnKObywdi9Tf7aYp5ZrbbyI36jcw1BcTCT/emU+L3/tYvL7J/Hvf1nPZx9cwvt7dSEy\nEb9QuYexIelJzL13Aj+fPoo9B45y7W/e4YdaGy/iCyr3MGdm3Dg2k0X3TebWomxmv7uTy3+2mJfX\na228SChTuQsAyfHR/PiG83j2ixfSJyGGLz65mrseXcmeA1obLxKKVO7yMWOz+/DCVybx/64Zzsqd\nB5n2i8U8sOgDGppbvI4mIqdB5S7/ICoygnsuymPhfZOZWpDO/Qu28qlfvc2S7VVeRxORTlK5y0n1\nT+7F726/gEfuGkdTSyu3/XE535i3hsojWhsv0tOp3KVDU4al89o3LuWrlw3hxXV7mfqzN3li2W6t\njRfpwVTu0im9oiO574phvPy1SxgxMJn/+OsGbvj9Et2kW6SHUrnLaRmSnshTnx/PL28ZTdmho3zm\ngXf4wQtaGy/S06jc5bSZGdePGcTCb05mRlE2jyzZydSfvcm8lXuob9KqGpGeoMNyN7PZZlZhZhtO\n8ny+mS01swYz+5fgR5SeKjk+mh/dcB7PffFC0pN68a1n1zPpvxfx89e2UnGk3ut4ImHNOjoL0cwu\nAWqBx5xzI0/wfDqQA1wPHHLO3d+ZHRcWFrri4uLTTyw9knOOJdsPMPudnSzcXEF0pHHtqIHcPSmP\nkYOSvY4n4htmtso5V9jRdlEdbeCce8vMck/xfAVQYWafPq2E4itmxqQhqUwaksqOylr+vGQXT68q\n5bnVZRTl9eXuSXlMG55BZIR5HVUkLGjMXYJucFoi379uJEu/M5X/e3UBZYeOMeuJVVz60zd4+O0d\nHK5v8jqiiO91a7mb2b1mVmxmxZWVld25a/FAclw0n79kMIv/dTK/v30sA5J78Z8vbeLC/1rE9194\nn90H6ryOKOJbHY65A7QNy7x4ojH3dtv8f6BWY+5yKutKq3nk3V28sHYvLc5xeUEGd0/KY8Lgvphp\nyEakI0EbcxcJpvMzU/jFLaP59qfyeWLZbp5cvofXNpZTMKA3d0/K5dpRA+kVHel1TJGQ15nVMnOA\nyUAqUA58D4gGcM49aGb9gWKgN9BKYGXNcOfc4VO9ro7cBaC+qYW/rSlj9ju72FJ+hNTEGG4fn8Pt\nE7JJT+rldTyRHqezR+6dGpbpCip3ae+TSyljIiO4dtRA7pqUq6WUIu1oWEZCysmWUj67upTxeX25\n+6I8Li/QUkqRztKRu/RYNceamL+yhEeX7KKs+hhZfeO488I8phdmktQr2ut4Ip7QsIz4RnNLKws2\nljP7nZ0U7z5EYmwUNxdmcueFueT0S/A6nki3UrmLL60tqeaRd3fy4rp9WkopYUnlLr5Wfriex5fu\n5qkVezhY18jwAb25+6I8rh01gNgoLaUU/1K5S1g40VLKGUXZ3DY+mwHJcV7HEwk6lbuElfZLKRdt\nqSDCjGkFGdwxMYeJ5/TTkI34hpZCSlhpv5Sy5OBRnli+m/krS3jl/f2ck5bAzAk5fPYCrbKR8KEj\nd/Gt+qYWXly3j8eX7mJtaQ3xMZHcMGYQd0zMZVj/JK/jiZwRDcuItLO2pJrHl+3m+bV7aWxupSiv\nLzMn5HDliP7EROnK1xI6VO4iJ3CorpH5xSU8sXw3JQePkZYUG5iALcqmf7KuZSM9n8pd5BRaWx2L\nt1by2NJdvLm1kggzrhiewcyJOUwcrAlY6bk0oSpyChERxpT8dKbkp7P7QB1PLt/D/OISXt6wn6Hp\nicycmMMNYwZpAlZClo7cRdrUN7Xwwtq9PL5sN+tKa0iIieSGsYEJ2HMzNAErPYOGZUTOwpqSah5b\nuosX1+2jsbmV8Xl9uWNiLleMyCA6UhOw4h2Vu0gQHKxrZN7KEp5Ytpuy6mOkfzgBOz6bjN6agJXu\np3IXCaKWVsebWyp4bOluFm+tJCrCuHJEf2ZOzGF8ni5aJt1HE6oiQRQZYUwtyGBqQQa7qup4Ytlu\n5heX8NL6fZybkcjMCTncMDaTxFj9SEnPoCN3kTN0rDEwAfvYsl1sKDtMYmwUN44dxMwJOQzVBKx0\nEQ3LiHQT5xzvlVTz+NLdvLRuH40tgQnYa84fwGUFGQxK0dUpJXhU7iIeqKptYH5xCU8Xl7Kzqg6A\n/P5JTC1IZ2pBBqMyU3QfWDkrKncRDznn2FFVx8JN5SzcVEHx7kO0tDr6JcQwJT+dqfnpXHxumsbo\n5bSp3EV6kOqjjSzeWsnCTRW8uaWCw/XNREcaEwb3Y2p+4Kg+q2+81zElBKjcRXqo5pZWVu0+xMLN\nFSzcVM72ysDwzdD0xLYVOemMze6j4Rs5IZW7SIjYVVV3vOhX7DxIc6ujT3w0k4elM7UgnUvOTaO3\nrnEjbVTuIiHocH0Tb22tZNGmCt7YUsGho01ERRhFeX25LD+dywsyyE1N8DqmeCho5W5ms4FrgArn\n3MgTPG/Ar4CrgaPAnc651R3tWOUucmotrY739nw0fLO1vBaAwWkJXF6QwWX56RTm9CFK17oJK8Es\n90uAWuCxk5T71cBXCZT7eOBXzrnxHe1Y5S5yekoOHg2svtlcwbIdB2hqcfTuFXV8+Gbyuekkx2v4\nxu+CdvkB59xbZpZ7ik2uI1D8DlhmZilmNsA5t6/TaUWkQ1l947lzUh53TsqjtqGZdz6o5PVNFbyx\nuYLn1+4lMsK4IKcPlxekc1l+BuekJeiaN2EsGItsBwEl7b4vbXtM5S7SRRJjo7hq5ACuGjmA1lbH\nmtJqFm2q4PVN5fz475v58d83k9svnsnD0hmb04cxWSlk9olT2YeRbj2DwszuBe4FyM7O7s5di/hW\nRIQxNrsPY7P78C9XDqOs+hiL2sbp56zYw6NLdgHQLyGG0VkpjMpKOf7f5DgN4/hVMMq9DMhq931m\n22P/wDn3EPAQBMbcg7BvEfmEQSlxzJyQw8wJOTS1tLJl/xHWlFQf/7NoSwUfTrUNTktgdGYKo7MD\nhZ/fvzcxUZqg9YNglPvzwFfMbC6BCdUajbeL9AzRkRGMHJTMyEHJ/NOEHCCw3HJ9aQ1rSqp5b081\nb31QxXPvBY7HYqIiGDGwN6Pbju7HZPUhq6+Gc0JRZ1bLzAEmA6lAOfA9IBrAOfdg21LIB4CrCCyF\nvMs51+EyGK2WEekZnHPsralnzZ5q1pQcYk1JNevLaqhvagWgb0IMozKTGZ3Vh9HZKYzKTCYlPsbj\n1OFLJzGJyBlrbmllS3nbcM6eataWVvNBRe3x4Zy81ITjR/ejslIoGJBEbFSkt6HDhMpdRILqSNtw\nznsl1axtG7+vONIAQExkBMPbDeeMzkohp1+8hnO6gMpdRLqUc459NfUfm6xdX1rDsaYWAPrERzMq\nK4VRmSmMy+1LUV5fTdYGge6hKiJdyswYmBLHwJQ4rj5vABAYztlaXsuadkf3i7d+gHOQFBvF5Px0\npg3PYPIwXQytq+nIXUS6VG1DM0u3H+C1jftZuKmCA3WNx69lP214BpcXZDBQtyLsNA3LiEiP8+HF\n0F7bWM5rG8vZ0XYrwpGDejOtoD/ThmdQMCBJY/WnoHIXkR5vW0Utr20sZ8HG/awpqcY5yOwTx+UF\nGVwxPINxeX2J1lUvP0blLiIhpeJIPQs3VfDaxnLe2VZFY3MryXHRXNY2Tn+J7jkLqNxFJITVNTTz\n9geVLNhYzqLNFVQfbSImMoILhwTG6acVZJDeu5fXMT2hchcRX2huaaV490fj9HsOHgVgVFYKVwzP\nYNrwDIamJ4bNOL3KXUR8xznH1vJaXtu4n9c2lrO2tAaAnH7xTCvI4IoR/bkgx983F1e5i4jv7a+p\n5/VNgSP6pdsP0NjSSt+EmOPj9BcPTSU+xl/j9Cp3EQkrR+qbWLy1ktc2lvPG5goO1zcTGxXBxUNT\nmTY8g6kFGaQmxnod86yp3EUkbDW1tLJi58Hj4/Rl1ccwg5y+8aQmxpKWFEtqYtufpJjjj6W1PRYX\n03MvgqZyFxEhME6/cd9hXt9YwbbKWqqONFBV20BlbQPVR5tO+HcSYiJJbVf2H34AtP9gSGt7vLuH\nfXRtGRERAtfAGTEwmREDk//hucbmVg7WNR4v+8q24q86EnisqraB7ZW1LN/ZwKGTfBDEx0S2+03g\nBB8C7T4YErpxnb7KXUTCVkxUBP2Te9E/ueM1800tgQ+CyiOBD4LAbwAffQhUHmlgZ1UdK3cd4mBd\n4wlfIy468EFwx8QcPnfx4GD/cz5G5S4i0gnRkRFk9O5FRidOnmpu+yCo+PA3gQ8/BNo+GNKSun5i\nV+UuIhJkUZERpPfu5elZtLoij4iID6ncRUR8SOUuIuJDKncRER9SuYuI+JDKXUTEh1TuIiI+pHIX\nEfEhzy4cZmaVwO4z/OupQFUQ44Q6vR8fp/fjI3ovPs4P70eOcy6to408K/ezYWbFnbkqWrjQ+/Fx\nej8+ovfi48Lp/dCwjIiID6ncRUR8KFTL/SGvA/Qwej8+Tu/HR/RefFzYvB8hOeYuIiKnFqpH7iIi\ncgohV+5mdpWZbTGzbWb2ba/zeMnMsszsDTPbaGbvm9nXvM7kNTOLNLP3zOxFr7N4zcxSzOwZM9ts\nZpvMbKLXmbxiZt9o+xnZYGZzzMy7C613k5AqdzOLBH4LfAoYDswws+HepvJUM3Cfc244MAH4cpi/\nHwBfAzZ5HaKH+BXwinMuHxhFmL4vZjYI+Geg0Dk3EogEbvU2VdcLqXIHioBtzrkdzrlGYC5wnceZ\nPOOc2+ecW9329RECP7yDvE3lHTPLBD4NPOx1Fq+ZWTJwCfAnAOdco3Ou2ttUnooC4swsCogH9nqc\np8uFWrkPAkrafV9KGJdZe2aWC4wBlnubxFO/BP4NaPU6SA+QB1QCj7QNUz1sZgleh/KCc64MuB/Y\nA+wDapxzC7xN1fVCrdzlBMwsEXgW+Lpz7rDXebxgZtcAFc65VV5n6SGigLHA751zY4A6ICznqMys\nD4Hf8POAgUCCmf2Tt6m6XqiVexmQ1e77zLbHwpaZRRMo9iedc895ncdDk4DPmNkuAsN1l5nZE95G\n8lQpUOqc+/A3uWcIlH04uhzY6ZyrdM41Ac8BF3qcqcuFWrmvBIaaWZ6ZxRCYFHne40yeMTMjMKa6\nyTn3c6/zeMk59x3nXKZzLpfA/xeLnHO+Pzo7GefcfqDEzIa1PTQV2OhhJC/tASaYWXzbz8xUwmBy\nOcrrAKfDOddsZl8BXiUw4z3bOfe+x7G8NAmYCaw3szVtj/27c+7vHmaSnuOrwJNtB0I7gLs8zuMJ\n59xyM3sGWE1ghdl7hMGZqjpDVUTEh0JtWEZERDpB5S4i4kMqdxERH1K5i4j4kMpdRMSHVO4iIj6k\nchcR8SGVu4iID/0v1LuCDsdgdDUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1fb660fd4a8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.5875\n"
     ]
    }
   ],
   "source": [
    "tf.reset_default_graph()\n",
    "X_img = tf.placeholder(tf.float32, shape=[None, 32,32,3])\n",
    "X = tf.reshape(X_img, [-1,32,32,3])\n",
    "Y = tf.placeholder(tf.uint8 , shape = [None , 10])\n",
    "conv1 = tf.layers.conv2d(X,32,3,activation=tf.nn.elu)\n",
    "conv11_pooling = tf.layers.max_pooling2d(conv1,2,2)\n",
    "conv11_flatten = tf.reshape(conv11_pooling,[-1,conv11_pooling.get_shape()[1:4].num_elements()])\n",
    "dense1 = tf.layers.dense(conv11_flatten,128)\n",
    "dense2 = tf.layers.dense(dense1,10)\n",
    "cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=Y, logits=dense2))\n",
    "optimizer = tf.train.AdamOptimizer().minimize(cost)\n",
    "costs = []\n",
    "batch_size = 32\n",
    "training_epochs = 10\n",
    "sess = tf.Session()\n",
    "sess.run(tf.global_variables_initializer())\n",
    "for epoch in range(training_epochs):\n",
    "    avg_cost = 0\n",
    "    total_batch = int(images_train.shape[0] / batch_size)\n",
    "\n",
    "    for i in range(total_batch):\n",
    "        batch_xs, batch_ys = random_batch(batch_size)\n",
    "        feed_dict = {X_img: batch_xs, Y: batch_ys}\n",
    "        c, _ = sess.run([cost, optimizer], feed_dict=feed_dict)\n",
    "        avg_cost += c / total_batch\n",
    "    costs.append(avg_cost)\n",
    "    print('Epoch:', '%04d' % (epoch + 1), 'cost =', '{:.9f}'.format(avg_cost))\n",
    "plt.plot(costs)\n",
    "plt.show()\n",
    "correct_prediction = tf.equal(tf.argmax(dense2, 1), tf.argmax(Y, 1))\n",
    "accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\n",
    "print('Accuracy:', sess.run(accuracy, feed_dict={\n",
    "      X_img: images_test, Y: labels_test}))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda root]",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}