Kaggle recently hosted a machine learning competition to recognize handwritten digits from 0 to 9. Handwritten digits have been taken from MNIST database (Modified National Institute of Standards and Technology). We decided to use Vowpal Wabbit for learning the pattern of handwritten digits in the training file and apply the learning on the ‘test’ dataset to predict what all digits it represented. Our score on Kaggle was 0.97943 i.e. 97.94% accurate prediction.
Vowpal Wabbit is very easy to install. Its installation on CentOS may not take more than 20 minutes. See the instructions here.
Dataset is in two files: train.csv and test.csv. File, train.csv, contains 42,000 images each of a single handwritten digit from 0-9. Each image is 28 X 28 pixels that is in all 784 pixels–all lined up in one long row. First five lines of train.csv appear as follows:
# No of lines in train.csv $ wc -l train.csv 42001 train.csv # No of lines in test.csv bash-4.1$ wc -l test.csv 28001 test.csv # Show first five lines of train.csv $ head --lines 5 train.csv label,pixel0,pixel1,pixel2,pixel3,pixel4,pixel5,pixel6,pixel7,pixel8,pixel9,pixel10,pixel11,pixel12,pixel13,pixel14,pixel15,pixel16,pixel17,pixel18,pixel19,pixel20,pixel21,pixel22,pixel23,pixel24,pixel25,pixel26,pixel27,pixel28,pixel29,pixel30,pixel31,pixel32,pixel33,pixel34,pixel35,pixel36,pixel37,pixel38,pixel39,pixel40,pixel41,pixel42,pixel43,pixel44,pixel45,pixel46,pixel47,pixel48,pixel49,pixel50,pixel51,pixel52,pixel53,pixel54,pixel55,pixel56,pixel57,pixel58,pixel59,pixel60,pixel61,pixel62,pixel63,pixel64,pixel65,pixel66,pixel67,pixel68,pixel69,pixel70,pixel71,pixel72,pixel73,pixel74,pixel75,pixel76,pixel77,pixel78,pixel79,pixel80,pixel81,pixel82,pixel83,pixel84,pixel85,pixel86,pixel87,pixel88,pixel89,pixel90,pixel91,pixel92,pixel93,pixel94,pixel95,pixel96,pixel97,pixel98,pixel99,pixel100,pixel101,pixel102,pixel103,pixel104,pixel105,pixel106,pixel107,pixel108,pixel109,pixel110,pixel111,pixel112,pixel113,pixel114,pixel115,pixel116,pixel117,pixel118,pixel119,pixel120,pixel121,pixel122,pixel123,pixel124,pixel125,pixel126,pixel127,pixel128,pixel129,pixel130,pixel131,pixel132,pixel133,pixel134,pixel135,pixel136,pixel137,pixel138,pixel139,pixel140,pixel141,pixel142,pixel143,pixel144,pixel145,pixel146,pixel147,pixel148,pixel149,pixel150,pixel151,pixel152,pixel153,pixel154,pixel155,pixel156,pixel157,pixel158,pixel159,pixel160,pixel161,pixel162,pixel163,pixel164,pixel165,pixel166,pixel167,pixel168,pixel169,pixel170,pixel171,pixel172,pixel173,pixel174,pixel175,pixel176,pixel177,pixel178,pixel179,pixel180,pixel181,pixel182,pixel183,pixel184,pixel185,pixel186,pixel187,pixel188,pixel189,pixel190,pixel191,pixel192,pixel193,pixel194,pixel195,pixel196,pixel197,pixel198,pixel199,pixel200,pixel201,pixel202,pixel203,pixel204,pixel205,pixel206,pixel207,pixel208,pixel209,pixel210,pixel211,pixel212,pixel213,pixel214,pixel215,pixel216,pixel217,pixel218,pixel219,pixel220,pixel221,pixel222,pixel223,pixel224,pixel225,pixel226,pixel227,pixel228,pixel229,pixel230,pixel231,pixel232,pixel233,pixel234,pixel235,pixel236,pixel237,pixel238,pixel239,pixel240,pixel241,pixel242,pixel243,pixel244,pixel245,pixel246,pixel247,pixel248,pixel249,pixel250,pixel251,pixel252,pixel253,pixel254,pixel255,pixel256,pixel257,pixel258,pixel259,pixel260,pixel261,pixel262,pixel263,pixel264,pixel265,pixel266,pixel267,pixel268,pixel269,pixel270,pixel271,pixel272,pixel273,pixel274,pixel275,pixel276,pixel277,pixel278,pixel279,pixel280,pixel281,pixel282,pixel283,pixel284,pixel285,pixel286,pixel287,pixel288,pixel289,pixel290,pixel291,pixel292,pixel293,pixel294,pixel295,pixel296,pixel297,pixel298,pixel299,pixel300,pixel301,pixel302,pixel303,pixel304,pixel305,pixel306,pixel307,pixel308,pixel309,pixel310,pixel311,pixel312,pixel313,pixel314,pixel315,pixel316,pixel317,pixel318,pixel319,pixel320,pixel321,pixel322,pixel323,pixel324,pixel325,pixel326,pixel327,pixel328,pixel329,pixel330,pixel331,pixel332,pixel333,pixel334,pixel335,pixel336,pixel337,pixel338,pixel339,pixel340,pixel341,pixel342,pixel343,pixel344,pixel345,pixel346,pixel347,pixel348,pixel349,pixel350,pixel351,pixel352,pixel353,pixel354,pixel355,pixel356,pixel357,pixel358,pixel359,pixel360,pixel361,pixel362,pixel363,pixel364,pixel365,pixel366,pixel367,pixel368,pixel369,pixel370,pixel371,pixel372,pixel373,pixel374,pixel375,pixel376,pixel377,pixel378,pixel379,pixel380,pixel381,pixel382,pixel383,pixel384,pixel385,pixel386,pixel387,pixel388,pixel389,pixel390,pixel391,pixel392,pixel393,pixel394,pixel395,pixel396,pixel397,pixel398,pixel399,pixel400,pixel401,pixel402,pixel403,pixel404,pixel405,pixel406,pixel407,pixel408,pixel409,pixel410,pixel411,pixel412,pixel413,pixel414,pixel415,pixel416,pixel417,pixel418,pixel419,pixel420,pixel421,pixel422,pixel423,pixel424,pixel425,pixel426,pixel427,pixel428,pixel429,pixel430,pixel431,pixel432,pixel433,pixel434,pixel435,pixel436,pixel437,pixel438,pixel439,pixel440,pixel441,pixel442,pixel443,pixel444,pixel445,pixel446,pixel447,pixel448,pixel449,pixel450,pixel451,pixel452,pixel453,pixel454,pixel455,pixel456,pixel457,pixel458,pixel459,pixel460,pixel461,pixel462,pixel463,pixel464,pixel465,pixel466,pixel467,pixel468,pixel469,pixel470,pixel471,pixel472,pixel473,pixel474,pixel475,pixel476,pixel477,pixel478,pixel479,pixel480,pixel481,pixel482,pixel483,pixel484,pixel485,pixel486,pixel487,pixel488,pixel489,pixel490,pixel491,pixel492,pixel493,pixel494,pixel495,pixel496,pixel497,pixel498,pixel499,pixel500,pixel501,pixel502,pixel503,pixel504,pixel505,pixel506,pixel507,pixel508,pixel509,pixel510,pixel511,pixel512,pixel513,pixel514,pixel515,pixel516,pixel517,pixel518,pixel519,pixel520,pixel521,pixel522,pixel523,pixel524,pixel525,pixel526,pixel527,pixel528,pixel529,pixel530,pixel531,pixel532,pixel533,pixel534,pixel535,pixel536,pixel537,pixel538,pixel539,pixel540,pixel541,pixel542,pixel543,pixel544,pixel545,pixel546,pixel547,pixel548,pixel549,pixel550,pixel551,pixel552,pixel553,pixel554,pixel555,pixel556,pixel557,pixel558,pixel559,pixel560,pixel561,pixel562,pixel563,pixel564,pixel565,pixel566,pixel567,pixel568,pixel569,pixel570,pixel571,pixel572,pixel573,pixel574,pixel575,pixel576,pixel577,pixel578,pixel579,pixel580,pixel581,pixel582,pixel583,pixel584,pixel585,pixel586,pixel587,pixel588,pixel589,pixel590,pixel591,pixel592,pixel593,pixel594,pixel595,pixel596,pixel597,pixel598,pixel599,pixel600,pixel601,pixel602,pixel603,pixel604,pixel605,pixel606,pixel607,pixel608,pixel609,pixel610,pixel611,pixel612,pixel613,pixel614,pixel615,pixel616,pixel617,pixel618,pixel619,pixel620,pixel621,pixel622,pixel623,pixel624,pixel625,pixel626,pixel627,pixel628,pixel629,pixel630,pixel631,pixel632,pixel633,pixel634,pixel635,pixel636,pixel637,pixel638,pixel639,pixel640,pixel641,pixel642,pixel643,pixel644,pixel645,pixel646,pixel647,pixel648,pixel649,pixel650,pixel651,pixel652,pixel653,pixel654,pixel655,pixel656,pixel657,pixel658,pixel659,pixel660,pixel661,pixel662,pixel663,pixel664,pixel665,pixel666,pixel667,pixel668,pixel669,pixel670,pixel671,pixel672,pixel673,pixel674,pixel675,pixel676,pixel677,pixel678,pixel679,pixel680,pixel681,pixel682,pixel683,pixel684,pixel685,pixel686,pixel687,pixel688,pixel689,pixel690,pixel691,pixel692,pixel693,pixel694,pixel695,pixel696,pixel697,pixel698,pixel699,pixel700,pixel701,pixel702,pixel703,pixel704,pixel705,pixel706,pixel707,pixel708,pixel709,pixel710,pixel711,pixel712,pixel713,pixel714,pixel715,pixel716,pixel717,pixel718,pixel719,pixel720,pixel721,pixel722,pixel723,pixel724,pixel725,pixel726,pixel727,pixel728,pixel729,pixel730,pixel731,pixel732,pixel733,pixel734,pixel735,pixel736,pixel737,pixel738,pixel739,pixel740,pixel741,pixel742,pixel743,pixel744,pixel745,pixel746,pixel747,pixel748,pixel749,pixel750,pixel751,pixel752,pixel753,pixel754,pixel755,pixel756,pixel757,pixel758,pixel759,pixel760,pixel761,pixel762,pixel763,pixel764,pixel765,pixel766,pixel767,pixel768,pixel769,pixel770,pixel771,pixel772,pixel773,pixel774,pixel775,pixel776,pixel777,pixel778,pixel779,pixel780,pixel781,pixel782,pixel783 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,188,255,94,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,191,250,253,93,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,123,248,253,167,10,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,80,247,253,208,13,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,29,207,253,235,77,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,54,209,253,253,88,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,93,254,253,238,170,17,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,23,210,254,253,159,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,16,209,253,254,240,81,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,27,253,253,254,13,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,20,206,254,254,198,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,168,253,253,196,7,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,20,203,253,248,76,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,22,188,253,245,93,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,103,253,253,191,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,89,240,253,195,25,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,15,220,253,253,80,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,94,253,253,253,94,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,89,251,253,250,131,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,214,218,95,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,18,30,137,137,192,86,72,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,13,86,250,254,254,254,254,217,246,151,32,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,16,179,254,254,254,254,254,254,254,254,254,231,54,15,0,0,0,0,0,0,0,0,0,0,0,0,0,0,72,254,254,254,254,254,254,254,254,254,254,254,254,104,0,0,0,0,0,0,0,0,0,0,0,0,0,61,191,254,254,254,254,254,109,83,199,254,254,254,254,243,85,0,0,0,0,0,0,0,0,0,0,0,0,172,254,254,254,202,147,147,45,0,11,29,200,254,254,254,171,0,0,0,0,0,0,0,0,0,0,0,1,174,254,254,89,67,0,0,0,0,0,0,128,252,254,254,212,76,0,0,0,0,0,0,0,0,0,0,47,254,254,254,29,0,0,0,0,0,0,0,0,83,254,254,254,153,0,0,0,0,0,0,0,0,0,0,80,254,254,240,24,0,0,0,0,0,0,0,0,25,240,254,254,153,0,0,0,0,0,0,0,0,0,0,64,254,254,186,7,0,0,0,0,0,0,0,0,0,166,254,254,224,12,0,0,0,0,0,0,0,0,14,232,254,254,254,29,0,0,0,0,0,0,0,0,0,75,254,254,254,17,0,0,0,0,0,0,0,0,18,254,254,254,254,29,0,0,0,0,0,0,0,0,0,48,254,254,254,17,0,0,0,0,0,0,0,0,2,163,254,254,254,29,0,0,0,0,0,0,0,0,0,48,254,254,254,17,0,0,0,0,0,0,0,0,0,94,254,254,254,200,12,0,0,0,0,0,0,0,16,209,254,254,150,1,0,0,0,0,0,0,0,0,0,15,206,254,254,254,202,66,0,0,0,0,0,21,161,254,254,245,31,0,0,0,0,0,0,0,0,0,0,0,60,212,254,254,254,194,48,48,34,41,48,209,254,254,254,171,0,0,0,0,0,0,0,0,0,0,0,0,0,86,243,254,254,254,254,254,233,243,254,254,254,254,254,86,0,0,0,0,0,0,0,0,0,0,0,0,0,0,114,254,254,254,254,254,254,254,254,254,254,239,86,11,0,0,0,0,0,0,0,0,0,0,0,0,0,0,13,182,254,254,254,254,254,254,254,254,243,70,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,8,76,146,254,255,254,255,146,19,15,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,3,141,139,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,9,254,254,8,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,9,254,254,8,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,9,254,254,106,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,9,254,254,184,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,9,254,254,184,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,9,254,254,184,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,6,185,254,184,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,89,254,184,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,4,146,254,184,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,9,254,254,184,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,9,254,254,184,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,9,254,254,184,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,9,254,254,184,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,9,254,254,184,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,156,254,254,184,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,185,255,255,184,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,185,254,254,184,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,185,254,254,184,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,63,254,254,62,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 4,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,220,179,6,0,0,0,0,0,0,0,0,9,77,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,28,247,17,0,0,0,0,0,0,0,0,27,202,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,242,155,0,0,0,0,0,0,0,0,27,254,63,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,160,207,6,0,0,0,0,0,0,0,27,254,65,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,127,254,21,0,0,0,0,0,0,0,20,239,65,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,77,254,21,0,0,0,0,0,0,0,0,195,65,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,70,254,21,0,0,0,0,0,0,0,0,195,142,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,56,251,21,0,0,0,0,0,0,0,0,195,227,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,222,153,5,0,0,0,0,0,0,0,120,240,13,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,67,251,40,0,0,0,0,0,0,0,94,255,69,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,234,184,0,0,0,0,0,0,0,19,245,69,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,234,169,0,0,0,0,0,0,0,3,199,182,10,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,154,205,4,0,0,26,72,128,203,208,254,254,131,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,61,254,129,113,186,245,251,189,75,56,136,254,73,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,15,216,233,233,159,104,52,0,0,0,38,254,73,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,18,254,73,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,18,254,73,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,5,206,106,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,186,159,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,6,209,101,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
The first column is ‘label’ the digit itself and the rest are 784 pixel (intensity) values, an integer between 0 and 255 (inclusive). These are gray-scale images. For any row, if you remove the label column and wrap pixels in groups of 28 (i.e. a matrix of 28 X 28), one after another, you will see the picture of the digit emerging. In the following R code, we save second row of train.csv in 28X28 matrix to a file :
> # Read all data
> data<-read.csv("train.csv",header=T)
> # Its dimesions?
> dim(data)
[1] 42000 785
> # Just the second row and only pixels not label
> pixels<-data[2,2:785]
> dim(pixels)
[1] 1 784
> # What is the label
> data[2,1]
[1] 0
># Save in a file but while saving round up [0,255] as [0,1]
> write.table(matrix(round(pixels/255),28,28), file = "second.txt", sep = " ", row.names = FALSE, col.names = FALSE)
># Append also to this file label value
>write.table(paste("label",data[2,1]), file = "second.txt", sep = " ", row.names = FALSE, col.names = FALSE,append=T)
>
Saved data in file ‘second.txt’ is as shown below. Pattern of ‘zero’ is obvious.
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 "label 0"
The following R code will directly plot this image:
train <- read.csv("train.csv", header=TRUE)
# Read 2nd row and ignore label column
data<-train[2,2:785]
data<-as.matrix(data,nrow=28,ncol=28)
dim(data)
data<-matrix(data,nrow=28,ncol=28)
dim(data)
##Color ramp def.
colors<-c('white','black')
# Create a function to generate a continuum of colors
# of desired number of colours from white to black
ramp_pal<-colorRampPalette(colors=colors)
# Draw an image of data over a grid of x(1:28), y(1:28)
image(1:28,1:28,data,main="IInd row. Label=0",col=ramp_pal(256))
We will use Vowpal Wabbit to train our classifier. It is a multiclass problem (as against binary) in the sense that label column has 10 classes 0 to 9. Anyone of these 10 classes is possible. Vowpal Wabbit provides a number of algorithms to train multiclass records. One among them is One-Against-All (oaa) or one-against-rest classifier. This technique works as follows:
Suppose for a record label, there are K classes. We then create K binary classifiers. Each classifier makes a test for one class (+ve) against the rest (classed as -ve). Binary classifier treats a particular class (one among K) as positive and others are treated as negative. The pseudcode is as follows (please see Wikipedia):
Inputs:
- 1. Labels, y, where yi ∈ {1, … K} is the label for the sample Xi
- 2. Samples, X
- 3. L, a learner (training algorithm for binary classifiers)
Output:
- A list of classifiers fk for k ∈ {1, …, K}
Procedure:
- For each k in {1, …, K}:
- Step 1: Construct a new label vector yi = 1 only when yi = k else 0 (or −1)
- Step 2: Apply L to (X, y) to obtain fk
Making decisions means applying all binary classifiers to an unseen sample x and predicting the label k for which the corresponding classifier reports the highest confidence score. Weaknesses of OAA strategy are that class distributions of classes may be significantly different. And even if they are balanced, one class typically sees a large number of negatives.
Vowpal Wabbit requires that csv file may first be converted to vw format. The input format for VW files is explained here. In our case all predictor variables (pixels) are numeric. Thus, VW format in our case would look like (five rows shown):
1 |image pixel1:0 pixel2:0 pixel3:0 pixel4:0 pixel5:0 pixel6:0 pixel7:0 pixel8:0 pixel9:0 pixel10:0 pixel11:0 pixel12:0 pixel13:0 pixel14:0 pixel15:0 pixel16:0 pixel17:0 pixel18:0 pixel19:0 pixel20:0 pixel21:0 pixel22:0 pixel23:0 pixel24:0 pixel25:0 pixel26:0 pixel27:0 pixel28:0 pixel29:0 pixel30:0 pixel31:0 pixel32:0 pixel33:0 pixel34:0 pixel35:0 pixel36:0 pixel37:0 pixel38:0 pixel39:0 pixel40:0 pixel41:0 pixel42:0 pixel43:0 pixel44:0 pixel45:0 pixel46:0 pixel47:0 pixel48:0 pixel49:0 pixel50:0 pixel51:0 pixel52:0 pixel53:0 pixel54:0 pixel55:0 pixel56:0 pixel57:0 pixel58:0 pixel59:0 pixel60:0 pixel61:0 pixel62:0 pixel63:0 pixel64:0 pixel65:0 pixel66:0 pixel67:0 pixel68:0 pixel69:0 pixel70:0 pixel71:0 pixel72:0 pixel73:0 pixel74:0 pixel75:0 pixel76:0 pixel77:0 pixel78:0 pixel79:0 pixel80:0 pixel81:0 pixel82:0 pixel83:0 pixel84:0 pixel85:0 pixel86:0 pixel87:0 pixel88:0 pixel89:0 pixel90:0 pixel91:0 pixel92:0 pixel93:0 pixel94:0 pixel95:0 pixel96:0 pixel97:0 pixel98:0 pixel99:0 pixel100:0 pixel101:0 pixel102:0 pixel103:0 pixel104:0 pixel105:0 pixel106:0 pixel107:0 pixel108:0 pixel109:0 pixel110:0 pixel111:0 pixel112:0 pixel113:0 pixel114:0 pixel115:0 pixel116:0 pixel117:0 pixel118:0 pixel119:0 pixel120:0 pixel121:0 pixel122:0 pixel123:0 pixel124:0 pixel125:0 pixel126:0 pixel127:0 pixel128:0 pixel129:0 pixel130:0 pixel131:0 pixel132:0 pixel133:0.737254901960784 pixel134:1 pixel135:0.368627450980392 pixel136:0 pixel137:0 pixel138:0 pixel139:0 pixel140:0 pixel141:0 pixel142:0 pixel143:0 pixel144:0 pixel145:0 pixel146:0 pixel147:0 pixel148:0 pixel149:0 pixel150:0 pixel151:0 pixel152:0 pixel153:0 pixel154:0 pixel155:0 pixel156:0 pixel157:0 pixel158:0 pixel159:0 pixel160:0.749019607843137 pixel161:0.980392156862745 pixel162:0.992156862745098 pixel163:0.364705882352941 pixel164:0 pixel165:0 pixel166:0 pixel167:0 pixel168:0 pixel169:0 pixel170:0 pixel171:0 pixel172:0 pixel173:0 pixel174:0 pixel175:0 pixel176:0 pixel177:0 pixel178:0 pixel179:0 pixel180:0 pixel181:0 pixel182:0 pixel183:0 pixel184:0 pixel185:0 pixel186:0 pixel187:0.482352941176471 pixel188:0.972549019607843 pixel189:0.992156862745098 pixel190:0.654901960784314 pixel191:0.0392156862745098 pixel192:0 pixel193:0 pixel194:0 pixel195:0 pixel196:0 pixel197:0 pixel198:0 pixel199:0 pixel200:0 pixel201:0 pixel202:0 pixel203:0 pixel204:0 pixel205:0 pixel206:0 pixel207:0 pixel208:0 pixel209:0 pixel210:0 pixel211:0 pixel212:0 pixel213:0 pixel214:0.313725490196078 pixel215:0.968627450980392 pixel216:0.992156862745098 pixel217:0.815686274509804 pixel218:0.0509803921568627 pixel219:0 pixel220:0 pixel221:0 pixel222:0 pixel223:0 pixel224:0 pixel225:0 pixel226:0 pixel227:0 pixel228:0 pixel229:0 pixel230:0 pixel231:0 pixel232:0 pixel233:0 pixel234:0 pixel235:0 pixel236:0 pixel237:0 pixel238:0 pixel239:0 pixel240:0 pixel241:0.113725490196078 pixel242:0.811764705882353 pixel243:0.992156862745098 pixel244:0.92156862745098 pixel245:0.301960784313725 pixel246:0 pixel247:0 pixel248:0 pixel249:0 pixel250:0 pixel251:0 pixel252:0 pixel253:0 pixel254:0 pixel255:0 pixel256:0 pixel257:0 pixel258:0 pixel259:0 pixel260:0 pixel261:0 pixel262:0 pixel263:0 pixel264:0 pixel265:0 pixel266:0 pixel267:0 pixel268:0.211764705882353 pixel269:0.819607843137255 pixel270:0.992156862745098 pixel271:0.992156862745098 pixel272:0.345098039215686 pixel273:0 pixel274:0 pixel275:0 pixel276:0 pixel277:0 pixel278:0 pixel279:0 pixel280:0 pixel281:0 pixel282:0 pixel283:0 pixel284:0 pixel285:0 pixel286:0 pixel287:0 pixel288:0 pixel289:0 pixel290:0 pixel291:0 pixel292:0 pixel293:0 pixel294:0 pixel295:0.364705882352941 pixel296:0.996078431372549 pixel297:0.992156862745098 pixel298:0.933333333333333 pixel299:0.666666666666667 pixel300:0.0666666666666667 pixel301:0 pixel302:0 pixel303:0 pixel304:0 pixel305:0 pixel306:0 pixel307:0 pixel308:0 pixel309:0 pixel310:0 pixel311:0 pixel312:0 pixel313:0 pixel314:0 pixel315:0 pixel316:0 pixel317:0 pixel318:0 pixel319:0 pixel320:0 pixel321:0 pixel322:0.0901960784313725 pixel323:0.823529411764706 pixel324:0.996078431372549 pixel325:0.992156862745098 pixel326:0.623529411764706 pixel327:0 pixel328:0 pixel329:0 pixel330:0 pixel331:0 pixel332:0 pixel333:0 pixel334:0 pixel335:0 pixel336:0 pixel337:0 pixel338:0 pixel339:0 pixel340:0 pixel341:0 pixel342:0 pixel343:0 pixel344:0 pixel345:0 pixel346:0 pixel347:0 pixel348:0 pixel349:0.0627450980392157 pixel350:0.819607843137255 pixel351:0.992156862745098 pixel352:0.996078431372549 pixel353:0.941176470588235 pixel354:0.317647058823529 pixel355:0 pixel356:0 pixel357:0 pixel358:0 pixel359:0 pixel360:0 pixel361:0 pixel362:0 pixel363:0 pixel364:0 pixel365:0 pixel366:0 pixel367:0 pixel368:0 pixel369:0 pixel370:0 pixel371:0 pixel372:0 pixel373:0 pixel374:0 pixel375:0 pixel376:0 pixel377:0.105882352941176 pixel378:0.992156862745098 pixel379:0.992156862745098 pixel380:0.996078431372549 pixel381:0.0509803921568627 pixel382:0 pixel383:0 pixel384:0 pixel385:0 pixel386:0 pixel387:0 pixel388:0 pixel389:0 pixel390:0 pixel391:0 pixel392:0 pixel393:0 pixel394:0 pixel395:0 pixel396:0 pixel397:0 pixel398:0 pixel399:0 pixel400:0 pixel401:0 pixel402:0 pixel403:0 pixel404:0.0784313725490196 pixel405:0.807843137254902 pixel406:0.996078431372549 pixel407:0.996078431372549 pixel408:0.776470588235294 pixel409:0.0274509803921569 pixel410:0 pixel411:0 pixel412:0 pixel413:0 pixel414:0 pixel415:0 pixel416:0 pixel417:0 pixel418:0 pixel419:0 pixel420:0 pixel421:0 pixel422:0 pixel423:0 pixel424:0 pixel425:0 pixel426:0 pixel427:0 pixel428:0 pixel429:0 pixel430:0 pixel431:0 pixel432:0.658823529411765 pixel433:0.992156862745098 pixel434:0.992156862745098 pixel435:0.768627450980392 pixel436:0.0274509803921569 pixel437:0 pixel438:0 pixel439:0 pixel440:0 pixel441:0 pixel442:0 pixel443:0 pixel444:0 pixel445:0 pixel446:0 pixel447:0 pixel448:0 pixel449:0 pixel450:0 pixel451:0 pixel452:0 pixel453:0 pixel454:0 pixel455:0 pixel456:0 pixel457:0 pixel458:0 pixel459:0.0784313725490196 pixel460:0.796078431372549 pixel461:0.992156862745098 pixel462:0.972549019607843 pixel463:0.298039215686275 pixel464:0 pixel465:0 pixel466:0 pixel467:0 pixel468:0 pixel469:0 pixel470:0 pixel471:0 pixel472:0 pixel473:0 pixel474:0 pixel475:0 pixel476:0 pixel477:0 pixel478:0 pixel479:0 pixel480:0 pixel481:0 pixel482:0 pixel483:0 pixel484:0 pixel485:0 pixel486:0.0862745098039216 pixel487:0.737254901960784 pixel488:0.992156862745098 pixel489:0.96078431372549 pixel490:0.364705882352941 pixel491:0 pixel492:0 pixel493:0 pixel494:0 pixel495:0 pixel496:0 pixel497:0 pixel498:0 pixel499:0 pixel500:0 pixel501:0 pixel502:0 pixel503:0 pixel504:0 pixel505:0 pixel506:0 pixel507:0 pixel508:0 pixel509:0 pixel510:0 pixel511:0 pixel512:0 pixel513:0 pixel514:0.403921568627451 pixel515:0.992156862745098 pixel516:0.992156862745098 pixel517:0.749019607843137 pixel518:0 pixel519:0 pixel520:0 pixel521:0 pixel522:0 pixel523:0 pixel524:0 pixel525:0 pixel526:0 pixel527:0 pixel528:0 pixel529:0 pixel530:0 pixel531:0 pixel532:0 pixel533:0 pixel534:0 pixel535:0 pixel536:0 pixel537:0 pixel538:0 pixel539:0 pixel540:0 pixel541:0.349019607843137 pixel542:0.941176470588235 pixel543:0.992156862745098 pixel544:0.764705882352941 pixel545:0.0980392156862745 pixel546:0 pixel547:0 pixel548:0 pixel549:0 pixel550:0 pixel551:0 pixel552:0 pixel553:0 pixel554:0 pixel555:0 pixel556:0 pixel557:0 pixel558:0 pixel559:0 pixel560:0 pixel561:0 pixel562:0 pixel563:0 pixel564:0 pixel565:0 pixel566:0 pixel567:0 pixel568:0.0588235294117647 pixel569:0.862745098039216 pixel570:0.992156862745098 pixel571:0.992156862745098 pixel572:0.313725490196078 pixel573:0 pixel574:0 pixel575:0 pixel576:0 pixel577:0 pixel578:0 pixel579:0 pixel580:0 pixel581:0 pixel582:0 pixel583:0 pixel584:0 pixel585:0 pixel586:0 pixel587:0 pixel588:0 pixel589:0 pixel590:0 pixel591:0 pixel592:0 pixel593:0 pixel594:0 pixel595:0 pixel596:0.368627450980392 pixel597:0.992156862745098 pixel598:0.992156862745098 pixel599:0.992156862745098 pixel600:0.368627450980392 pixel601:0 pixel602:0 pixel603:0 pixel604:0 pixel605:0 pixel606:0 pixel607:0 pixel608:0 pixel609:0 pixel610:0 pixel611:0 pixel612:0 pixel613:0 pixel614:0 pixel615:0 pixel616:0 pixel617:0 pixel618:0 pixel619:0 pixel620:0 pixel621:0 pixel622:0 pixel623:0 pixel624:0.349019607843137 pixel625:0.984313725490196 pixel626:0.992156862745098 pixel627:0.980392156862745 pixel628:0.513725490196078 pixel629:0 pixel630:0 pixel631:0 pixel632:0 pixel633:0 pixel634:0 pixel635:0 pixel636:0 pixel637:0 pixel638:0 pixel639:0 pixel640:0 pixel641:0 pixel642:0 pixel643:0 pixel644:0 pixel645:0 pixel646:0 pixel647:0 pixel648:0 pixel649:0 pixel650:0 pixel651:0 pixel652:0 pixel653:0.83921568627451 pixel654:0.854901960784314 pixel655:0.372549019607843 pixel656:0 pixel657:0 pixel658:0 pixel659:0 pixel660:0 pixel661:0 pixel662:0 pixel663:0 pixel664:0 pixel665:0 pixel666:0 pixel667:0 pixel668:0 pixel669:0 pixel670:0 pixel671:0 pixel672:0 pixel673:0 pixel674:0 pixel675:0 pixel676:0 pixel677:0 pixel678:0 pixel679:0 pixel680:0 pixel681:0 pixel682:0 pixel683:0 pixel684:0 pixel685:0 pixel686:0 pixel687:0 pixel688:0 pixel689:0 pixel690:0 pixel691:0 pixel692:0 pixel693:0 pixel694:0 pixel695:0 pixel696:0 pixel697:0 pixel698:0 pixel699:0 pixel700:0 pixel701:0 pixel702:0 pixel703:0 pixel704:0 pixel705:0 pixel706:0 pixel707:0 pixel708:0 pixel709:0 pixel710:0 pixel711:0 pixel712:0 pixel713:0 pixel714:0 pixel715:0 pixel716:0 pixel717:0 pixel718:0 pixel719:0 pixel720:0 pixel721:0 pixel722:0 pixel723:0 pixel724:0 pixel725:0 pixel726:0 pixel727:0 pixel728:0 pixel729:0 pixel730:0 pixel731:0 pixel732:0 pixel733:0 pixel734:0 pixel735:0 pixel736:0 pixel737:0 pixel738:0 pixel739:0 pixel740:0 pixel741:0 pixel742:0 pixel743:0 pixel744:0 pixel745:0 pixel746:0 pixel747:0 pixel748:0 pixel749:0 pixel750:0 pixel751:0 pixel752:0 pixel753:0 pixel754:0 pixel755:0 pixel756:0 pixel757:0 pixel758:0 pixel759:0 pixel760:0 pixel761:0 pixel762:0 pixel763:0 pixel764:0 pixel765:0 pixel766:0 pixel767:0 pixel768:0 pixel769:0 pixel770:0 pixel771:0 pixel772:0 pixel773:0 pixel774:0 pixel775:0 pixel776:0 pixel777:0 pixel778:0 pixel779:0 pixel780:0 pixel781:0 pixel782:0 pixel783:0 pixel784:0 10 |image pixel1:0 pixel2:0 pixel3:0 pixel4:0 pixel5:0 pixel6:0 pixel7:0 pixel8:0 pixel9:0 pixel10:0 pixel11:0 pixel12:0 pixel13:0 pixel14:0 pixel15:0 pixel16:0 pixel17:0 pixel18:0 pixel19:0 pixel20:0 pixel21:0 pixel22:0 pixel23:0 pixel24:0 pixel25:0 pixel26:0 pixel27:0 pixel28:0 pixel29:0 pixel30:0 pixel31:0 pixel32:0 pixel33:0 pixel34:0 pixel35:0 pixel36:0 pixel37:0 pixel38:0 pixel39:0 pixel40:0 pixel41:0 pixel42:0 pixel43:0 pixel44:0 pixel45:0 pixel46:0 pixel47:0 pixel48:0 pixel49:0 pixel50:0 pixel51:0 pixel52:0 pixel53:0 pixel54:0 pixel55:0 pixel56:0 pixel57:0 pixel58:0 pixel59:0 pixel60:0 pixel61:0 pixel62:0 pixel63:0 pixel64:0 pixel65:0 pixel66:0 pixel67:0 pixel68:0 pixel69:0 pixel70:0 pixel71:0 pixel72:0 pixel73:0 pixel74:0 pixel75:0 pixel76:0 pixel77:0 pixel78:0 pixel79:0 pixel80:0 pixel81:0 pixel82:0 pixel83:0 pixel84:0 pixel85:0 pixel86:0 pixel87:0 pixel88:0 pixel89:0 pixel90:0 pixel91:0 pixel92:0 pixel93:0 pixel94:0 pixel95:0 pixel96:0 pixel97:0 pixel98:0 pixel99:0 pixel100:0 pixel101:0 pixel102:0 pixel103:0 pixel104:0 pixel105:0 pixel106:0 pixel107:0 pixel108:0 pixel109:0 pixel110:0 pixel111:0 pixel112:0 pixel113:0 pixel114:0 pixel115:0 pixel116:0 pixel117:0 pixel118:0 pixel119:0 pixel120:0 pixel121:0 pixel122:0 pixel123:0.0705882352941176 pixel124:0.117647058823529 pixel125:0.537254901960784 pixel126:0.537254901960784 pixel127:0.752941176470588 pixel128:0.337254901960784 pixel129:0.282352941176471 pixel130:0.00392156862745098 pixel131:0 pixel132:0 pixel133:0 pixel134:0 pixel135:0 pixel136:0 pixel137:0 pixel138:0 pixel139:0 pixel140:0 pixel141:0 pixel142:0 pixel143:0 pixel144:0 pixel145:0 pixel146:0 pixel147:0 pixel148:0 pixel149:0.0509803921568627 pixel150:0.337254901960784 pixel151:0.980392156862745 pixel152:0.996078431372549 pixel153:0.996078431372549 pixel154:0.996078431372549 pixel155:0.996078431372549 pixel156:0.850980392156863 pixel157:0.964705882352941 pixel158:0.592156862745098 pixel159:0.125490196078431 pixel160:0 pixel161:0 pixel162:0 pixel163:0 pixel164:0 pixel165:0 pixel166:0 pixel167:0 pixel168:0 pixel169:0 pixel170:0 pixel171:0 pixel172:0 pixel173:0 pixel174:0 pixel175:0 pixel176:0.0627450980392157 pixel177:0.701960784313725 pixel178:0.996078431372549 pixel179:0.996078431372549 pixel180:0.996078431372549 pixel181:0.996078431372549 pixel182:0.996078431372549 pixel183:0.996078431372549 pixel184:0.996078431372549 pixel185:0.996078431372549 pixel186:0.996078431372549 pixel187:0.905882352941176 pixel188:0.211764705882353 pixel189:0.0588235294117647 pixel190:0 pixel191:0 pixel192:0 pixel193:0 pixel194:0 pixel195:0 pixel196:0 pixel197:0 pixel198:0 pixel199:0 pixel200:0 pixel201:0 pixel202:0 pixel203:0 pixel204:0.282352941176471 pixel205:0.996078431372549 pixel206:0.996078431372549 pixel207:0.996078431372549 pixel208:0.996078431372549 pixel209:0.996078431372549 pixel210:0.996078431372549 pixel211:0.996078431372549 pixel212:0.996078431372549 pixel213:0.996078431372549 pixel214:0.996078431372549 pixel215:0.996078431372549 pixel216:0.996078431372549 pixel217:0.407843137254902 pixel218:0 pixel219:0 pixel220:0 pixel221:0 pixel222:0 pixel223:0 pixel224:0 pixel225:0 pixel226:0 pixel227:0 pixel228:0 pixel229:0 pixel230:0 pixel231:0.23921568627451 pixel232:0.749019607843137 pixel233:0.996078431372549 pixel234:0.996078431372549 pixel235:0.996078431372549 pixel236:0.996078431372549 pixel237:0.996078431372549 pixel238:0.427450980392157 pixel239:0.325490196078431 pixel240:0.780392156862745 pixel241:0.996078431372549 pixel242:0.996078431372549 pixel243:0.996078431372549 pixel244:0.996078431372549 pixel245:0.952941176470588 pixel246:0.333333333333333 pixel247:0 pixel248:0 pixel249:0 pixel250:0 pixel251:0 pixel252:0 pixel253:0 pixel254:0 pixel255:0 pixel256:0 pixel257:0 pixel258:0 pixel259:0.674509803921569 pixel260:0.996078431372549 pixel261:0.996078431372549 pixel262:0.996078431372549 pixel263:0.792156862745098 pixel264:0.576470588235294 pixel265:0.576470588235294 pixel266:0.176470588235294 pixel267:0 pixel268:0.0431372549019608 pixel269:0.113725490196078 pixel270:0.784313725490196 pixel271:0.996078431372549 pixel272:0.996078431372549 pixel273:0.996078431372549 pixel274:0.670588235294118 pixel275:0 pixel276:0 pixel277:0 pixel278:0 pixel279:0 pixel280:0 pixel281:0 pixel282:0 pixel283:0 pixel284:0 pixel285:0 pixel286:0.00392156862745098 pixel287:0.682352941176471 pixel288:0.996078431372549 pixel289:0.996078431372549 pixel290:0.349019607843137 pixel291:0.262745098039216 pixel292:0 pixel293:0 pixel294:0 pixel295:0 pixel296:0 pixel297:0 pixel298:0.501960784313725 pixel299:0.988235294117647 pixel300:0.996078431372549 pixel301:0.996078431372549 pixel302:0.831372549019608 pixel303:0.298039215686275 pixel304:0 pixel305:0 pixel306:0 pixel307:0 pixel308:0 pixel309:0 pixel310:0 pixel311:0 pixel312:0 pixel313:0 pixel314:0.184313725490196 pixel315:0.996078431372549 pixel316:0.996078431372549 pixel317:0.996078431372549 pixel318:0.113725490196078 pixel319:0 pixel320:0 pixel321:0 pixel322:0 pixel323:0 pixel324:0 pixel325:0 pixel326:0 pixel327:0.325490196078431 pixel328:0.996078431372549 pixel329:0.996078431372549 pixel330:0.996078431372549 pixel331:0.6 pixel332:0 pixel333:0 pixel334:0 pixel335:0 pixel336:0 pixel337:0 pixel338:0 pixel339:0 pixel340:0 pixel341:0 pixel342:0.313725490196078 pixel343:0.996078431372549 pixel344:0.996078431372549 pixel345:0.941176470588235 pixel346:0.0941176470588235 pixel347:0 pixel348:0 pixel349:0 pixel350:0 pixel351:0 pixel352:0 pixel353:0 pixel354:0 pixel355:0.0980392156862745 pixel356:0.941176470588235 pixel357:0.996078431372549 pixel358:0.996078431372549 pixel359:0.6 pixel360:0 pixel361:0 pixel362:0 pixel363:0 pixel364:0 pixel365:0 pixel366:0 pixel367:0 pixel368:0 pixel369:0 pixel370:0.250980392156863 pixel371:0.996078431372549 pixel372:0.996078431372549 pixel373:0.729411764705882 pixel374:0.0274509803921569 pixel375:0 pixel376:0 pixel377:0 pixel378:0 pixel379:0 pixel380:0 pixel381:0 pixel382:0 pixel383:0 pixel384:0.650980392156863 pixel385:0.996078431372549 pixel386:0.996078431372549 pixel387:0.87843137254902 pixel388:0.0470588235294118 pixel389:0 pixel390:0 pixel391:0 pixel392:0 pixel393:0 pixel394:0 pixel395:0 pixel396:0 pixel397:0.0549019607843137 pixel398:0.909803921568627 pixel399:0.996078431372549 pixel400:0.996078431372549 pixel401:0.996078431372549 pixel402:0.113725490196078 pixel403:0 pixel404:0 pixel405:0 pixel406:0 pixel407:0 pixel408:0 pixel409:0 pixel410:0 pixel411:0 pixel412:0.294117647058824 pixel413:0.996078431372549 pixel414:0.996078431372549 pixel415:0.996078431372549 pixel416:0.0666666666666667 pixel417:0 pixel418:0 pixel419:0 pixel420:0 pixel421:0 pixel422:0 pixel423:0 pixel424:0 pixel425:0.0705882352941176 pixel426:0.996078431372549 pixel427:0.996078431372549 pixel428:0.996078431372549 pixel429:0.996078431372549 pixel430:0.113725490196078 pixel431:0 pixel432:0 pixel433:0 pixel434:0 pixel435:0 pixel436:0 pixel437:0 pixel438:0 pixel439:0 pixel440:0.188235294117647 pixel441:0.996078431372549 pixel442:0.996078431372549 pixel443:0.996078431372549 pixel444:0.0666666666666667 pixel445:0 pixel446:0 pixel447:0 pixel448:0 pixel449:0 pixel450:0 pixel451:0 pixel452:0 pixel453:0.00784313725490196 pixel454:0.63921568627451 pixel455:0.996078431372549 pixel456:0.996078431372549 pixel457:0.996078431372549 pixel458:0.113725490196078 pixel459:0 pixel460:0 pixel461:0 pixel462:0 pixel463:0 pixel464:0 pixel465:0 pixel466:0 pixel467:0 pixel468:0.188235294117647 pixel469:0.996078431372549 pixel470:0.996078431372549 pixel471:0.996078431372549 pixel472:0.0666666666666667 pixel473:0 pixel474:0 pixel475:0 pixel476:0 pixel477:0 pixel478:0 pixel479:0 pixel480:0 pixel481:0 pixel482:0.368627450980392 pixel483:0.996078431372549 pixel484:0.996078431372549 pixel485:0.996078431372549 pixel486:0.784313725490196 pixel487:0.0470588235294118 pixel488:0 pixel489:0 pixel490:0 pixel491:0 pixel492:0 pixel493:0 pixel494:0 pixel495:0.0627450980392157 pixel496:0.819607843137255 pixel497:0.996078431372549 pixel498:0.996078431372549 pixel499:0.588235294117647 pixel500:0.00392156862745098 pixel501:0 pixel502:0 pixel503:0 pixel504:0 pixel505:0 pixel506:0 pixel507:0 pixel508:0 pixel509:0 pixel510:0.0588235294117647 pixel511:0.807843137254902 pixel512:0.996078431372549 pixel513:0.996078431372549 pixel514:0.996078431372549 pixel515:0.792156862745098 pixel516:0.258823529411765 pixel517:0 pixel518:0 pixel519:0 pixel520:0 pixel521:0 pixel522:0.0823529411764706 pixel523:0.631372549019608 pixel524:0.996078431372549 pixel525:0.996078431372549 pixel526:0.96078431372549 pixel527:0.12156862745098 pixel528:0 pixel529:0 pixel530:0 pixel531:0 pixel532:0 pixel533:0 pixel534:0 pixel535:0 pixel536:0 pixel537:0 pixel538:0 pixel539:0.235294117647059 pixel540:0.831372549019608 pixel541:0.996078431372549 pixel542:0.996078431372549 pixel543:0.996078431372549 pixel544:0.76078431372549 pixel545:0.188235294117647 pixel546:0.188235294117647 pixel547:0.133333333333333 pixel548:0.16078431372549 pixel549:0.188235294117647 pixel550:0.819607843137255 pixel551:0.996078431372549 pixel552:0.996078431372549 pixel553:0.996078431372549 pixel554:0.670588235294118 pixel555:0 pixel556:0 pixel557:0 pixel558:0 pixel559:0 pixel560:0 pixel561:0 pixel562:0 pixel563:0 pixel564:0 pixel565:0 pixel566:0 pixel567:0 pixel568:0.337254901960784 pixel569:0.952941176470588 pixel570:0.996078431372549 pixel571:0.996078431372549 pixel572:0.996078431372549 pixel573:0.996078431372549 pixel574:0.996078431372549 pixel575:0.913725490196078 pixel576:0.952941176470588 pixel577:0.996078431372549 pixel578:0.996078431372549 pixel579:0.996078431372549 pixel580:0.996078431372549 pixel581:0.996078431372549 pixel582:0.337254901960784 pixel583:0 pixel584:0 pixel585:0 pixel586:0 pixel587:0 pixel588:0 pixel589:0 pixel590:0 pixel591:0 pixel592:0 pixel593:0 pixel594:0 pixel595:0 pixel596:0 pixel597:0.447058823529412 pixel598:0.996078431372549 pixel599:0.996078431372549 pixel600:0.996078431372549 pixel601:0.996078431372549 pixel602:0.996078431372549 pixel603:0.996078431372549 pixel604:0.996078431372549 pixel605:0.996078431372549 pixel606:0.996078431372549 pixel607:0.996078431372549 pixel608:0.937254901960784 pixel609:0.337254901960784 pixel610:0.0431372549019608 pixel611:0 pixel612:0 pixel613:0 pixel614:0 pixel615:0 pixel616:0 pixel617:0 pixel618:0 pixel619:0 pixel620:0 pixel621:0 pixel622:0 pixel623:0 pixel624:0 pixel625:0.0509803921568627 pixel626:0.713725490196078 pixel627:0.996078431372549 pixel628:0.996078431372549 pixel629:0.996078431372549 pixel630:0.996078431372549 pixel631:0.996078431372549 pixel632:0.996078431372549 pixel633:0.996078431372549 pixel634:0.996078431372549 pixel635:0.952941176470588 pixel636:0.274509803921569 pixel637:0 pixel638:0 pixel639:0 pixel640:0 pixel641:0 pixel642:0 pixel643:0 pixel644:0 pixel645:0 pixel646:0 pixel647:0 pixel648:0 pixel649:0 pixel650:0 pixel651:0 pixel652:0 pixel653:0 pixel654:0.0313725490196078 pixel655:0.298039215686275 pixel656:0.572549019607843 pixel657:0.996078431372549 pixel658:1 pixel659:0.996078431372549 pixel660:1 pixel661:0.572549019607843 pixel662:0.0745098039215686 pixel663:0.0588235294117647 pixel664:0 pixel665:0 pixel666:0 pixel667:0 pixel668:0 pixel669:0 pixel670:0 pixel671:0 pixel672:0 pixel673:0 pixel674:0 pixel675:0 pixel676:0 pixel677:0 pixel678:0 pixel679:0 pixel680:0 pixel681:0 pixel682:0 pixel683:0 pixel684:0 pixel685:0 pixel686:0 pixel687:0 pixel688:0 pixel689:0 pixel690:0 pixel691:0 pixel692:0 pixel693:0 pixel694:0 pixel695:0 pixel696:0 pixel697:0 pixel698:0 pixel699:0 pixel700:0 pixel701:0 pixel702:0 pixel703:0 pixel704:0 pixel705:0 pixel706:0 pixel707:0 pixel708:0 pixel709:0 pixel710:0 pixel711:0 pixel712:0 pixel713:0 pixel714:0 pixel715:0 pixel716:0 pixel717:0 pixel718:0 pixel719:0 pixel720:0 pixel721:0 pixel722:0 pixel723:0 pixel724:0 pixel725:0 pixel726:0 pixel727:0 pixel728:0 pixel729:0 pixel730:0 pixel731:0 pixel732:0 pixel733:0 pixel734:0 pixel735:0 pixel736:0 pixel737:0 pixel738:0 pixel739:0 pixel740:0 pixel741:0 pixel742:0 pixel743:0 pixel744:0 pixel745:0 pixel746:0 pixel747:0 pixel748:0 pixel749:0 pixel750:0 pixel751:0 pixel752:0 pixel753:0 pixel754:0 pixel755:0 pixel756:0 pixel757:0 pixel758:0 pixel759:0 pixel760:0 pixel761:0 pixel762:0 pixel763:0 pixel764:0 pixel765:0 pixel766:0 pixel767:0 pixel768:0 pixel769:0 pixel770:0 pixel771:0 pixel772:0 pixel773:0 pixel774:0 pixel775:0 pixel776:0 pixel777:0 pixel778:0 pixel779:0 pixel780:0 pixel781:0 pixel782:0 pixel783:0 pixel784:0 1 |image pixel1:0 pixel2:0 pixel3:0 pixel4:0 pixel5:0 pixel6:0 pixel7:0 pixel8:0 pixel9:0 pixel10:0 pixel11:0 pixel12:0 pixel13:0 pixel14:0 pixel15:0 pixel16:0 pixel17:0 pixel18:0 pixel19:0 pixel20:0 pixel21:0 pixel22:0 pixel23:0 pixel24:0 pixel25:0 pixel26:0 pixel27:0 pixel28:0 pixel29:0 pixel30:0 pixel31:0 pixel32:0 pixel33:0 pixel34:0 pixel35:0 pixel36:0 pixel37:0 pixel38:0 pixel39:0 pixel40:0 pixel41:0 pixel42:0 pixel43:0 pixel44:0 pixel45:0 pixel46:0 pixel47:0 pixel48:0 pixel49:0 pixel50:0 pixel51:0 pixel52:0 pixel53:0 pixel54:0 pixel55:0 pixel56:0 pixel57:0 pixel58:0 pixel59:0 pixel60:0 pixel61:0 pixel62:0 pixel63:0 pixel64:0 pixel65:0 pixel66:0 pixel67:0 pixel68:0 pixel69:0 pixel70:0 pixel71:0 pixel72:0 pixel73:0 pixel74:0 pixel75:0 pixel76:0 pixel77:0 pixel78:0 pixel79:0 pixel80:0 pixel81:0 pixel82:0 pixel83:0 pixel84:0 pixel85:0 pixel86:0 pixel87:0 pixel88:0 pixel89:0 pixel90:0 pixel91:0 pixel92:0 pixel93:0 pixel94:0 pixel95:0 pixel96:0 pixel97:0 pixel98:0 pixel99:0 pixel100:0 pixel101:0 pixel102:0 pixel103:0 pixel104:0 pixel105:0 pixel106:0 pixel107:0 pixel108:0 pixel109:0 pixel110:0 pixel111:0 pixel112:0 pixel113:0 pixel114:0 pixel115:0 pixel116:0 pixel117:0 pixel118:0 pixel119:0 pixel120:0 pixel121:0 pixel122:0 pixel123:0 pixel124:0 pixel125:0.0117647058823529 pixel126:0.552941176470588 pixel127:0.545098039215686 pixel128:0.0117647058823529 pixel129:0 pixel130:0 pixel131:0 pixel132:0 pixel133:0 pixel134:0 pixel135:0 pixel136:0 pixel137:0 pixel138:0 pixel139:0 pixel140:0 pixel141:0 pixel142:0 pixel143:0 pixel144:0 pixel145:0 pixel146:0 pixel147:0 pixel148:0 pixel149:0 pixel150:0 pixel151:0 pixel152:0 pixel153:0.0352941176470588 pixel154:0.996078431372549 pixel155:0.996078431372549 pixel156:0.0313725490196078 pixel157:0 pixel158:0 pixel159:0 pixel160:0 pixel161:0 pixel162:0 pixel163:0 pixel164:0 pixel165:0 pixel166:0 pixel167:0 pixel168:0 pixel169:0 pixel170:0 pixel171:0 pixel172:0 pixel173:0 pixel174:0 pixel175:0 pixel176:0 pixel177:0 pixel178:0 pixel179:0 pixel180:0 pixel181:0.0352941176470588 pixel182:0.996078431372549 pixel183:0.996078431372549 pixel184:0.0313725490196078 pixel185:0 pixel186:0 pixel187:0 pixel188:0 pixel189:0 pixel190:0 pixel191:0 pixel192:0 pixel193:0 pixel194:0 pixel195:0 pixel196:0 pixel197:0 pixel198:0 pixel199:0 pixel200:0 pixel201:0 pixel202:0 pixel203:0 pixel204:0 pixel205:0 pixel206:0 pixel207:0 pixel208:0 pixel209:0.0352941176470588 pixel210:0.996078431372549 pixel211:0.996078431372549 pixel212:0.415686274509804 pixel213:0 pixel214:0 pixel215:0 pixel216:0 pixel217:0 pixel218:0 pixel219:0 pixel220:0 pixel221:0 pixel222:0 pixel223:0 pixel224:0 pixel225:0 pixel226:0 pixel227:0 pixel228:0 pixel229:0 pixel230:0 pixel231:0 pixel232:0 pixel233:0 pixel234:0 pixel235:0 pixel236:0 pixel237:0.0352941176470588 pixel238:0.996078431372549 pixel239:0.996078431372549 pixel240:0.72156862745098 pixel241:0 pixel242:0 pixel243:0 pixel244:0 pixel245:0 pixel246:0 pixel247:0 pixel248:0 pixel249:0 pixel250:0 pixel251:0 pixel252:0 pixel253:0 pixel254:0 pixel255:0 pixel256:0 pixel257:0 pixel258:0 pixel259:0 pixel260:0 pixel261:0 pixel262:0 pixel263:0 pixel264:0 pixel265:0.0352941176470588 pixel266:0.996078431372549 pixel267:0.996078431372549 pixel268:0.72156862745098 pixel269:0 pixel270:0 pixel271:0 pixel272:0 pixel273:0 pixel274:0 pixel275:0 pixel276:0 pixel277:0 pixel278:0 pixel279:0 pixel280:0 pixel281:0 pixel282:0 pixel283:0 pixel284:0 pixel285:0 pixel286:0 pixel287:0 pixel288:0 pixel289:0 pixel290:0 pixel291:0 pixel292:0 pixel293:0.0352941176470588 pixel294:0.996078431372549 pixel295:0.996078431372549 pixel296:0.72156862745098 pixel297:0 pixel298:0 pixel299:0 pixel300:0 pixel301:0 pixel302:0 pixel303:0 pixel304:0 pixel305:0 pixel306:0 pixel307:0 pixel308:0 pixel309:0 pixel310:0 pixel311:0 pixel312:0 pixel313:0 pixel314:0 pixel315:0 pixel316:0 pixel317:0 pixel318:0 pixel319:0 pixel320:0 pixel321:0.0235294117647059 pixel322:0.725490196078431 pixel323:0.996078431372549 pixel324:0.72156862745098 pixel325:0 pixel326:0 pixel327:0 pixel328:0 pixel329:0 pixel330:0 pixel331:0 pixel332:0 pixel333:0 pixel334:0 pixel335:0 pixel336:0 pixel337:0 pixel338:0 pixel339:0 pixel340:0 pixel341:0 pixel342:0 pixel343:0 pixel344:0 pixel345:0 pixel346:0 pixel347:0 pixel348:0 pixel349:0 pixel350:0.349019607843137 pixel351:0.996078431372549 pixel352:0.72156862745098 pixel353:0 pixel354:0 pixel355:0 pixel356:0 pixel357:0 pixel358:0 pixel359:0 pixel360:0 pixel361:0 pixel362:0 pixel363:0 pixel364:0 pixel365:0 pixel366:0 pixel367:0 pixel368:0 pixel369:0 pixel370:0 pixel371:0 pixel372:0 pixel373:0 pixel374:0 pixel375:0 pixel376:0 pixel377:0.0156862745098039 pixel378:0.572549019607843 pixel379:0.996078431372549 pixel380:0.72156862745098 pixel381:0 pixel382:0 pixel383:0 pixel384:0 pixel385:0 pixel386:0 pixel387:0 pixel388:0 pixel389:0 pixel390:0 pixel391:0 pixel392:0 pixel393:0 pixel394:0 pixel395:0 pixel396:0 pixel397:0 pixel398:0 pixel399:0 pixel400:0 pixel401:0 pixel402:0 pixel403:0 pixel404:0 pixel405:0.0352941176470588 pixel406:0.996078431372549 pixel407:0.996078431372549 pixel408:0.72156862745098 pixel409:0 pixel410:0 pixel411:0 pixel412:0 pixel413:0 pixel414:0 pixel415:0 pixel416:0 pixel417:0 pixel418:0 pixel419:0 pixel420:0 pixel421:0 pixel422:0 pixel423:0 pixel424:0 pixel425:0 pixel426:0 pixel427:0 pixel428:0 pixel429:0 pixel430:0 pixel431:0 pixel432:0 pixel433:0.0352941176470588 pixel434:0.996078431372549 pixel435:0.996078431372549 pixel436:0.72156862745098 pixel437:0 pixel438:0 pixel439:0 pixel440:0 pixel441:0 pixel442:0 pixel443:0 pixel444:0 pixel445:0 pixel446:0 pixel447:0 pixel448:0 pixel449:0 pixel450:0 pixel451:0 pixel452:0 pixel453:0 pixel454:0 pixel455:0 pixel456:0 pixel457:0 pixel458:0 pixel459:0 pixel460:0 pixel461:0.0352941176470588 pixel462:0.996078431372549 pixel463:0.996078431372549 pixel464:0.72156862745098 pixel465:0 pixel466:0 pixel467:0 pixel468:0 pixel469:0 pixel470:0 pixel471:0 pixel472:0 pixel473:0 pixel474:0 pixel475:0 pixel476:0 pixel477:0 pixel478:0 pixel479:0 pixel480:0 pixel481:0 pixel482:0 pixel483:0 pixel484:0 pixel485:0 pixel486:0 pixel487:0 pixel488:0 pixel489:0.0352941176470588 pixel490:0.996078431372549 pixel491:0.996078431372549 pixel492:0.72156862745098 pixel493:0 pixel494:0 pixel495:0 pixel496:0 pixel497:0 pixel498:0 pixel499:0 pixel500:0 pixel501:0 pixel502:0 pixel503:0 pixel504:0 pixel505:0 pixel506:0 pixel507:0 pixel508:0 pixel509:0 pixel510:0 pixel511:0 pixel512:0 pixel513:0 pixel514:0 pixel515:0 pixel516:0 pixel517:0.0352941176470588 pixel518:0.996078431372549 pixel519:0.996078431372549 pixel520:0.72156862745098 pixel521:0 pixel522:0 pixel523:0 pixel524:0 pixel525:0 pixel526:0 pixel527:0 pixel528:0 pixel529:0 pixel530:0 pixel531:0 pixel532:0 pixel533:0 pixel534:0 pixel535:0 pixel536:0 pixel537:0 pixel538:0 pixel539:0 pixel540:0 pixel541:0 pixel542:0 pixel543:0 pixel544:0 pixel545:0.611764705882353 pixel546:0.996078431372549 pixel547:0.996078431372549 pixel548:0.72156862745098 pixel549:0 pixel550:0 pixel551:0 pixel552:0 pixel553:0 pixel554:0 pixel555:0 pixel556:0 pixel557:0 pixel558:0 pixel559:0 pixel560:0 pixel561:0 pixel562:0 pixel563:0 pixel564:0 pixel565:0 pixel566:0 pixel567:0 pixel568:0 pixel569:0 pixel570:0 pixel571:0 pixel572:0 pixel573:0.725490196078431 pixel574:1 pixel575:1 pixel576:0.72156862745098 pixel577:0 pixel578:0 pixel579:0 pixel580:0 pixel581:0 pixel582:0 pixel583:0 pixel584:0 pixel585:0 pixel586:0 pixel587:0 pixel588:0 pixel589:0 pixel590:0 pixel591:0 pixel592:0 pixel593:0 pixel594:0 pixel595:0 pixel596:0 pixel597:0 pixel598:0 pixel599:0 pixel600:0 pixel601:0.725490196078431 pixel602:0.996078431372549 pixel603:0.996078431372549 pixel604:0.72156862745098 pixel605:0 pixel606:0 pixel607:0 pixel608:0 pixel609:0 pixel610:0 pixel611:0 pixel612:0 pixel613:0 pixel614:0 pixel615:0 pixel616:0 pixel617:0 pixel618:0 pixel619:0 pixel620:0 pixel621:0 pixel622:0 pixel623:0 pixel624:0 pixel625:0 pixel626:0 pixel627:0 pixel628:0 pixel629:0.725490196078431 pixel630:0.996078431372549 pixel631:0.996078431372549 pixel632:0.72156862745098 pixel633:0 pixel634:0 pixel635:0 pixel636:0 pixel637:0 pixel638:0 pixel639:0 pixel640:0 pixel641:0 pixel642:0 pixel643:0 pixel644:0 pixel645:0 pixel646:0 pixel647:0 pixel648:0 pixel649:0 pixel650:0 pixel651:0 pixel652:0 pixel653:0 pixel654:0 pixel655:0 pixel656:0 pixel657:0.247058823529412 pixel658:0.996078431372549 pixel659:0.996078431372549 pixel660:0.243137254901961 pixel661:0 pixel662:0 pixel663:0 pixel664:0 pixel665:0 pixel666:0 pixel667:0 pixel668:0 pixel669:0 pixel670:0 pixel671:0 pixel672:0 pixel673:0 pixel674:0 pixel675:0 pixel676:0 pixel677:0 pixel678:0 pixel679:0 pixel680:0 pixel681:0 pixel682:0 pixel683:0 pixel684:0 pixel685:0 pixel686:0 pixel687:0 pixel688:0 pixel689:0 pixel690:0 pixel691:0 pixel692:0 pixel693:0 pixel694:0 pixel695:0 pixel696:0 pixel697:0 pixel698:0 pixel699:0 pixel700:0 pixel701:0 pixel702:0 pixel703:0 pixel704:0 pixel705:0 pixel706:0 pixel707:0 pixel708:0 pixel709:0 pixel710:0 pixel711:0 pixel712:0 pixel713:0 pixel714:0 pixel715:0 pixel716:0 pixel717:0 pixel718:0 pixel719:0 pixel720:0 pixel721:0 pixel722:0 pixel723:0 pixel724:0 pixel725:0 pixel726:0 pixel727:0 pixel728:0 pixel729:0 pixel730:0 pixel731:0 pixel732:0 pixel733:0 pixel734:0 pixel735:0 pixel736:0 pixel737:0 pixel738:0 pixel739:0 pixel740:0 pixel741:0 pixel742:0 pixel743:0 pixel744:0 pixel745:0 pixel746:0 pixel747:0 pixel748:0 pixel749:0 pixel750:0 pixel751:0 pixel752:0 pixel753:0 pixel754:0 pixel755:0 pixel756:0 pixel757:0 pixel758:0 pixel759:0 pixel760:0 pixel761:0 pixel762:0 pixel763:0 pixel764:0 pixel765:0 pixel766:0 pixel767:0 pixel768:0 pixel769:0 pixel770:0 pixel771:0 pixel772:0 pixel773:0 pixel774:0 pixel775:0 pixel776:0 pixel777:0 pixel778:0 pixel779:0 pixel780:0 pixel781:0 pixel782:0 pixel783:0 pixel784:0 4 |image pixel1:0 pixel2:0 pixel3:0 pixel4:0 pixel5:0 pixel6:0 pixel7:0 pixel8:0 pixel9:0 pixel10:0 pixel11:0 pixel12:0 pixel13:0 pixel14:0 pixel15:0 pixel16:0 pixel17:0 pixel18:0 pixel19:0 pixel20:0 pixel21:0 pixel22:0 pixel23:0 pixel24:0 pixel25:0 pixel26:0 pixel27:0 pixel28:0 pixel29:0 pixel30:0 pixel31:0 pixel32:0 pixel33:0 pixel34:0 pixel35:0 pixel36:0 pixel37:0 pixel38:0 pixel39:0 pixel40:0 pixel41:0 pixel42:0 pixel43:0 pixel44:0 pixel45:0 pixel46:0 pixel47:0 pixel48:0 pixel49:0 pixel50:0 pixel51:0 pixel52:0 pixel53:0 pixel54:0 pixel55:0 pixel56:0 pixel57:0 pixel58:0 pixel59:0 pixel60:0 pixel61:0 pixel62:0 pixel63:0 pixel64:0 pixel65:0 pixel66:0 pixel67:0 pixel68:0 pixel69:0 pixel70:0 pixel71:0 pixel72:0 pixel73:0 pixel74:0 pixel75:0 pixel76:0 pixel77:0 pixel78:0 pixel79:0 pixel80:0 pixel81:0 pixel82:0 pixel83:0 pixel84:0 pixel85:0 pixel86:0 pixel87:0 pixel88:0 pixel89:0 pixel90:0 pixel91:0 pixel92:0 pixel93:0 pixel94:0 pixel95:0 pixel96:0 pixel97:0 pixel98:0 pixel99:0 pixel100:0 pixel101:0 pixel102:0 pixel103:0 pixel104:0 pixel105:0 pixel106:0 pixel107:0 pixel108:0 pixel109:0 pixel110:0 pixel111:0 pixel112:0 pixel113:0 pixel114:0 pixel115:0 pixel116:0 pixel117:0 pixel118:0 pixel119:0 pixel120:0 pixel121:0 pixel122:0 pixel123:0 pixel124:0 pixel125:0 pixel126:0 pixel127:0 pixel128:0 pixel129:0 pixel130:0 pixel131:0 pixel132:0 pixel133:0 pixel134:0 pixel135:0 pixel136:0 pixel137:0 pixel138:0 pixel139:0 pixel140:0 pixel141:0 pixel142:0 pixel143:0 pixel144:0 pixel145:0 pixel146:0 pixel147:0.862745098039216 pixel148:0.701960784313725 pixel149:0.0235294117647059 pixel150:0 pixel151:0 pixel152:0 pixel153:0 pixel154:0 pixel155:0 pixel156:0 pixel157:0 pixel158:0.0352941176470588 pixel159:0.301960784313725 pixel160:0 pixel161:0 pixel162:0 pixel163:0 pixel164:0 pixel165:0 pixel166:0 pixel167:0 pixel168:0 pixel169:0 pixel170:0 pixel171:0 pixel172:0 pixel173:0 pixel174:0 pixel175:0.109803921568627 pixel176:0.968627450980392 pixel177:0.0666666666666667 pixel178:0 pixel179:0 pixel180:0 pixel181:0 pixel182:0 pixel183:0 pixel184:0 pixel185:0 pixel186:0.105882352941176 pixel187:0.792156862745098 pixel188:0 pixel189:0 pixel190:0 pixel191:0 pixel192:0 pixel193:0 pixel194:0 pixel195:0 pixel196:0 pixel197:0 pixel198:0 pixel199:0 pixel200:0 pixel201:0 pixel202:0 pixel203:0 pixel204:0.949019607843137 pixel205:0.607843137254902 pixel206:0 pixel207:0 pixel208:0 pixel209:0 pixel210:0 pixel211:0 pixel212:0 pixel213:0 pixel214:0.105882352941176 pixel215:0.996078431372549 pixel216:0.247058823529412 pixel217:0 pixel218:0 pixel219:0 pixel220:0 pixel221:0 pixel222:0 pixel223:0 pixel224:0 pixel225:0 pixel226:0 pixel227:0 pixel228:0 pixel229:0 pixel230:0 pixel231:0 pixel232:0.627450980392157 pixel233:0.811764705882353 pixel234:0.0235294117647059 pixel235:0 pixel236:0 pixel237:0 pixel238:0 pixel239:0 pixel240:0 pixel241:0 pixel242:0.105882352941176 pixel243:0.996078431372549 pixel244:0.254901960784314 pixel245:0 pixel246:0 pixel247:0 pixel248:0 pixel249:0 pixel250:0 pixel251:0 pixel252:0 pixel253:0 pixel254:0 pixel255:0 pixel256:0 pixel257:0 pixel258:0 pixel259:0 pixel260:0.498039215686275 pixel261:0.996078431372549 pixel262:0.0823529411764706 pixel263:0 pixel264:0 pixel265:0 pixel266:0 pixel267:0 pixel268:0 pixel269:0 pixel270:0.0784313725490196 pixel271:0.937254901960784 pixel272:0.254901960784314 pixel273:0 pixel274:0 pixel275:0 pixel276:0 pixel277:0 pixel278:0 pixel279:0 pixel280:0 pixel281:0 pixel282:0 pixel283:0 pixel284:0 pixel285:0 pixel286:0 pixel287:0 pixel288:0.301960784313725 pixel289:0.996078431372549 pixel290:0.0823529411764706 pixel291:0 pixel292:0 pixel293:0 pixel294:0 pixel295:0 pixel296:0 pixel297:0 pixel298:0 pixel299:0.764705882352941 pixel300:0.254901960784314 pixel301:0 pixel302:0 pixel303:0 pixel304:0 pixel305:0 pixel306:0 pixel307:0 pixel308:0 pixel309:0 pixel310:0 pixel311:0 pixel312:0 pixel313:0 pixel314:0 pixel315:0 pixel316:0.274509803921569 pixel317:0.996078431372549 pixel318:0.0823529411764706 pixel319:0 pixel320:0 pixel321:0 pixel322:0 pixel323:0 pixel324:0 pixel325:0 pixel326:0 pixel327:0.764705882352941 pixel328:0.556862745098039 pixel329:0 pixel330:0 pixel331:0 pixel332:0 pixel333:0 pixel334:0 pixel335:0 pixel336:0 pixel337:0 pixel338:0 pixel339:0 pixel340:0 pixel341:0 pixel342:0 pixel343:0 pixel344:0.219607843137255 pixel345:0.984313725490196 pixel346:0.0823529411764706 pixel347:0 pixel348:0 pixel349:0 pixel350:0 pixel351:0 pixel352:0 pixel353:0 pixel354:0 pixel355:0.764705882352941 pixel356:0.890196078431372 pixel357:0 pixel358:0 pixel359:0 pixel360:0 pixel361:0 pixel362:0 pixel363:0 pixel364:0 pixel365:0 pixel366:0 pixel367:0 pixel368:0 pixel369:0 pixel370:0 pixel371:0 pixel372:0 pixel373:0.870588235294118 pixel374:0.6 pixel375:0.0196078431372549 pixel376:0 pixel377:0 pixel378:0 pixel379:0 pixel380:0 pixel381:0 pixel382:0 pixel383:0.470588235294118 pixel384:0.941176470588235 pixel385:0.0509803921568627 pixel386:0 pixel387:0 pixel388:0 pixel389:0 pixel390:0 pixel391:0 pixel392:0 pixel393:0 pixel394:0 pixel395:0 pixel396:0 pixel397:0 pixel398:0 pixel399:0 pixel400:0 pixel401:0.262745098039216 pixel402:0.984313725490196 pixel403:0.156862745098039 pixel404:0 pixel405:0 pixel406:0 pixel407:0 pixel408:0 pixel409:0 pixel410:0 pixel411:0.368627450980392 pixel412:1 pixel413:0.270588235294118 pixel414:0 pixel415:0 pixel416:0 pixel417:0 pixel418:0 pixel419:0 pixel420:0 pixel421:0 pixel422:0 pixel423:0 pixel424:0 pixel425:0 pixel426:0 pixel427:0 pixel428:0 pixel429:0 pixel430:0.917647058823529 pixel431:0.72156862745098 pixel432:0 pixel433:0 pixel434:0 pixel435:0 pixel436:0 pixel437:0 pixel438:0 pixel439:0.0745098039215686 pixel440:0.96078431372549 pixel441:0.270588235294118 pixel442:0 pixel443:0 pixel444:0 pixel445:0 pixel446:0 pixel447:0 pixel448:0 pixel449:0 pixel450:0 pixel451:0 pixel452:0 pixel453:0 pixel454:0 pixel455:0 pixel456:0 pixel457:0 pixel458:0.917647058823529 pixel459:0.662745098039216 pixel460:0 pixel461:0 pixel462:0 pixel463:0 pixel464:0 pixel465:0 pixel466:0 pixel467:0.0117647058823529 pixel468:0.780392156862745 pixel469:0.713725490196078 pixel470:0.0392156862745098 pixel471:0 pixel472:0 pixel473:0 pixel474:0 pixel475:0 pixel476:0 pixel477:0 pixel478:0 pixel479:0 pixel480:0 pixel481:0 pixel482:0 pixel483:0 pixel484:0 pixel485:0 pixel486:0.603921568627451 pixel487:0.803921568627451 pixel488:0.0156862745098039 pixel489:0 pixel490:0 pixel491:0.101960784313725 pixel492:0.282352941176471 pixel493:0.501960784313725 pixel494:0.796078431372549 pixel495:0.815686274509804 pixel496:0.996078431372549 pixel497:0.996078431372549 pixel498:0.513725490196078 pixel499:0 pixel500:0 pixel501:0 pixel502:0 pixel503:0 pixel504:0 pixel505:0 pixel506:0 pixel507:0 pixel508:0 pixel509:0 pixel510:0 pixel511:0 pixel512:0 pixel513:0 pixel514:0.23921568627451 pixel515:0.996078431372549 pixel516:0.505882352941176 pixel517:0.443137254901961 pixel518:0.729411764705882 pixel519:0.96078431372549 pixel520:0.984313725490196 pixel521:0.741176470588235 pixel522:0.294117647058824 pixel523:0.219607843137255 pixel524:0.533333333333333 pixel525:0.996078431372549 pixel526:0.286274509803922 pixel527:0 pixel528:0 pixel529:0 pixel530:0 pixel531:0 pixel532:0 pixel533:0 pixel534:0 pixel535:0 pixel536:0 pixel537:0 pixel538:0 pixel539:0 pixel540:0 pixel541:0 pixel542:0.0588235294117647 pixel543:0.847058823529412 pixel544:0.913725490196078 pixel545:0.913725490196078 pixel546:0.623529411764706 pixel547:0.407843137254902 pixel548:0.203921568627451 pixel549:0 pixel550:0 pixel551:0 pixel552:0.149019607843137 pixel553:0.996078431372549 pixel554:0.286274509803922 pixel555:0 pixel556:0 pixel557:0 pixel558:0 pixel559:0 pixel560:0 pixel561:0 pixel562:0 pixel563:0 pixel564:0 pixel565:0 pixel566:0 pixel567:0 pixel568:0 pixel569:0 pixel570:0 pixel571:0 pixel572:0 pixel573:0 pixel574:0 pixel575:0 pixel576:0 pixel577:0 pixel578:0 pixel579:0 pixel580:0.0705882352941176 pixel581:0.996078431372549 pixel582:0.286274509803922 pixel583:0 pixel584:0 pixel585:0 pixel586:0 pixel587:0 pixel588:0 pixel589:0 pixel590:0 pixel591:0 pixel592:0 pixel593:0 pixel594:0 pixel595:0 pixel596:0 pixel597:0 pixel598:0 pixel599:0 pixel600:0 pixel601:0 pixel602:0 pixel603:0 pixel604:0 pixel605:0 pixel606:0 pixel607:0 pixel608:0.0705882352941176 pixel609:0.996078431372549 pixel610:0.286274509803922 pixel611:0 pixel612:0 pixel613:0 pixel614:0 pixel615:0 pixel616:0 pixel617:0 pixel618:0 pixel619:0 pixel620:0 pixel621:0 pixel622:0 pixel623:0 pixel624:0 pixel625:0 pixel626:0 pixel627:0 pixel628:0 pixel629:0 pixel630:0 pixel631:0 pixel632:0 pixel633:0 pixel634:0 pixel635:0 pixel636:0.0196078431372549 pixel637:0.807843137254902 pixel638:0.415686274509804 pixel639:0 pixel640:0 pixel641:0 pixel642:0 pixel643:0 pixel644:0 pixel645:0 pixel646:0 pixel647:0 pixel648:0 pixel649:0 pixel650:0 pixel651:0 pixel652:0 pixel653:0 pixel654:0 pixel655:0 pixel656:0 pixel657:0 pixel658:0 pixel659:0 pixel660:0 pixel661:0 pixel662:0 pixel663:0 pixel664:0 pixel665:0.729411764705882 pixel666:0.623529411764706 pixel667:0 pixel668:0 pixel669:0 pixel670:0 pixel671:0 pixel672:0 pixel673:0 pixel674:0 pixel675:0 pixel676:0 pixel677:0 pixel678:0 pixel679:0 pixel680:0 pixel681:0 pixel682:0 pixel683:0 pixel684:0 pixel685:0 pixel686:0 pixel687:0 pixel688:0 pixel689:0 pixel690:0 pixel691:0 pixel692:0.0235294117647059 pixel693:0.819607843137255 pixel694:0.396078431372549 pixel695:0 pixel696:0 pixel697:0 pixel698:0 pixel699:0 pixel700:0 pixel701:0 pixel702:0 pixel703:0 pixel704:0 pixel705:0 pixel706:0 pixel707:0 pixel708:0 pixel709:0 pixel710:0 pixel711:0 pixel712:0 pixel713:0 pixel714:0 pixel715:0 pixel716:0 pixel717:0 pixel718:0 pixel719:0 pixel720:0 pixel721:0 pixel722:0 pixel723:0 pixel724:0 pixel725:0 pixel726:0 pixel727:0 pixel728:0 pixel729:0 pixel730:0 pixel731:0 pixel732:0 pixel733:0 pixel734:0 pixel735:0 pixel736:0 pixel737:0 pixel738:0 pixel739:0 pixel740:0 pixel741:0 pixel742:0 pixel743:0 pixel744:0 pixel745:0 pixel746:0 pixel747:0 pixel748:0 pixel749:0 pixel750:0 pixel751:0 pixel752:0 pixel753:0 pixel754:0 pixel755:0 pixel756:0 pixel757:0 pixel758:0 pixel759:0 pixel760:0 pixel761:0 pixel762:0 pixel763:0 pixel764:0 pixel765:0 pixel766:0 pixel767:0 pixel768:0 pixel769:0 pixel770:0 pixel771:0 pixel772:0 pixel773:0 pixel774:0 pixel775:0 pixel776:0 pixel777:0 pixel778:0 pixel779:0 pixel780:0 pixel781:0 pixel782:0 pixel783:0 pixel784:0 10 |image pixel1:0 pixel2:0 pixel3:0 pixel4:0 pixel5:0 pixel6:0 pixel7:0 pixel8:0 pixel9:0 pixel10:0 pixel11:0 pixel12:0 pixel13:0 pixel14:0 pixel15:0 pixel16:0 pixel17:0 pixel18:0 pixel19:0 pixel20:0 pixel21:0 pixel22:0 pixel23:0 pixel24:0 pixel25:0 pixel26:0 pixel27:0 pixel28:0 pixel29:0 pixel30:0 pixel31:0 pixel32:0 pixel33:0 pixel34:0 pixel35:0 pixel36:0 pixel37:0 pixel38:0 pixel39:0 pixel40:0 pixel41:0 pixel42:0 pixel43:0 pixel44:0 pixel45:0 pixel46:0 pixel47:0 pixel48:0 pixel49:0 pixel50:0 pixel51:0 pixel52:0 pixel53:0 pixel54:0 pixel55:0 pixel56:0 pixel57:0 pixel58:0 pixel59:0 pixel60:0 pixel61:0 pixel62:0 pixel63:0 pixel64:0 pixel65:0 pixel66:0 pixel67:0 pixel68:0 pixel69:0 pixel70:0 pixel71:0 pixel72:0 pixel73:0 pixel74:0 pixel75:0 pixel76:0 pixel77:0 pixel78:0 pixel79:0 pixel80:0 pixel81:0 pixel82:0 pixel83:0 pixel84:0 pixel85:0 pixel86:0 pixel87:0 pixel88:0 pixel89:0 pixel90:0 pixel91:0 pixel92:0 pixel93:0 pixel94:0 pixel95:0 pixel96:0 pixel97:0 pixel98:0 pixel99:0 pixel100:0 pixel101:0 pixel102:0 pixel103:0 pixel104:0 pixel105:0 pixel106:0 pixel107:0 pixel108:0 pixel109:0 pixel110:0 pixel111:0 pixel112:0 pixel113:0 pixel114:0 pixel115:0 pixel116:0 pixel117:0 pixel118:0 pixel119:0 pixel120:0 pixel121:0 pixel122:0.00392156862745098 pixel123:0.0980392156862745 pixel124:0.509803921568627 pixel125:0.607843137254902 pixel126:0.996078431372549 pixel127:0.996078431372549 pixel128:0.996078431372549 pixel129:0.615686274509804 pixel130:0.117647058823529 pixel131:0.00784313725490196 pixel132:0 pixel133:0 pixel134:0 pixel135:0 pixel136:0 pixel137:0 pixel138:0 pixel139:0 pixel140:0 pixel141:0 pixel142:0 pixel143:0 pixel144:0 pixel145:0 pixel146:0 pixel147:0 pixel148:0 pixel149:0.0313725490196078 pixel150:0.403921568627451 pixel151:0.992156862745098 pixel152:0.992156862745098 pixel153:0.992156862745098 pixel154:0.992156862745098 pixel155:0.992156862745098 pixel156:0.992156862745098 pixel157:0.992156862745098 pixel158:0.992156862745098 pixel159:0.447058823529412 pixel160:0.00784313725490196 pixel161:0 pixel162:0 pixel163:0 pixel164:0 pixel165:0 pixel166:0 pixel167:0 pixel168:0 pixel169:0 pixel170:0 pixel171:0 pixel172:0 pixel173:0 pixel174:0 pixel175:0 pixel176:0.0431372549019608 pixel177:0.815686274509804 pixel178:0.992156862745098 pixel179:0.992156862745098 pixel180:0.992156862745098 pixel181:0.992156862745098 pixel182:0.992156862745098 pixel183:0.992156862745098 pixel184:0.992156862745098 pixel185:0.992156862745098 pixel186:0.992156862745098 pixel187:0.992156862745098 pixel188:0.419607843137255 pixel189:0 pixel190:0 pixel191:0 pixel192:0 pixel193:0 pixel194:0 pixel195:0 pixel196:0 pixel197:0 pixel198:0 pixel199:0 pixel200:0 pixel201:0 pixel202:0 pixel203:0 pixel204:0.12156862745098 pixel205:0.992156862745098 pixel206:0.992156862745098 pixel207:0.992156862745098 pixel208:0.992156862745098 pixel209:0.992156862745098 pixel210:0.992156862745098 pixel211:0.992156862745098 pixel212:0.992156862745098 pixel213:0.992156862745098 pixel214:0.992156862745098 pixel215:0.992156862745098 pixel216:0.843137254901961 pixel217:0.396078431372549 pixel218:0.0117647058823529 pixel219:0 pixel220:0 pixel221:0 pixel222:0 pixel223:0 pixel224:0 pixel225:0 pixel226:0 pixel227:0 pixel228:0 pixel229:0 pixel230:0 pixel231:0.0901960784313725 pixel232:0.823529411764706 pixel233:0.992156862745098 pixel234:0.992156862745098 pixel235:0.992156862745098 pixel236:0.972549019607843 pixel237:0.631372549019608 pixel238:0.870588235294118 pixel239:0.870588235294118 pixel240:0.964705882352941 pixel241:0.992156862745098 pixel242:0.992156862745098 pixel243:0.992156862745098 pixel244:0.992156862745098 pixel245:0.992156862745098 pixel246:0.152941176470588 pixel247:0 pixel248:0 pixel249:0 pixel250:0 pixel251:0 pixel252:0 pixel253:0 pixel254:0 pixel255:0 pixel256:0 pixel257:0 pixel258:0 pixel259:0.533333333333333 pixel260:0.992156862745098 pixel261:0.992156862745098 pixel262:0.992156862745098 pixel263:0.898039215686275 pixel264:0.301960784313725 pixel265:0 pixel266:0 pixel267:0 pixel268:0.274509803921569 pixel269:0.854901960784314 pixel270:0.992156862745098 pixel271:0.992156862745098 pixel272:0.992156862745098 pixel273:0.992156862745098 pixel274:0.843137254901961 pixel275:0.356862745098039 pixel276:0 pixel277:0 pixel278:0 pixel279:0 pixel280:0 pixel281:0 pixel282:0 pixel283:0 pixel284:0 pixel285:0 pixel286:0.0196078431372549 pixel287:0.83921568627451 pixel288:0.992156862745098 pixel289:0.992156862745098 pixel290:0.992156862745098 pixel291:0.764705882352941 pixel292:0 pixel293:0 pixel294:0 pixel295:0 pixel296:0 pixel297:0.407843137254902 pixel298:0.87843137254902 pixel299:0.992156862745098 pixel300:0.992156862745098 pixel301:0.992156862745098 pixel302:0.992156862745098 pixel303:0.843137254901961 pixel304:0.113725490196078 pixel305:0 pixel306:0 pixel307:0 pixel308:0 pixel309:0 pixel310:0 pixel311:0 pixel312:0 pixel313:0 pixel314:0.454901960784314 pixel315:0.992156862745098 pixel316:0.992156862745098 pixel317:0.992156862745098 pixel318:0.968627450980392 pixel319:0.294117647058824 pixel320:0 pixel321:0 pixel322:0 pixel323:0 pixel324:0 pixel325:0 pixel326:0.101960784313725 pixel327:0.784313725490196 pixel328:0.992156862745098 pixel329:0.992156862745098 pixel330:0.992156862745098 pixel331:0.992156862745098 pixel332:0.847058823529412 pixel333:0.0156862745098039 pixel334:0 pixel335:0 pixel336:0 pixel337:0 pixel338:0 pixel339:0 pixel340:0 pixel341:0 pixel342:0.996078431372549 pixel343:0.992156862745098 pixel344:0.992156862745098 pixel345:0.992156862745098 pixel346:0.764705882352941 pixel347:0 pixel348:0 pixel349:0 pixel350:0 pixel351:0 pixel352:0 pixel353:0 pixel354:0 pixel355:0.101960784313725 pixel356:0.784313725490196 pixel357:0.992156862745098 pixel358:0.992156862745098 pixel359:0.992156862745098 pixel360:0.992156862745098 pixel361:0.0196078431372549 pixel362:0 pixel363:0 pixel364:0 pixel365:0 pixel366:0 pixel367:0 pixel368:0 pixel369:0 pixel370:0.996078431372549 pixel371:0.992156862745098 pixel372:0.992156862745098 pixel373:0.992156862745098 pixel374:0.388235294117647 pixel375:0 pixel376:0 pixel377:0 pixel378:0 pixel379:0 pixel380:0 pixel381:0 pixel382:0 pixel383:0 pixel384:0.0980392156862745 pixel385:0.905882352941176 pixel386:0.992156862745098 pixel387:0.992156862745098 pixel388:0.992156862745098 pixel389:0.141176470588235 pixel390:0 pixel391:0 pixel392:0 pixel393:0 pixel394:0 pixel395:0 pixel396:0 pixel397:0 pixel398:0.996078431372549 pixel399:0.992156862745098 pixel400:0.992156862745098 pixel401:0.992156862745098 pixel402:0.388235294117647 pixel403:0 pixel404:0 pixel405:0 pixel406:0 pixel407:0 pixel408:0 pixel409:0 pixel410:0 pixel411:0 pixel412:0 pixel413:0.874509803921569 pixel414:0.992156862745098 pixel415:0.992156862745098 pixel416:0.992156862745098 pixel417:0.505882352941176 pixel418:0 pixel419:0 pixel420:0 pixel421:0 pixel422:0 pixel423:0 pixel424:0 pixel425:0 pixel426:0.996078431372549 pixel427:0.992156862745098 pixel428:0.992156862745098 pixel429:0.992156862745098 pixel430:0.388235294117647 pixel431:0 pixel432:0 pixel433:0 pixel434:0 pixel435:0 pixel436:0 pixel437:0 pixel438:0 pixel439:0 pixel440:0 pixel441:0.498039215686275 pixel442:0.992156862745098 pixel443:0.992156862745098 pixel444:0.992156862745098 pixel445:0.505882352941176 pixel446:0 pixel447:0 pixel448:0 pixel449:0 pixel450:0 pixel451:0 pixel452:0 pixel453:0 pixel454:0.996078431372549 pixel455:0.992156862745098 pixel456:0.992156862745098 pixel457:0.992156862745098 pixel458:0.388235294117647 pixel459:0 pixel460:0 pixel461:0 pixel462:0 pixel463:0 pixel464:0 pixel465:0 pixel466:0 pixel467:0 pixel468:0 pixel469:0.545098039215686 pixel470:0.992156862745098 pixel471:0.992156862745098 pixel472:0.992156862745098 pixel473:0.352941176470588 pixel474:0 pixel475:0 pixel476:0 pixel477:0 pixel478:0 pixel479:0 pixel480:0 pixel481:0 pixel482:0.996078431372549 pixel483:0.992156862745098 pixel484:0.992156862745098 pixel485:0.992156862745098 pixel486:0.388235294117647 pixel487:0 pixel488:0 pixel489:0 pixel490:0 pixel491:0 pixel492:0 pixel493:0 pixel494:0 pixel495:0 pixel496:0.305882352941176 pixel497:0.972549019607843 pixel498:0.992156862745098 pixel499:0.992156862745098 pixel500:0.992156862745098 pixel501:0.0196078431372549 pixel502:0 pixel503:0 pixel504:0 pixel505:0 pixel506:0 pixel507:0 pixel508:0 pixel509:0 pixel510:0.996078431372549 pixel511:0.992156862745098 pixel512:0.992156862745098 pixel513:0.992156862745098 pixel514:0.847058823529412 pixel515:0.133333333333333 pixel516:0 pixel517:0 pixel518:0 pixel519:0 pixel520:0 pixel521:0 pixel522:0 pixel523:0.129411764705882 pixel524:0.596078431372549 pixel525:0.992156862745098 pixel526:0.992156862745098 pixel527:0.992156862745098 pixel528:0.419607843137255 pixel529:0.00392156862745098 pixel530:0 pixel531:0 pixel532:0 pixel533:0 pixel534:0 pixel535:0 pixel536:0 pixel537:0 pixel538:0.807843137254902 pixel539:0.992156862745098 pixel540:0.992156862745098 pixel541:0.992156862745098 pixel542:0.992156862745098 pixel543:0.549019607843137 pixel544:0 pixel545:0 pixel546:0 pixel547:0 pixel548:0 pixel549:0.117647058823529 pixel550:0.545098039215686 pixel551:0.917647058823529 pixel552:0.992156862745098 pixel553:0.992156862745098 pixel554:0.992156862745098 pixel555:0.603921568627451 pixel556:0.00784313725490196 pixel557:0 pixel558:0 pixel559:0 pixel560:0 pixel561:0 pixel562:0 pixel563:0 pixel564:0 pixel565:0 pixel566:0.0627450980392157 pixel567:0.803921568627451 pixel568:0.992156862745098 pixel569:0.992156862745098 pixel570:0.992156862745098 pixel571:0.980392156862745 pixel572:0.815686274509804 pixel573:0.415686274509804 pixel574:0.415686274509804 pixel575:0.415686274509804 pixel576:0.784313725490196 pixel577:0.929411764705882 pixel578:0.992156862745098 pixel579:0.992156862745098 pixel580:0.992156862745098 pixel581:0.992156862745098 pixel582:0.819607843137255 pixel583:0.0862745098039216 pixel584:0 pixel585:0 pixel586:0 pixel587:0 pixel588:0 pixel589:0 pixel590:0 pixel591:0 pixel592:0 pixel593:0 pixel594:0 pixel595:0.32156862745098 pixel596:0.992156862745098 pixel597:0.992156862745098 pixel598:0.992156862745098 pixel599:0.992156862745098 pixel600:0.992156862745098 pixel601:0.992156862745098 pixel602:0.992156862745098 pixel603:0.992156862745098 pixel604:0.992156862745098 pixel605:0.992156862745098 pixel606:0.992156862745098 pixel607:0.992156862745098 pixel608:0.992156862745098 pixel609:0.819607843137255 pixel610:0.0862745098039216 pixel611:0 pixel612:0 pixel613:0 pixel614:0 pixel615:0 pixel616:0 pixel617:0 pixel618:0 pixel619:0 pixel620:0 pixel621:0 pixel622:0 pixel623:0.00392156862745098 pixel624:0.356862745098039 pixel625:0.992156862745098 pixel626:0.992156862745098 pixel627:0.992156862745098 pixel628:0.992156862745098 pixel629:0.992156862745098 pixel630:0.992156862745098 pixel631:0.992156862745098 pixel632:0.992156862745098 pixel633:0.992156862745098 pixel634:0.992156862745098 pixel635:0.835294117647059 pixel636:0.352941176470588 pixel637:0.0274509803921569 pixel638:0 pixel639:0 pixel640:0 pixel641:0 pixel642:0 pixel643:0 pixel644:0 pixel645:0 pixel646:0 pixel647:0 pixel648:0 pixel649:0 pixel650:0 pixel651:0 pixel652:0.00392156862745098 pixel653:0.0705882352941176 pixel654:0.505882352941176 pixel655:0.815686274509804 pixel656:0.992156862745098 pixel657:0.992156862745098 pixel658:0.992156862745098 pixel659:0.992156862745098 pixel660:0.623529411764706 pixel661:0.505882352941176 pixel662:0.352941176470588 pixel663:0.0156862745098039 pixel664:0 pixel665:0 pixel666:0 pixel667:0 pixel668:0 pixel669:0 pixel670:0 pixel671:0 pixel672:0 pixel673:0 pixel674:0 pixel675:0 pixel676:0 pixel677:0 pixel678:0 pixel679:0 pixel680:0 pixel681:0 pixel682:0 pixel683:0 pixel684:0 pixel685:0 pixel686:0 pixel687:0 pixel688:0 pixel689:0 pixel690:0 pixel691:0 pixel692:0 pixel693:0 pixel694:0 pixel695:0 pixel696:0 pixel697:0 pixel698:0 pixel699:0 pixel700:0 pixel701:0 pixel702:0 pixel703:0 pixel704:0 pixel705:0 pixel706:0 pixel707:0 pixel708:0 pixel709:0 pixel710:0 pixel711:0 pixel712:0 pixel713:0 pixel714:0 pixel715:0 pixel716:0 pixel717:0 pixel718:0 pixel719:0 pixel720:0 pixel721:0 pixel722:0 pixel723:0 pixel724:0 pixel725:0 pixel726:0 pixel727:0 pixel728:0 pixel729:0 pixel730:0 pixel731:0 pixel732:0 pixel733:0 pixel734:0 pixel735:0 pixel736:0 pixel737:0 pixel738:0 pixel739:0 pixel740:0 pixel741:0 pixel742:0 pixel743:0 pixel744:0 pixel745:0 pixel746:0 pixel747:0 pixel748:0 pixel749:0 pixel750:0 pixel751:0 pixel752:0 pixel753:0 pixel754:0 pixel755:0 pixel756:0 pixel757:0 pixel758:0 pixel759:0 pixel760:0 pixel761:0 pixel762:0 pixel763:0 pixel764:0 pixel765:0 pixel766:0 pixel767:0 pixel768:0 pixel769:0 pixel770:0 pixel771:0 pixel772:0 pixel773:0 pixel774:0 pixel775:0 pixel776:0 pixel777:0 pixel778:0 pixel779:0 pixel780:0 pixel781:0 pixel782:0 pixel783:0 pixel784:0
Format is simple (see below). ‘labelValue’ is one of [1,10]; we write digit 0 as 10. Namespace is any name for a collection of features. In this particular case all features are similar (just pixel intensities) so they get all clubbed under one namespace. You may read about distinction between namespace and features here. I may mention in passing that a namespace is identified by the first letter of its name (‘ i ‘ in our case) rather than the full name (‘image’).
labelValue |namespaceName var1:value1 var2:value2 var3:value3 var4:value4 var5:value5
We, therefore, need to convert ‘train.csv’ and ‘test.csv’ files to VW format. Even though, test.csv, does not contain any digit identifier (label), we arbitrarily put digit of 1. While making prediction this label is ignored by Vowpal Wabbit. The conversion script in R is as below. I have taken the code from here and made minor changes.
# Data conversion to VW format
# ============================
# Read train.csv
data<-read.csv("train.csv",header=TRUE)
# Read just label values
y<-data[,1]
# Label 0 be written as label 10 (VW's requirement)
y[y[]==0]<-10
# Scale rest of data between [0,1]
x<-(data[,-1])/255
# Indicies where in row 1, x is not zero.
# Just to see data
which(x[1,]>0)
x[,which(x[1,]>0)]
# Function to convert csv file VW format
write_vwformat = function( filename, y, x ) {
# Open file where to write output to
f = file( filename, 'w' )
# loop over all rows
for ( i in 1:nrow( x )) {
# How many columns are there.
# Generate a vector of all column index numbers
indexes = 1:ncol(x)
# What values exist at those index numbers
# Store all values in a vector
values = x[i, indexes]
# Create a prefix for all column names: pixel1, pixel2,...
prefix = paste("pixel", indexes, sep="")
# Concatenate to each prefix, column values.pixel1:0 pixel2:0.987..." )
iv_pairs = paste( prefix, values, sep = ":", collapse = " " )
# add label in the front and newline at the end
output_line = paste( y[i], " |image ", iv_pairs, "\n", sep = "" )
# write to file
cat( output_line, file = f )
# print progress
if ( i %% 1000 == 0 ) {
print( i )
}
}
# Close the connection
close( f )
}
# Invoke the function to convert
write_vwformat("train.vw",y,x)
Once ‘train.vw’ file is available, modeling can begin. We have used One-against-all process. You may please refer here for an example of its usage. The model building code (first line) and its output (rest of lines) are as below:
$ vw -d train.vw --cache_file mnist_cache --passes 25 --oaa 10 -f mnist.model -b20 -q ii creating quadratic features for pairs: ii final_regressor = mnist.model Num weight bits = 20 learning rate = 0.5 initial_t = 0 power_t = 0.5 decay_learning_rate = 1 can't open: mnist_cache, error = No such file or directory creating cache_file = mnist_cache Reading datafile = train.vw num sources = 1 average since example example current current current loss last counter weight label predict features 0.000000 0.000000 1 1.0 1 1 9507 0.500000 1.000000 2 2.0 10 1 60271 0.750000 1.000000 4 4.0 4 1 12883 0.875000 1.000000 8 8.0 3 7 31863 0.812500 0.750000 16 16.0 2 1 44733 0.812500 0.812500 32 32.0 2 3 22953 0.671875 0.531250 64 64.0 3 3 28731 0.523438 0.375000 128 128.0 10 10 62251 0.417969 0.312500 256 256.0 9 9 16003 0.289062 0.160156 512 512.0 2 2 14521 0.221680 0.154297 1024 1024.0 8 8 37057 0.162598 0.103516 2048 2048.0 5 5 23871 0.119873 0.077148 4096 4096.0 4 4 13573 0.094482 0.069092 8192 8192.0 4 4 9313 0.072754 0.051025 16384 16384.0 8 8 19741 0.056641 0.040527 32768 32768.0 1 1 6481 0.044087 0.044087 65536 65536.0 6 6 19741 h 0.034952 0.025817 131072 131072.0 5 5 9121 h 0.030728 0.026504 262144 262144.0 9 9 26733 h 0.027157 0.023586 524288 524288.0 2 2 35533 h finished run number of examples per pass = 37800 passes used = 15 weighted example sum = 567000 weighted label sum = 0 average loss = 0.0230952 h total feature number = 13850936880
An explanation of arguments to vw is in order. Flag --cache_file mnist_cache first converts train.vw to a binary file for future faster processing. Next time, we go through the model building again, this cache file and not the train.vw file will be read (if by and large arguments to vw remain the same). Argument --passes 25 is the number of passes and --oaa 10 refers to oaa learning algorithm with 10 classes (1 to 10). -q ii creates interaction between variables in the two referred to namespaces which here are the same i.e. ‘image’ namespace. An interaction variable is created from two variables ‘A’ and ‘B’ by multiplying the values of ‘A’ and ‘B’. It is a general technique and you may read more about it in Wikipedia. -f mnist.model refers to file where model will be saved. -b 22 refers to number of bits in the feature table. Default number is 18 but as we have increased the number of features much more by introducing interaction features, value of ‘-b’ has been increased to 22. You may read more about VW command line arguments here. Model construction can be done on an 8GB machine and does not consume much time or RAM.
Once model is ready, we need to convert ‘test.csv’ to ‘test.vw’ with an arbitrary label value. The code for conversion is as below.
# For comments read earlier R code
# ============================
# Read test data and scale it
data<-read.csv("test.csv",header=TRUE)
x<-(data[,])/255
# Function to convert csv file VW format
write_vwformat = function( filename, y, x ) {
# open file where to write output to
f = file( filename, 'w' )
# loop over all rows
for ( i in 1:nrow( x )) {
indexes = 1:ncol(x)
values = x[i, indexes]
prefix = paste("pixel", indexes, sep="")
iv_pairs = paste( prefix, values, sep = ":", collapse = " " )
output_line = paste( "1 |image ", iv_pairs, "\n", sep = "" )
cat( output_line, file = f )
if ( i %% 1000 == 0 ) {
print( i )
}
}
close( f )
}
write_vwformat("test.vw",y,x)
Let us now make the prediction for each image listed in test.vw. The vw prediction command is as below.
$ vw -t -i mnist.model test.vw -p predict.txt
Flag -t is for test file; -i specifies the model file created earlier and class predictions [1,10] are saved to ‘predict.txt’ file. Kaggle requires submission in a particular format. R code for this is below. While doing so, we reconvert 10 back to 0.
data<-read.csv("predict.txt",header=FALSE)
# Reconvert 10 to 0
data[data[]==10]<-0
# Create a dataframe with one column
# of IDs 1 to 28000
cpy<-as.data.frame(1:28000)
# Create a second column of predicted digits
cpy[2]<-data
# Assign column names
colnames(cpy)<-c("ImageId","Label")
# Write to file for submission
write.csv(cpy,file="submit.csv",row.names=F,quote=F)
File ‘submit.csv’ can be submitted to Kaggle. The score is 0.97943.
Kaggle Accuracy score
Performance Evaluation
One can attempt performance evaluation with a test dataset when digits that images represent in this set are known. From our ‘train.csv’, we sample 10% of records (without replacement). There are many ways this can be done. For example, sample() function in R can be used to sample data from ‘train.csv’ and store the sampled records to a file. We have used bash script to extract 4200 lines from ‘train.csv’ without replacement. The bash script is below (it does take time, but is in the spirit of Vowpal Wabbit in that the complete file is not read to RAM beforehand):
#!/bin/bash
#
# Generate cross-validation file of specified lines (size) with replacement
# Creates two files: training and tovalidate.
# Original file remains undisturbed.
# Usage: ./cross_validate.sh (no arguments: Specify two below)
####User to fill in following two constants##########
datadir="Documents/kaggle"
originalfile="train.csv"
# Get home folder
cd ~
homefolder=`pwd`
# your data folder
datadir=$homefolder/$datadir
echo "Your data directory is $datadir"
echo "Your original data file is $originalfile"
echo -n "Press Enter to continue or ctrl+c to terminate"; read x
#########Begin#############
trainfile="training"
samplefile="tovalidate"
cd $datadir
# Delete sample file, if it exists, & recreate it
echo "Deleting $samplefile file"
rm -f $datadir/$samplefile
touch $datadir/$samplefile
# Delete training file,if it exists, and recreate copy of originalfile
echo "Deleting $trainfile file"
rm -f $datadir/$trainfile
echo "Creating $trainfile file"
cp $datadir/$originalfile $datadir/$trainfile
echo "Created file $trainfile"
# Delete temp file, if it exists
rm -f temp.txt
# Number of lines in given file
nooflines=`sed -n '$=' $datadir/$trainfile`
echo "No of lines in $datadir/$originalfile are: $nooflines"
echo -n "Your sample size (recommended 10% of orig file)? " ; read samplesize
# Default is 100
if [ -z $samplesize ] ; then
echo "Default value of sample size = 100"
samplesize=100
fi
# Bash loop to generate random numbers and test file
echo "Will generate $samplesize random numbers"
echo "Original file size is $nooflines lines"
echo "Wait...it takes time...."
for (( i = 1 ; i <= $samplesize ; ++i ));
do
arr[i]=$(( ( RANDOM % $nooflines ) + 2 ));
lineno="${arr[i]}"
# Append lines to sample file
sed -n "${lineno}p" $datadir/$trainfile >> $datadir/$samplefile
# Delete the same line from $trainfile
sed "${lineno}d" $datadir/$trainfile > temp.txt
mv temp.txt $datadir/$trainfile
# Print number of lines appended in multiples of 50
a=$(( ( $i % 50 ) + 2 ));
if [ $a == "2" ] ; then
echo "Lines appended: $i";
fi
done
trlines=`sed -n '$=' $datadir/$trainfile`
samplines=`sed -n '$=' $datadir/$samplefile`
# Delete temp file
rm -f temp.txt
echo "---------------------------------"
echo "Lines in sample file $samplefile: $samplines"
echo "Lines in training file $trainfile : $trlines" ;
echo "Data folder: $datadir"
echo "---------------------------------"
##############END######################
We convert files ‘training.csv’ and ‘testing.csv’ to VW format as before. We then run the following simple vw commands first to create model then to make predictions.
# Create model vw -d training.vw --cache_file t_cache --passes 25 --oaa 10 -f t_model # Make predictions vw -t -i t_model tovalidate.vw -p p.txt
We now have the predicted digits (in file ‘p.txt’) and actual digits (first column of ‘tovalidate.vw’). We compare the two and prepare i) Confusion matrix and ii) Classification report. Unfortunately, unlike for two class predictions, in multiclass cases not many tools are available to prepare accuracy reports. We will use sklearn.metrics (python) module. The code is very simple. Just read the data and print reports.
from sklearn.metrics import confusion_matrix
from sklearn.metrics import classification_report
import pandas as pd
# Read tovalidate.vw and p.txt files
obs=pd.read_table ("/home/ashokharnal/Documents/kaggle/tovalidate.vw",sep=" ",header=None)
pred=pd.read_csv ("/home/ashokharnal/Documents/kaggle/p.txt",sep=" ",header=None)
# Print confusion matrix
confusion_matrix(obs.iloc[:,0],pred)
array([[486, 2, 2, 1, 0, 1, 1, 4, 0, 0],
[ 5, 375, 12, 6, 2, 3, 13, 9, 3, 1],
[ 2, 8, 383, 0, 17, 2, 1, 7, 4, 0],
[ 1, 1, 1, 405, 1, 3, 0, 3, 13, 1],
[ 1, 3, 23, 8, 297, 8, 2, 13, 6, 8],
[ 1, 3, 0, 5, 9, 366, 0, 2, 0, 3],
[ 4, 4, 5, 11, 4, 0, 419, 1, 17, 3],
[ 3, 6, 21, 1, 9, 1, 5, 343, 7, 2],
[ 0, 1, 14, 12, 0, 0, 11, 5, 389, 0],
[ 0, 0, 1, 0, 3, 1, 0, 1, 1, 357]])
target_names = ['class a', 'class b', 'class c', 'class d','class e','class f','class g','class h','class j','class k']
# Print classification report
print(classification_report(obs.iloc[:,0], pred, target_names=target_names))
precision recall f1-score support
class a 0.97 0.98 0.97 497
class b 0.93 0.87 0.90 429
class c 0.83 0.90 0.86 424
class d 0.90 0.94 0.92 429
class e 0.87 0.80 0.84 369
class f 0.95 0.94 0.95 389
class g 0.93 0.90 0.91 468
class h 0.88 0.86 0.87 398
class j 0.88 0.90 0.89 432
class k 0.95 0.98 0.97 364
avg / total 0.91 0.91 0.91 4199
# The following additional code displays confusion matrix visually (see image below)
%pylab
import matplotlib.pyplot as plt
# Show confusion matrix in a separate window
plt.matshow(cam)
plt.title('Confusion matrix')
plt.colorbar()
plt.ylabel('True label')
plt.xlabel('Predicted label')
plt.show()
Confusion Matrix
This finishes our experiment on Handwritten Digit Recognition.
Tags: handwritten digit recognizer, kaggle competition, Vowpal Wabbit
Linux Uncle 
Leave a comment