CocoCaptions in PyTorch (1)

Buy Me a Coffee

*Memos:

  • My post explains CocoCaptions() using train2017 with captions_train2017.json, instances_train2017.json and person_keypoints_train2017.json, val2017 with captions_val2017.json, instances_val2017.json and person_keypoints_val2017.json and test2017 with image_info_test2017.json and image_info_test-dev2017.json.
  • My post explains CocoCaptions() using train2017 with stuff_train2017.json, val2017 with stuff_val2017.json, stuff_train2017_pixelmaps with stuff_train2017.json, stuff_val2017_pixelmaps with stuff_val2017.json, panoptic_train2017 with panoptic_train2017.json, panoptic_val2017 with panoptic_val2017.json and unlabeled2017 with image_info_unlabeled2017.json.
  • My post explains CocoDetection() using train2014 with captions_train2014.json, instances_train2014.json and person_keypoints_train2014.json, val2014 with captions_val2014.json, instances_val2014.json and person_keypoints_val2014.json and test2017 with image_info_test2014.json, image_info_test2015.json and image_info_test-dev2015.json.
  • My post explains CocoDetection() using train2017 with captions_train2017.json, instances_train2017.json and person_keypoints_train2017.json, val2017 with captions_val2017.json, instances_val2017.json and person_keypoints_val2017.json and test2017 with image_info_test2017.json and image_info_test-dev2017.json.
  • My post explains CocoDetection() using train2017 with stuff_train2017.json, val2017 with stuff_val2017.json, stuff_train2017_pixelmaps with stuff_train2017.json, stuff_val2017_pixelmaps with stuff_val2017.json, panoptic_train2017 with panoptic_train2017.json, panoptic_val2017 with panoptic_val2017.json and unlabeled2017 with image_info_unlabeled2017.json.
  • My post explains MS COCO.

CocoCaptions() can use MS COCO dataset as shown below. *This is for train2014 with captions_train2014.json, instances_train2014.json and person_keypoints_train2014.json, val2014 with captions_val2014.json, instances_val2014.json and person_keypoints_val2014.json and test2017 with image_info_test2014.json, image_info_test2015.json and image_info_test-dev2015.json:

*Memos:

  • The 1st argument is root(Required-Type:str or pathlib.Path): *Memos:
    • It’s the path to the images.
    • An absolute or relative path is possible.
  • The 2nd argument is annFile(Required-Type:str or pathlib.Path): *Memos:
    • It’s the path to the annotations.
    • An absolute or relative path is possible.
  • The 3rd argument is transform(Optional-Default:None-Type:callable).
  • The 4th argument is target_transform(Optional-Default:None-Type:callable).
  • The 5th argument is transforms(Optional-Default:None-Type:callable).
  • It must need pycocotools on Windows, Linux and macOS: *Memos:
    • e.g. pip install pycocotools.
    • e.g. conda install conda-forge::pycocotools.
    • Don’t use the ways to install pycocotools from cocodataset/cocoapi and philferriere/cocoapi because they don’t work and even if they are possible, they take a long time to install pycocotools.
  • You need to manually download and extract the datasets(images and annotations) which you want to coco/ from here as shown below. *You can use other folder structure:
data
└-coco
|-imgs
| |-train2014
| | |-COCO_train2014_000000000009.jpg
| | |-COCO_train2014_000000000025.jpg
| | |-COCO_train2014_000000000030.jpg
| | ...
| |-val2014/
| |-test2014/
| |-test2015/
| |-train2017/
| |-val2017/
| |-test2017/
| └-unlabeled2017/
└-anns
|-trainval2014
| |-captions_train2014.json
| |-instances_train2014.json
| |-person_keypoints_train2014.json
| |-captions_val2014.json
| |-instances_val2014.json
| └-person_keypoints_val2014.json
|-test2014
| └-image_info_test2014.json
|-test2015
| |-image_info_test2015.json
| └-image_info_test-dev2015.json
|-trainval2017
| |-captions_train2017.json
| |-instances_train2017.json
| |-person_keypoints_train2017.json
| |-captions_val2017.json
| |-instances_val2017.json
| └-person_keypoints_val2017.json
|-test2017
| |-image_info_test2017.json
| └-image_info_test-dev2017.json
|-stuff_trainval2017
| |-stuff_train2017.json
| |-stuff_val2017.json
| |-stuff_train2017_pixelmaps/
| | |-000000000009.png
| | |-000000000025.png
| | |-000000000030.png
| | ...
| |-stuff_val2017_pixelmaps/
| └-deprecated-challenge2017
| |-train-ids.txt
| └-val-ids.txt
|-panoptic_trainval2017
| |-panoptic_train2017.json
| |-panoptic_val2017.json
| |-panoptic_train2017/
| | |-000000000389.png
| | |-000000000404.png
| | |-000000000438.png
| | ...
| └-panoptic_val2017/
└-unlabeled2017
└-image_info_unlabeled2017.json
data 
 └-coco
    |-imgs
    |  |-train2014
    |  |  |-COCO_train2014_000000000009.jpg
    |  |  |-COCO_train2014_000000000025.jpg
    |  |  |-COCO_train2014_000000000030.jpg
    |  |  ...
    |  |-val2014/
    |  |-test2014/
    |  |-test2015/
    |  |-train2017/
    |  |-val2017/
    |  |-test2017/
    |  └-unlabeled2017/
    └-anns
       |-trainval2014
       |  |-captions_train2014.json
       |  |-instances_train2014.json
       |  |-person_keypoints_train2014.json
       |  |-captions_val2014.json
       |  |-instances_val2014.json
       |  └-person_keypoints_val2014.json
       |-test2014
       |  └-image_info_test2014.json
       |-test2015
       |  |-image_info_test2015.json
       |  └-image_info_test-dev2015.json
       |-trainval2017
       |  |-captions_train2017.json
       |  |-instances_train2017.json
       |  |-person_keypoints_train2017.json
       |  |-captions_val2017.json
       |  |-instances_val2017.json
       |  └-person_keypoints_val2017.json
       |-test2017
       |  |-image_info_test2017.json
       |  └-image_info_test-dev2017.json
       |-stuff_trainval2017
       |  |-stuff_train2017.json
       |  |-stuff_val2017.json
       |  |-stuff_train2017_pixelmaps/
       |  |  |-000000000009.png
       |  |  |-000000000025.png
       |  |  |-000000000030.png
       |  |  ...
       |  |-stuff_val2017_pixelmaps/
       |  └-deprecated-challenge2017
       |     |-train-ids.txt
       |     └-val-ids.txt
       |-panoptic_trainval2017
       |  |-panoptic_train2017.json
       |  |-panoptic_val2017.json
       |  |-panoptic_train2017/
       |  |  |-000000000389.png
       |  |  |-000000000404.png
       |  |  |-000000000438.png
       |  |  ...
       |  └-panoptic_val2017/
       └-unlabeled2017
          └-image_info_unlabeled2017.json
data └-coco |-imgs | |-train2014 | | |-COCO_train2014_000000000009.jpg | | |-COCO_train2014_000000000025.jpg | | |-COCO_train2014_000000000030.jpg | | ... | |-val2014/ | |-test2014/ | |-test2015/ | |-train2017/ | |-val2017/ | |-test2017/ | └-unlabeled2017/ └-anns |-trainval2014 | |-captions_train2014.json | |-instances_train2014.json | |-person_keypoints_train2014.json | |-captions_val2014.json | |-instances_val2014.json | └-person_keypoints_val2014.json |-test2014 | └-image_info_test2014.json |-test2015 | |-image_info_test2015.json | └-image_info_test-dev2015.json |-trainval2017 | |-captions_train2017.json | |-instances_train2017.json | |-person_keypoints_train2017.json | |-captions_val2017.json | |-instances_val2017.json | └-person_keypoints_val2017.json |-test2017 | |-image_info_test2017.json | └-image_info_test-dev2017.json |-stuff_trainval2017 | |-stuff_train2017.json | |-stuff_val2017.json | |-stuff_train2017_pixelmaps/ | | |-000000000009.png | | |-000000000025.png | | |-000000000030.png | | ... | |-stuff_val2017_pixelmaps/ | └-deprecated-challenge2017 | |-train-ids.txt | └-val-ids.txt |-panoptic_trainval2017 | |-panoptic_train2017.json | |-panoptic_val2017.json | |-panoptic_train2017/ | | |-000000000389.png | | |-000000000404.png | | |-000000000438.png | | ... | └-panoptic_val2017/ └-unlabeled2017 └-image_info_unlabeled2017.json

Enter fullscreen mode Exit fullscreen mode

<span>from</span> <span>torchvision.datasets</span> <span>import</span> <span>CocoCaptions</span>
<span>cap_train2014_data</span> <span>=</span> <span>CocoCaptions</span><span>(</span>
<span>root</span><span>=</span><span>"</span><span>data/coco/imgs/train2014</span><span>"</span><span>,</span>
<span>annFile</span><span>=</span><span>"</span><span>data/coco/anns/trainval2014/captions_train2014.json</span><span>"</span>
<span>)</span>
<span>cap_train2014_data</span> <span>=</span> <span>CocoCaptions</span><span>(</span>
<span>root</span><span>=</span><span>"</span><span>data/coco/imgs/train2014</span><span>"</span><span>,</span>
<span>annFile</span><span>=</span><span>"</span><span>data/coco/anns/trainval2014/captions_train2014.json</span><span>"</span><span>,</span>
<span>transform</span><span>=</span><span>None</span><span>,</span>
<span>target_transform</span><span>=</span><span>None</span><span>,</span>
<span>transforms</span><span>=</span><span>None</span>
<span>)</span>
<span>ins_train2014_data</span> <span>=</span> <span>CocoCaptions</span><span>(</span>
<span>root</span><span>=</span><span>"</span><span>data/coco/imgs/train2014</span><span>"</span><span>,</span>
<span>annFile</span><span>=</span><span>"</span><span>data/coco/anns/trainval2014/instances_train2014.json</span><span>"</span>
<span>)</span>
<span>pk_train2014_data</span> <span>=</span> <span>CocoCaptions</span><span>(</span>
<span>root</span><span>=</span><span>"</span><span>data/coco/imgs/train2014</span><span>"</span><span>,</span>
<span>annFile</span><span>=</span><span>"</span><span>data/coco/anns/trainval2014/person_keypoints_train2014.json</span><span>"</span>
<span>)</span>
<span>len</span><span>(</span><span>cap_train2014_data</span><span>),</span> <span>len</span><span>(</span><span>ins_train2014_data</span><span>),</span> <span>len</span><span>(</span><span>pk_train2014_data</span><span>)</span>
<span># (82783, 82783, 82783) </span>
<span>cap_val2014_data</span> <span>=</span> <span>CocoCaptions</span><span>(</span>
<span>root</span><span>=</span><span>"</span><span>data/coco/imgs/val2014</span><span>"</span><span>,</span>
<span>annFile</span><span>=</span><span>"</span><span>data/coco/anns/trainval2014/captions_val2014.json</span><span>"</span>
<span>)</span>
<span>ins_val2014_data</span> <span>=</span> <span>CocoCaptions</span><span>(</span>
<span>root</span><span>=</span><span>"</span><span>data/coco/imgs/val2014</span><span>"</span><span>,</span>
<span>annFile</span><span>=</span><span>"</span><span>data/coco/anns/trainval2014/instances_val2014.json</span><span>"</span>
<span>)</span>
<span>pk_val2014_data</span> <span>=</span> <span>CocoCaptions</span><span>(</span>
<span>root</span><span>=</span><span>"</span><span>data/coco/imgs/val2014</span><span>"</span><span>,</span>
<span>annFile</span><span>=</span><span>"</span><span>data/coco/anns/trainval2014/person_keypoints_val2014.json</span><span>"</span>
<span>)</span>
<span>len</span><span>(</span><span>cap_val2014_data</span><span>),</span> <span>len</span><span>(</span><span>ins_val2014_data</span><span>),</span> <span>len</span><span>(</span><span>pk_val2014_data</span><span>)</span>
<span># (40504, 40504, 40504) </span>
<span>test2014_data</span> <span>=</span> <span>CocoCaptions</span><span>(</span>
<span>root</span><span>=</span><span>"</span><span>data/coco/imgs/test2014</span><span>"</span><span>,</span>
<span>annFile</span><span>=</span><span>"</span><span>data/coco/anns/test2014/image_info_test2014.json</span><span>"</span>
<span>)</span>
<span>test2015_data</span> <span>=</span> <span>CocoCaptions</span><span>(</span>
<span>root</span><span>=</span><span>"</span><span>data/coco/imgs/test2015</span><span>"</span><span>,</span>
<span>annFile</span><span>=</span><span>"</span><span>data/coco/anns/test2015/image_info_test2015.json</span><span>"</span>
<span>)</span>
<span>testdev2015_data</span> <span>=</span> <span>CocoCaptions</span><span>(</span>
<span>root</span><span>=</span><span>"</span><span>data/coco/imgs/test2015</span><span>"</span><span>,</span>
<span>annFile</span><span>=</span><span>"</span><span>data/coco/anns/test2015/image_info_test-dev2015.json</span><span>"</span>
<span>)</span>
<span>len</span><span>(</span><span>test2014_data</span><span>),</span> <span>len</span><span>(</span><span>test2015_data</span><span>),</span> <span>len</span><span>(</span><span>testdev2015_data</span><span>)</span>
<span># (40775, 81434, 20288) </span>
<span>cap_train2014_data</span>
<span># Dataset CocoCaptions # Number of datapoints: 82783 # Root location: data/coco/imgs/train2014 </span>
<span>cap_train2014_data</span><span>.</span><span>root</span>
<span># 'data/coco/imgs/train2014' </span>
<span>print</span><span>(</span><span>cap_train2014_data</span><span>.</span><span>transform</span><span>)</span>
<span># None </span>
<span>print</span><span>(</span><span>cap_train2014_data</span><span>.</span><span>target_transform</span><span>)</span>
<span># None </span>
<span>print</span><span>(</span><span>cap_train2014_data</span><span>.</span><span>transforms</span><span>)</span>
<span># None </span>
<span>cap_train2014_data</span><span>.</span><span>coco</span>
<span># <pycocotools.coco.COCO at 0x759028ee1d00> </span>
<span>cap_train2014_data</span><span>[</span><span>26</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=427x640>, # ['three zeebras standing in a grassy field walking', # 'Three zebras are standing in an open field.', # 'Three zebra are walking through the grass of a field.', # 'Three zebras standing on a grassy dirt field.', # 'Three zebras grazing in green grass field area.']) </span>
<span>cap_train2014_data</span><span>[</span><span>179</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=480x640>, # ['a young guy walking in a forrest holding an object in his hand', # 'A partially black and white photo of a man throwing ... the woods.', # 'A disc golfer releases a throw from a dirt tee ... wooded course.', # 'The person is in the clearing of a wooded area. ', # 'a person throwing a frisbee at many trees ']) </span>
<span>cap_train2014_data</span><span>[</span><span>194</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=428x640>, # ['A person on a court with a tennis racket.', # 'A man that is holding a racquet standing in the grass.', # 'A tennis player hits the ball during a match.', # 'The tennis player is poised to serve a ball.', # 'Man in white playing tennis on a court.']) </span>
<span>ins_train2014_data</span><span>[</span><span>26</span><span>]</span> <span># Error </span>
<span>ins_train2014_data</span><span>[</span><span>179</span><span>]</span> <span># Error </span>
<span>ins_train2014_data</span><span>[</span><span>194</span><span>]</span> <span># Error </span>
<span>pk_train2014_data</span><span>[</span><span>26</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=427x640>, []) </span>
<span>pk_train2014_data</span><span>[</span><span>179</span><span>]</span> <span># Error </span>
<span>pk_train2014_data</span><span>[</span><span>194</span><span>]</span> <span># Error </span>
<span>cap_val2014_data</span><span>[</span><span>26</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x360>, # ['a close up of a child next to a cake with balloons', # 'A baby sitting in front of a cake wearing a tie.', # 'The young boy is dressed in a tie that matches his cake. ', # 'A child eating a birthday cake near some balloons.', # 'A baby eating a cake with a tie around ... the background.']) </span>
<span>cap_val2014_data</span><span>[</span><span>179</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=500x302>, # ['Many small children are posing together in the ... white photo. ', # 'A vintage school picture of grade school aged children.', # 'A black and white photo of a group of kids.', # 'A group of children standing next to each other.', # 'A group of children standing and sitting beside each other. ']) </span>
<span>cap_val2014_data</span><span>[</span><span>194</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x427>, # ['A man hitting a tennis ball with a racquet.', # 'champion tennis player swats at the ball hoping to win', # 'A man is hitting his tennis ball with a recket on the court.', # 'a tennis player on a court with a racket', # 'A professional tennis player hits a ball as fans watch.']) </span>
<span>ins_val2014_data</span><span>[</span><span>26</span><span>]</span> <span># Error </span>
<span>ins_val2014_data</span><span>[</span><span>179</span><span>]</span> <span># Error </span>
<span>ins_val2014_data</span><span>[</span><span>194</span><span>]</span> <span># Error </span>
<span>pk_val2014_data</span><span>[</span><span>26</span><span>]</span> <span># Error </span>
<span>pk_val2014_data</span><span>[</span><span>179</span><span>]</span> <span># Error </span>
<span>pk_val2014_data</span><span>[</span><span>194</span><span>]</span> <span># Error </span>
<span>test2014_data</span><span>[</span><span>26</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x640>, []) </span>
<span>test2014_data</span><span>[</span><span>179</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x480>, []) </span>
<span>test2014_data</span><span>[</span><span>194</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x360>, []) </span>
<span>test2015_data</span><span>[</span><span>26</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x480>, []) </span>
<span>test2015_data</span><span>[</span><span>179</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x426>, []) </span>
<span>test2015_data</span><span>[</span><span>194</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x480>, []) </span>
<span>testdev2015_data</span><span>[</span><span>26</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x360>, []) </span>
<span>testdev2015_data</span><span>[</span><span>179</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x480>, []) </span>
<span>testdev2015_data</span><span>[</span><span>194</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x480>, []) </span>
<span>import</span> <span>matplotlib.pyplot</span> <span>as</span> <span>plt</span>
<span>def</span> <span>show_images</span><span>(</span><span>data</span><span>,</span> <span>ims</span><span>,</span> <span>main_title</span><span>=</span><span>None</span><span>):</span>
<span>file</span> <span>=</span> <span>data</span><span>.</span><span>root</span><span>.</span><span>split</span><span>(</span><span>'</span><span>/</span><span>'</span><span>)[</span><span>-</span><span>1</span><span>]</span>
<span>fig</span><span>,</span> <span>axes</span> <span>=</span> <span>plt</span><span>.</span><span>subplots</span><span>(</span><span>nrows</span><span>=</span><span>1</span><span>,</span> <span>ncols</span><span>=</span><span>3</span><span>,</span> <span>figsize</span><span>=</span><span>(</span><span>14</span><span>,</span> <span>8</span><span>))</span>
<span>fig</span><span>.</span><span>suptitle</span><span>(</span><span>t</span><span>=</span><span>main_title</span><span>,</span> <span>y</span><span>=</span><span>0.9</span><span>,</span> <span>fontsize</span><span>=</span><span>14</span><span>)</span>
<span>x_crd</span> <span>=</span> <span>0.02</span>
<span>for</span> <span>i</span><span>,</span> <span>axis</span> <span>in</span> <span>zip</span><span>(</span><span>ims</span><span>,</span> <span>axes</span><span>.</span><span>ravel</span><span>()):</span>
<span>if</span> <span>data</span><span>[</span><span>i</span><span>][</span><span>1</span><span>]:</span>
<span>im</span><span>,</span> <span>anns</span> <span>=</span> <span>data</span><span>[</span><span>i</span><span>]</span>
<span>axis</span><span>.</span><span>imshow</span><span>(</span><span>X</span><span>=</span><span>im</span><span>)</span>
<span>y_crd</span> <span>=</span> <span>0.0</span>
<span>for</span> <span>j</span><span>,</span> <span>ann</span> <span>in</span> <span>enumerate</span><span>(</span><span>iterable</span><span>=</span><span>anns</span><span>):</span>
<span>text_list</span> <span>=</span> <span>ann</span><span>.</span><span>split</span><span>()</span>
<span>if</span> <span>len</span><span>(</span><span>text_list</span><span>)</span> <span>></span> <span>9</span><span>:</span>
<span>text</span> <span>=</span> <span>"</span><span> </span><span>"</span><span>.</span><span>join</span><span>(</span><span>text_list</span><span>[</span><span>0</span><span>:</span><span>10</span><span>])</span> <span>+</span> <span>"</span><span> ...</span><span>"</span>
<span>else</span><span>:</span>
<span>text</span> <span>=</span> <span>"</span><span> </span><span>"</span><span>.</span><span>join</span><span>(</span><span>text_list</span><span>)</span>
<span>plt</span><span>.</span><span>figtext</span><span>(</span><span>x</span><span>=</span><span>x_crd</span><span>,</span> <span>y</span><span>=</span><span>y_crd</span><span>,</span> <span>fontsize</span><span>=</span><span>10</span><span>,</span>
<span>s</span><span>=</span><span>f</span><span>'</span><span>{</span><span>j</span><span>}</span><span>:</span><span>\n</span><span>{</span><span>text</span><span>}</span><span>'</span><span>)</span>
<span>y_crd</span> <span>-=</span> <span>0.06</span>
<span>x_crd</span> <span>+=</span> <span>0.325</span>
<span>if</span> <span>i</span> <span>==</span> <span>2</span> <span>and</span> <span>file</span> <span>==</span> <span>"</span><span>val2017</span><span>"</span><span>:</span>
<span>x_crd</span> <span>+=</span> <span>0.06</span>
<span>elif</span> <span>not</span> <span>data</span><span>[</span><span>i</span><span>][</span><span>1</span><span>]:</span>
<span>im</span><span>,</span> <span>_</span> <span>=</span> <span>data</span><span>[</span><span>i</span><span>]</span>
<span>axis</span><span>.</span><span>imshow</span><span>(</span><span>X</span><span>=</span><span>im</span><span>)</span>
<span>fig</span><span>.</span><span>tight_layout</span><span>()</span>
<span>plt</span><span>.</span><span>show</span><span>()</span>
<span>ims</span> <span>=</span> <span>(</span><span>26</span><span>,</span> <span>179</span><span>,</span> <span>194</span><span>)</span>
<span>show_images</span><span>(</span><span>data</span><span>=</span><span>cap_train2014_data</span><span>,</span> <span>ims</span><span>=</span><span>ims</span><span>,</span>
<span>main_title</span><span>=</span><span>"</span><span>cap_train2014_data</span><span>"</span><span>)</span>
<span>show_images</span><span>(</span><span>data</span><span>=</span><span>cap_val2014_data</span><span>,</span> <span>ims</span><span>=</span><span>ims</span><span>,</span>
<span>main_title</span><span>=</span><span>"</span><span>cap_val2014_data</span><span>"</span><span>)</span>
<span>show_images</span><span>(</span><span>data</span><span>=</span><span>test2014_data</span><span>,</span> <span>ims</span><span>=</span><span>ims</span><span>,</span>
<span>main_title</span><span>=</span><span>"</span><span>test2014_data</span><span>"</span><span>)</span>
<span>show_images</span><span>(</span><span>data</span><span>=</span><span>test2015_data</span><span>,</span> <span>ims</span><span>=</span><span>ims</span><span>,</span>
<span>main_title</span><span>=</span><span>"</span><span>test2015_data</span><span>"</span><span>)</span>
<span>show_images</span><span>(</span><span>data</span><span>=</span><span>testdev2015_data</span><span>,</span> <span>ims</span><span>=</span><span>ims</span><span>,</span>
<span>main_title</span><span>=</span><span>"</span><span>testdev2015_data</span><span>"</span><span>)</span>
<span>from</span> <span>torchvision.datasets</span> <span>import</span> <span>CocoCaptions</span>

<span>cap_train2014_data</span> <span>=</span> <span>CocoCaptions</span><span>(</span>
    <span>root</span><span>=</span><span>"</span><span>data/coco/imgs/train2014</span><span>"</span><span>,</span>
    <span>annFile</span><span>=</span><span>"</span><span>data/coco/anns/trainval2014/captions_train2014.json</span><span>"</span>
<span>)</span>

<span>cap_train2014_data</span> <span>=</span> <span>CocoCaptions</span><span>(</span>
    <span>root</span><span>=</span><span>"</span><span>data/coco/imgs/train2014</span><span>"</span><span>,</span>
    <span>annFile</span><span>=</span><span>"</span><span>data/coco/anns/trainval2014/captions_train2014.json</span><span>"</span><span>,</span>
    <span>transform</span><span>=</span><span>None</span><span>,</span>
    <span>target_transform</span><span>=</span><span>None</span><span>,</span>
    <span>transforms</span><span>=</span><span>None</span>
<span>)</span>

<span>ins_train2014_data</span> <span>=</span> <span>CocoCaptions</span><span>(</span>
    <span>root</span><span>=</span><span>"</span><span>data/coco/imgs/train2014</span><span>"</span><span>,</span>
    <span>annFile</span><span>=</span><span>"</span><span>data/coco/anns/trainval2014/instances_train2014.json</span><span>"</span>
<span>)</span>

<span>pk_train2014_data</span> <span>=</span> <span>CocoCaptions</span><span>(</span>
    <span>root</span><span>=</span><span>"</span><span>data/coco/imgs/train2014</span><span>"</span><span>,</span>
    <span>annFile</span><span>=</span><span>"</span><span>data/coco/anns/trainval2014/person_keypoints_train2014.json</span><span>"</span>
<span>)</span>

<span>len</span><span>(</span><span>cap_train2014_data</span><span>),</span> <span>len</span><span>(</span><span>ins_train2014_data</span><span>),</span> <span>len</span><span>(</span><span>pk_train2014_data</span><span>)</span>
<span># (82783, 82783, 82783) </span>
<span>cap_val2014_data</span> <span>=</span> <span>CocoCaptions</span><span>(</span>
    <span>root</span><span>=</span><span>"</span><span>data/coco/imgs/val2014</span><span>"</span><span>,</span>
    <span>annFile</span><span>=</span><span>"</span><span>data/coco/anns/trainval2014/captions_val2014.json</span><span>"</span>
<span>)</span>

<span>ins_val2014_data</span> <span>=</span> <span>CocoCaptions</span><span>(</span>
    <span>root</span><span>=</span><span>"</span><span>data/coco/imgs/val2014</span><span>"</span><span>,</span>
    <span>annFile</span><span>=</span><span>"</span><span>data/coco/anns/trainval2014/instances_val2014.json</span><span>"</span>
<span>)</span>

<span>pk_val2014_data</span> <span>=</span> <span>CocoCaptions</span><span>(</span>
    <span>root</span><span>=</span><span>"</span><span>data/coco/imgs/val2014</span><span>"</span><span>,</span>
    <span>annFile</span><span>=</span><span>"</span><span>data/coco/anns/trainval2014/person_keypoints_val2014.json</span><span>"</span>
<span>)</span>

<span>len</span><span>(</span><span>cap_val2014_data</span><span>),</span> <span>len</span><span>(</span><span>ins_val2014_data</span><span>),</span> <span>len</span><span>(</span><span>pk_val2014_data</span><span>)</span>
<span># (40504, 40504, 40504) </span>
<span>test2014_data</span> <span>=</span> <span>CocoCaptions</span><span>(</span>
    <span>root</span><span>=</span><span>"</span><span>data/coco/imgs/test2014</span><span>"</span><span>,</span>
    <span>annFile</span><span>=</span><span>"</span><span>data/coco/anns/test2014/image_info_test2014.json</span><span>"</span>
<span>)</span>

<span>test2015_data</span> <span>=</span> <span>CocoCaptions</span><span>(</span>
    <span>root</span><span>=</span><span>"</span><span>data/coco/imgs/test2015</span><span>"</span><span>,</span>
    <span>annFile</span><span>=</span><span>"</span><span>data/coco/anns/test2015/image_info_test2015.json</span><span>"</span>
<span>)</span>

<span>testdev2015_data</span> <span>=</span> <span>CocoCaptions</span><span>(</span>
    <span>root</span><span>=</span><span>"</span><span>data/coco/imgs/test2015</span><span>"</span><span>,</span>
    <span>annFile</span><span>=</span><span>"</span><span>data/coco/anns/test2015/image_info_test-dev2015.json</span><span>"</span>
<span>)</span>

<span>len</span><span>(</span><span>test2014_data</span><span>),</span> <span>len</span><span>(</span><span>test2015_data</span><span>),</span> <span>len</span><span>(</span><span>testdev2015_data</span><span>)</span>
<span># (40775, 81434, 20288) </span>
<span>cap_train2014_data</span>
<span># Dataset CocoCaptions # Number of datapoints: 82783 # Root location: data/coco/imgs/train2014 </span>
<span>cap_train2014_data</span><span>.</span><span>root</span>
<span># 'data/coco/imgs/train2014' </span>
<span>print</span><span>(</span><span>cap_train2014_data</span><span>.</span><span>transform</span><span>)</span>
<span># None </span>
<span>print</span><span>(</span><span>cap_train2014_data</span><span>.</span><span>target_transform</span><span>)</span>
<span># None </span>
<span>print</span><span>(</span><span>cap_train2014_data</span><span>.</span><span>transforms</span><span>)</span>
<span># None </span>
<span>cap_train2014_data</span><span>.</span><span>coco</span>
<span># <pycocotools.coco.COCO at 0x759028ee1d00> </span>
<span>cap_train2014_data</span><span>[</span><span>26</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=427x640>, # ['three zeebras standing in a grassy field walking', # 'Three zebras are standing in an open field.', # 'Three zebra are walking through the grass of a field.', # 'Three zebras standing on a grassy dirt field.', # 'Three zebras grazing in green grass field area.']) </span>
<span>cap_train2014_data</span><span>[</span><span>179</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=480x640>, # ['a young guy walking in a forrest holding an object in his hand', # 'A partially black and white photo of a man throwing ... the woods.', # 'A disc golfer releases a throw from a dirt tee ... wooded course.', # 'The person is in the clearing of a wooded area. ', # 'a person throwing a frisbee at many trees ']) </span>
<span>cap_train2014_data</span><span>[</span><span>194</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=428x640>, # ['A person on a court with a tennis racket.', # 'A man that is holding a racquet standing in the grass.', # 'A tennis player hits the ball during a match.', # 'The tennis player is poised to serve a ball.', # 'Man in white playing tennis on a court.']) </span>
<span>ins_train2014_data</span><span>[</span><span>26</span><span>]</span> <span># Error </span>
<span>ins_train2014_data</span><span>[</span><span>179</span><span>]</span> <span># Error </span>
<span>ins_train2014_data</span><span>[</span><span>194</span><span>]</span> <span># Error </span>
<span>pk_train2014_data</span><span>[</span><span>26</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=427x640>, []) </span>
<span>pk_train2014_data</span><span>[</span><span>179</span><span>]</span> <span># Error </span>
<span>pk_train2014_data</span><span>[</span><span>194</span><span>]</span> <span># Error </span>
<span>cap_val2014_data</span><span>[</span><span>26</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x360>, # ['a close up of a child next to a cake with balloons', # 'A baby sitting in front of a cake wearing a tie.', # 'The young boy is dressed in a tie that matches his cake. ', # 'A child eating a birthday cake near some balloons.', # 'A baby eating a cake with a tie around ... the background.']) </span>
<span>cap_val2014_data</span><span>[</span><span>179</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=500x302>, # ['Many small children are posing together in the ... white photo. ', # 'A vintage school picture of grade school aged children.', # 'A black and white photo of a group of kids.', # 'A group of children standing next to each other.', # 'A group of children standing and sitting beside each other. ']) </span>
<span>cap_val2014_data</span><span>[</span><span>194</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x427>, # ['A man hitting a tennis ball with a racquet.', # 'champion tennis player swats at the ball hoping to win', # 'A man is hitting his tennis ball with a recket on the court.', # 'a tennis player on a court with a racket', # 'A professional tennis player hits a ball as fans watch.']) </span>
<span>ins_val2014_data</span><span>[</span><span>26</span><span>]</span> <span># Error </span>
<span>ins_val2014_data</span><span>[</span><span>179</span><span>]</span> <span># Error </span>
<span>ins_val2014_data</span><span>[</span><span>194</span><span>]</span> <span># Error </span>
<span>pk_val2014_data</span><span>[</span><span>26</span><span>]</span> <span># Error </span>
<span>pk_val2014_data</span><span>[</span><span>179</span><span>]</span> <span># Error </span>
<span>pk_val2014_data</span><span>[</span><span>194</span><span>]</span> <span># Error </span>
<span>test2014_data</span><span>[</span><span>26</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x640>, []) </span>
<span>test2014_data</span><span>[</span><span>179</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x480>, []) </span>
<span>test2014_data</span><span>[</span><span>194</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x360>, []) </span>
<span>test2015_data</span><span>[</span><span>26</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x480>, []) </span>
<span>test2015_data</span><span>[</span><span>179</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x426>, []) </span>
<span>test2015_data</span><span>[</span><span>194</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x480>, []) </span>
<span>testdev2015_data</span><span>[</span><span>26</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x360>, []) </span>
<span>testdev2015_data</span><span>[</span><span>179</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x480>, []) </span>
<span>testdev2015_data</span><span>[</span><span>194</span><span>]</span>
<span># (<PIL.Image.Image image mode=RGB size=640x480>, []) </span>
<span>import</span> <span>matplotlib.pyplot</span> <span>as</span> <span>plt</span>

<span>def</span> <span>show_images</span><span>(</span><span>data</span><span>,</span> <span>ims</span><span>,</span> <span>main_title</span><span>=</span><span>None</span><span>):</span>
    <span>file</span> <span>=</span> <span>data</span><span>.</span><span>root</span><span>.</span><span>split</span><span>(</span><span>'</span><span>/</span><span>'</span><span>)[</span><span>-</span><span>1</span><span>]</span>
    <span>fig</span><span>,</span> <span>axes</span> <span>=</span> <span>plt</span><span>.</span><span>subplots</span><span>(</span><span>nrows</span><span>=</span><span>1</span><span>,</span> <span>ncols</span><span>=</span><span>3</span><span>,</span> <span>figsize</span><span>=</span><span>(</span><span>14</span><span>,</span> <span>8</span><span>))</span>
    <span>fig</span><span>.</span><span>suptitle</span><span>(</span><span>t</span><span>=</span><span>main_title</span><span>,</span> <span>y</span><span>=</span><span>0.9</span><span>,</span> <span>fontsize</span><span>=</span><span>14</span><span>)</span>
    <span>x_crd</span> <span>=</span> <span>0.02</span>
    <span>for</span> <span>i</span><span>,</span> <span>axis</span> <span>in</span> <span>zip</span><span>(</span><span>ims</span><span>,</span> <span>axes</span><span>.</span><span>ravel</span><span>()):</span>
        <span>if</span> <span>data</span><span>[</span><span>i</span><span>][</span><span>1</span><span>]:</span>
            <span>im</span><span>,</span> <span>anns</span> <span>=</span> <span>data</span><span>[</span><span>i</span><span>]</span>
            <span>axis</span><span>.</span><span>imshow</span><span>(</span><span>X</span><span>=</span><span>im</span><span>)</span>
            <span>y_crd</span> <span>=</span> <span>0.0</span>
            <span>for</span> <span>j</span><span>,</span> <span>ann</span> <span>in</span> <span>enumerate</span><span>(</span><span>iterable</span><span>=</span><span>anns</span><span>):</span>
                <span>text_list</span> <span>=</span> <span>ann</span><span>.</span><span>split</span><span>()</span>
                <span>if</span> <span>len</span><span>(</span><span>text_list</span><span>)</span> <span>></span> <span>9</span><span>:</span>
                    <span>text</span> <span>=</span> <span>"</span><span> </span><span>"</span><span>.</span><span>join</span><span>(</span><span>text_list</span><span>[</span><span>0</span><span>:</span><span>10</span><span>])</span> <span>+</span> <span>"</span><span> ...</span><span>"</span>
                <span>else</span><span>:</span>
                    <span>text</span> <span>=</span> <span>"</span><span> </span><span>"</span><span>.</span><span>join</span><span>(</span><span>text_list</span><span>)</span>
                <span>plt</span><span>.</span><span>figtext</span><span>(</span><span>x</span><span>=</span><span>x_crd</span><span>,</span> <span>y</span><span>=</span><span>y_crd</span><span>,</span> <span>fontsize</span><span>=</span><span>10</span><span>,</span>
                            <span>s</span><span>=</span><span>f</span><span>'</span><span>{</span><span>j</span><span>}</span><span>:</span><span>\n</span><span>{</span><span>text</span><span>}</span><span>'</span><span>)</span>
                <span>y_crd</span> <span>-=</span> <span>0.06</span>
            <span>x_crd</span> <span>+=</span> <span>0.325</span>
            <span>if</span> <span>i</span> <span>==</span> <span>2</span> <span>and</span> <span>file</span> <span>==</span> <span>"</span><span>val2017</span><span>"</span><span>:</span>
                <span>x_crd</span> <span>+=</span> <span>0.06</span>
        <span>elif</span> <span>not</span> <span>data</span><span>[</span><span>i</span><span>][</span><span>1</span><span>]:</span>
            <span>im</span><span>,</span> <span>_</span> <span>=</span> <span>data</span><span>[</span><span>i</span><span>]</span>
            <span>axis</span><span>.</span><span>imshow</span><span>(</span><span>X</span><span>=</span><span>im</span><span>)</span>
    <span>fig</span><span>.</span><span>tight_layout</span><span>()</span>
    <span>plt</span><span>.</span><span>show</span><span>()</span>

<span>ims</span> <span>=</span> <span>(</span><span>26</span><span>,</span> <span>179</span><span>,</span> <span>194</span><span>)</span>

<span>show_images</span><span>(</span><span>data</span><span>=</span><span>cap_train2014_data</span><span>,</span> <span>ims</span><span>=</span><span>ims</span><span>,</span>
             <span>main_title</span><span>=</span><span>"</span><span>cap_train2014_data</span><span>"</span><span>)</span>
<span>show_images</span><span>(</span><span>data</span><span>=</span><span>cap_val2014_data</span><span>,</span> <span>ims</span><span>=</span><span>ims</span><span>,</span> 
             <span>main_title</span><span>=</span><span>"</span><span>cap_val2014_data</span><span>"</span><span>)</span>
<span>show_images</span><span>(</span><span>data</span><span>=</span><span>test2014_data</span><span>,</span> <span>ims</span><span>=</span><span>ims</span><span>,</span>
             <span>main_title</span><span>=</span><span>"</span><span>test2014_data</span><span>"</span><span>)</span>
<span>show_images</span><span>(</span><span>data</span><span>=</span><span>test2015_data</span><span>,</span> <span>ims</span><span>=</span><span>ims</span><span>,</span>
             <span>main_title</span><span>=</span><span>"</span><span>test2015_data</span><span>"</span><span>)</span>
<span>show_images</span><span>(</span><span>data</span><span>=</span><span>testdev2015_data</span><span>,</span> <span>ims</span><span>=</span><span>ims</span><span>,</span>
             <span>main_title</span><span>=</span><span>"</span><span>testdev2015_data</span><span>"</span><span>)</span>
from torchvision.datasets import CocoCaptions cap_train2014_data = CocoCaptions( root="data/coco/imgs/train2014", annFile="data/coco/anns/trainval2014/captions_train2014.json" ) cap_train2014_data = CocoCaptions( root="data/coco/imgs/train2014", annFile="data/coco/anns/trainval2014/captions_train2014.json", transform=None, target_transform=None, transforms=None ) ins_train2014_data = CocoCaptions( root="data/coco/imgs/train2014", annFile="data/coco/anns/trainval2014/instances_train2014.json" ) pk_train2014_data = CocoCaptions( root="data/coco/imgs/train2014", annFile="data/coco/anns/trainval2014/person_keypoints_train2014.json" ) len(cap_train2014_data), len(ins_train2014_data), len(pk_train2014_data) # (82783, 82783, 82783) cap_val2014_data = CocoCaptions( root="data/coco/imgs/val2014", annFile="data/coco/anns/trainval2014/captions_val2014.json" ) ins_val2014_data = CocoCaptions( root="data/coco/imgs/val2014", annFile="data/coco/anns/trainval2014/instances_val2014.json" ) pk_val2014_data = CocoCaptions( root="data/coco/imgs/val2014", annFile="data/coco/anns/trainval2014/person_keypoints_val2014.json" ) len(cap_val2014_data), len(ins_val2014_data), len(pk_val2014_data) # (40504, 40504, 40504) test2014_data = CocoCaptions( root="data/coco/imgs/test2014", annFile="data/coco/anns/test2014/image_info_test2014.json" ) test2015_data = CocoCaptions( root="data/coco/imgs/test2015", annFile="data/coco/anns/test2015/image_info_test2015.json" ) testdev2015_data = CocoCaptions( root="data/coco/imgs/test2015", annFile="data/coco/anns/test2015/image_info_test-dev2015.json" ) len(test2014_data), len(test2015_data), len(testdev2015_data) # (40775, 81434, 20288) cap_train2014_data # Dataset CocoCaptions # Number of datapoints: 82783 # Root location: data/coco/imgs/train2014 cap_train2014_data.root # 'data/coco/imgs/train2014' print(cap_train2014_data.transform) # None print(cap_train2014_data.target_transform) # None print(cap_train2014_data.transforms) # None cap_train2014_data.coco # <pycocotools.coco.COCO at 0x759028ee1d00> cap_train2014_data[26] # (<PIL.Image.Image image mode=RGB size=427x640>, # ['three zeebras standing in a grassy field walking', # 'Three zebras are standing in an open field.', # 'Three zebra are walking through the grass of a field.', # 'Three zebras standing on a grassy dirt field.', # 'Three zebras grazing in green grass field area.']) cap_train2014_data[179] # (<PIL.Image.Image image mode=RGB size=480x640>, # ['a young guy walking in a forrest holding an object in his hand', # 'A partially black and white photo of a man throwing ... the woods.', # 'A disc golfer releases a throw from a dirt tee ... wooded course.', # 'The person is in the clearing of a wooded area. ', # 'a person throwing a frisbee at many trees ']) cap_train2014_data[194] # (<PIL.Image.Image image mode=RGB size=428x640>, # ['A person on a court with a tennis racket.', # 'A man that is holding a racquet standing in the grass.', # 'A tennis player hits the ball during a match.', # 'The tennis player is poised to serve a ball.', # 'Man in white playing tennis on a court.']) ins_train2014_data[26] # Error ins_train2014_data[179] # Error ins_train2014_data[194] # Error pk_train2014_data[26] # (<PIL.Image.Image image mode=RGB size=427x640>, []) pk_train2014_data[179] # Error pk_train2014_data[194] # Error cap_val2014_data[26] # (<PIL.Image.Image image mode=RGB size=640x360>, # ['a close up of a child next to a cake with balloons', # 'A baby sitting in front of a cake wearing a tie.', # 'The young boy is dressed in a tie that matches his cake. ', # 'A child eating a birthday cake near some balloons.', # 'A baby eating a cake with a tie around ... the background.']) cap_val2014_data[179] # (<PIL.Image.Image image mode=RGB size=500x302>, # ['Many small children are posing together in the ... white photo. ', # 'A vintage school picture of grade school aged children.', # 'A black and white photo of a group of kids.', # 'A group of children standing next to each other.', # 'A group of children standing and sitting beside each other. ']) cap_val2014_data[194] # (<PIL.Image.Image image mode=RGB size=640x427>, # ['A man hitting a tennis ball with a racquet.', # 'champion tennis player swats at the ball hoping to win', # 'A man is hitting his tennis ball with a recket on the court.', # 'a tennis player on a court with a racket', # 'A professional tennis player hits a ball as fans watch.']) ins_val2014_data[26] # Error ins_val2014_data[179] # Error ins_val2014_data[194] # Error pk_val2014_data[26] # Error pk_val2014_data[179] # Error pk_val2014_data[194] # Error test2014_data[26] # (<PIL.Image.Image image mode=RGB size=640x640>, []) test2014_data[179] # (<PIL.Image.Image image mode=RGB size=640x480>, []) test2014_data[194] # (<PIL.Image.Image image mode=RGB size=640x360>, []) test2015_data[26] # (<PIL.Image.Image image mode=RGB size=640x480>, []) test2015_data[179] # (<PIL.Image.Image image mode=RGB size=640x426>, []) test2015_data[194] # (<PIL.Image.Image image mode=RGB size=640x480>, []) testdev2015_data[26] # (<PIL.Image.Image image mode=RGB size=640x360>, []) testdev2015_data[179] # (<PIL.Image.Image image mode=RGB size=640x480>, []) testdev2015_data[194] # (<PIL.Image.Image image mode=RGB size=640x480>, []) import matplotlib.pyplot as plt def show_images(data, ims, main_title=None): file = data.root.split('/')[-1] fig, axes = plt.subplots(nrows=1, ncols=3, figsize=(14, 8)) fig.suptitle(t=main_title, y=0.9, fontsize=14) x_crd = 0.02 for i, axis in zip(ims, axes.ravel()): if data[i][1]: im, anns = data[i] axis.imshow(X=im) y_crd = 0.0 for j, ann in enumerate(iterable=anns): text_list = ann.split() if len(text_list) > 9: text = " ".join(text_list[0:10]) + " ..." else: text = " ".join(text_list) plt.figtext(x=x_crd, y=y_crd, fontsize=10, s=f'{j}:\n{text}') y_crd -= 0.06 x_crd += 0.325 if i == 2 and file == "val2017": x_crd += 0.06 elif not data[i][1]: im, _ = data[i] axis.imshow(X=im) fig.tight_layout() plt.show() ims = (26, 179, 194) show_images(data=cap_train2014_data, ims=ims, main_title="cap_train2014_data") show_images(data=cap_val2014_data, ims=ims, main_title="cap_val2014_data") show_images(data=test2014_data, ims=ims, main_title="test2014_data") show_images(data=test2015_data, ims=ims, main_title="test2015_data") show_images(data=testdev2015_data, ims=ims, main_title="testdev2015_data")

Enter fullscreen mode Exit fullscreen mode

原文链接:CocoCaptions in PyTorch (1)

© 版权声明
THE END
喜欢就支持一下吧
点赞5 分享
Thank you push me off a cliff let me see the whole sky.
感谢你将我推下悬崖让我看清整片天空
评论 抢沙发

请登录后发表评论

    暂无评论内容