Logical Consistent QA's (L-ConVQA): json file
Each Visual Genome image-id (in case the image-id doesn't exactly match the official Visual Genome id, the image location is at
https://filebox.ece.vt.edu/~ray93/VGImages/images/VG_<id>.png) has a separate json file named qas_<id>.json. The json file has two keys- consistent and inconsistent.
'Consistent' contains list of sets of consistent QA pairs. 'Inconsistent' has a list of pairs of inconsistent QA's.
We shall release bounding boxes and scene graph edges corresponding to these QA's soon.
AMT Worker Cleaned Logical Consistent QA's Test set (L-ConVQA Test): json file
Contains a list of cleaned sets of consistent QA pairs with the key being the Visual Genome image id. This is the data used to report results on L-ConVQA in the paper.
Human-annotated common-sense based consistency data (CS-ConVQA), Consistent Sets: json file
Contains a dictionary with keys as the VQA image and values as a list of lists of consistent QA sets. This is the split
of the dataset used to report results on CS-ConVQA in the paper.
Human-annotated common-sense based consistency data (CS-ConVQA), Inconsistent Pairs: json file
Contains a dictionary with keys as the VQA image and values as a list of pairs of inconsistent QA's. The inconsistent pairs were generated by automatically switching answers in selected human-annotated consistent QA pairs.