Model Zoo

Here is the list of models currently implemented in MMF:

ModelKeyDatasetsNotes
BANbantextvqa, vizwiz, vqa2BAN support is preliminary and hasn't been properly fine-tuned yet.
BUTDbutdcocopaper
CNN LSTMcnn_lstmclevr
FUSIONSconcat_bert, late_fusion, concat_bowhateful_memes
LoRRAlorravqa2, textvqa, vizwizpaper
LXMERTlxmertcoco, gqa, visual_genome, vqa2paper
M4Cm4cocrvqa, stvqa, textvqapaper
M4C Captionerm4c_captionercoco, textcapspaper
MMBTmmbthateful_memes, coco, mmimdb, okvqa, vqa2paper
MMF Transformermmf_transformerhateful_memes, okvqa, vqa2
Movie MCANmovie_mcanvqa2paper
Pythiapythiatextvqa, vizwiz, vqa2, visual_genomepaper
Unimodalunimodalhateful_memes
VilBERTvilberthateful_memes, coco, conceptual_captions, mmimdb, nlvr2, visual_entailment, vizwiz, vqa2paper
ViLTviltcoco, vqa2paper
Visual BERTvisual_bertgqa, hateful_memes, localized_narratives, coco, conceptual_captions, sbu, vqa2, mmimdb, nlvr2, visual_entailment, vizwizpaper

We are adding many more new models which will be available soon.

Last updated on by ryan-qiyu-jiang