Foundational Robustness of Foundation Models (NeurIPS 2022 tutorial)

This repository holds code and other relevant files for the NeurIPS 2022 tutorial: Foundational Robustness of Foundation Models by Pin-Yu Chen (IBM Research), Sijia Liu (Michigan State University), and Sayak Paul (Hugging Face).

For details on schedule and the tutorial outline, please refer to our tutorial website. You can also find the tutorial listing on IBM Research.

Update January 13, 2023: Our tutorial video is now public. Find it here.

Navigating the codebase

We provide code for analytical tools for two types of models: vision and code. Below provides a high-level overview of what code and vision_models directories contain:

vision_models
├── probing_transformer_models
│   ├── attention_distance
│   ├── attention_maps
│   ├── linear_projections
│   └── positional_embeddings
├── representation_effectiveness
│   ├── fourier_heatmap
│   ├── masking
│   ├── pgd_attacks
│   └── spectral_decomposition
└── robustness_eval

code
├── Attack.ipynb

Each directory provides a standalone README.md with instructions about executing the scripts / notebooks.

Slides

You can find the slides in the slides directory.

Acknowledgements

We thank Jinghan Jia (Michigan State University) for contributing the code for evaluating "code" models.