Trustworthiness in the Presence of Adversaries and Strategic Agents in ML

Logistics

• Date: Thursday, October 12th
  
• In-person Location: Northwestern University: Mudd Library 3rd floor, 2233 Tech Drive, Evanston
• Click here to watch the workshop

Titles and Abstracts

Speaker: Saba Ahmadi

Title: Fundamental Bounds on Online Strategic Classification

Abstract: In this talk, I will discuss online learning in the presence of strategic behavior. In this setting, a sequence of agents arrive one by one and wish to be classified as positive. They observe the current prediction rule and manipulate their features in predefined ways, modeled by a manipulation graph, in order to receive a positive classification. We show how to achieve a bounded number of mistakes when the target function belongs to a known hypothesis set and give upper and lower bounds showing how the graph structure impacts the achievable mistake bounds.

 Based on joint work with Avrim Blum and Kunhe Yang.

____

Speaker: Pranjal Awasthi

Title: Improving Length-Generalization in Transformers via Task Hinting

Abstract: It has been observed in recent years that transformers have problems with length generalization for certain types of reasoning and arithmetic tasks. This work proposes an approach based on task hinting towards addressing length generalization. Our key idea is that while training the model on task-specific data, it is helpful to simultaneously train the model to solve a simpler but related auxiliary task as well. We study the classical sorting problem and show that task hinting significantly improves length generalization.

Speaker: Bo Li

Title: Assessing Trustworthiness and Risks of Generative Models

Abstract: Large language models have captivated both practitioners and the public with their remarkable capabilities. However, the extent of our understanding regarding the trustworthiness of these models remains limited. The allure of deploying adept generative pre-trained transformer (GPT) models in sensitive domains like healthcare and finance, where prediction errors can carry significant consequences, has prompted the need for a thorough investigation. In response, our recent research endeavors to design a unified trustworthiness evaluation platform for large language models from different safety and trustworthiness perspectives. In this talk, I will briefly introduce our platform DecodingTrust, and discuss our evaluation principles, red-teaming approaches, and findings, with a specific focus on GPT-4 and GPT-3.5.

The DecodingTrust evaluation platform encompasses a wide array of perspectives, including toxicity, stereotype bias, adversarial robustness, out-of-distribution robustness, robustness against adversarial demonstrations, privacy, machine ethics, and fairness. Through these diverse lenses, we examine the multifaceted dimensions of trustworthiness that GPT models must adhere to in real-world applications. Our evaluations unearth previously undisclosed vulnerabilities that underscore potential trustworthiness threats. Among our findings, we unveil that GPT models can be easily steered to produce content that is toxic and biased, raising red flags for their deployment in sensitive contexts. Furthermore, our evaluation identifies instances of private information leakage from both training data and ongoing conversations, unveiling a dimension of concern that necessitates immediate attention. We also find that despite the superior performance of GPT-4 over GPT-3.5 on standard benchmarks, GPT-4 exhibits heightened vulnerability when faced with challenging adversarial, possibly due to its meticulous adherence to instructions, even when misleading. This talk aims to shed light on some critical gaps in the trustworthiness of language models and provide an illustrative demonstration of the safety evaluation platform DecodingTrust, hoping to pave the way for more responsible and secure machine learning systems in the future.

____

Speaker: Ren Wang

Title: Robust Mode Connectivity-Oriented Defense

Abstract: Deep learning models exhibit significant vulnerabilities to adversarial attacks, especially as the strategies of attackers advance in sophistication. Traditional adversarial training (AT) methods do offer a degree of protection, but their typically narrow focus on defending against a single form of attack—such as the L_inf-norm attack — limits their effectiveness against a variety of attacks. In this talk, I will present a new approach to adversarial defense centered around robust mode connectivity (RMC), encompassing two distinct phases of population-based learning. The initial phase is designed to explore the model parameter space between two pre-trained models, identifying a pathway populated with points that exhibit heightened robustness against diverse L_p attacks. Given the proven effectiveness of RMC, a secondary phase — RMC-based optimization — is introduced, wherein RMC acts as the foundational unit to further amplify the diversified L_p robustness of neural networks. Following this, I will unveil a computationally streamlined defense strategy named the Efficient Robust Mode Connectivity (EMRC) method. This technique integrates the L_1- and L_inf-norm AT solutions, ensuring enhanced adversarial robustness across a spectrum of p values.

____

Speaker: Haifeng Xu

Title: An Isotonic Mechanism for Overlapping Ownership

Abstract: This paper extends the Isotonic Mechanism from the single-owner to multi-owner settings, in an effort to make it applicable to peer review where a paper often has multiple authors. Our approach starts by partitioning all submissions of a machine learning conference into disjoint blocks, each of which shares a common set of co-authors. We then employ the Isotonic Mechanism to elicit a ranking of the submissions from each author and to produce adjusted review scores that align with both the reported ranking and the original review scores. The generalized mechanism uses a weighted average of the adjusted scores on each block. We show that, under certain conditions, truth-telling by all authors is a Nash equilibrium for any valid partition of the overlapping ownership sets. However, we demonstrate that while the mechanism’s performance in terms of estimation accuracy depends on the partition structure, optimizing this structure is computationally intractable in general. We develop a nearly linear-time greedy algorithm that provably finds a performant partition with appealing robust approximation guarantees. Extensive experiments on both synthetic data and real-world conference review data demonstrate the effectiveness of this generalized Isotonic Mechanism.

____

Speaker: Han Zhao

Title: Fair and Optimal Prediction via Post-Processing

Abstract: To mitigate the bias exhibited by machine learning models, fairness criteria can be integrated into the training process to ensure fair treatment across all demographics, but it often comes at the expense of model performance. Understanding such tradeoffs, therefore, underlies the design of optimal and fair algorithms. In this talk, I will first discuss our recent work on characterizing the inherent tradeoff between fairness and accuracy in both classification and regression problems, where we show that the cost of fairness could be characterized by the optimal value of a Wasserstein-barycenter problem. Then I will show that the complexity of learning the optimal fair predictor is the same as learning the Bayes predictor, and present a post-processing algorithm based on the solution to the Wasserstein-barycenter problem that derives the optimal fair predictors from Bayes score functions. I will also present the empirical results of our fair algorithm and conclude the talk with some discussion on the close interplay between algorithmic fairness and domain generalization.