On the Generalization Ability of Next-Token-Prediction Pretraining

Zhihao Li, Xue Jiang, Liyuan Liu, Xuelin Zhang, Hong Chen and Feng Zheng.

International Conference on Machine Learning 2025 [C]

• This study establishes a theoretical framework for Next-Token-Prediction (NTP) pre-training based on Rademacher complexity, introduces a novel decomposition method, and provides the first generalization bounds for NTP. The findings offer valuable insights into how model parameters influence generalization and have been empirically validated, advancing both the theoretical comprehension and practical application of large language models.

Generalized Sparse Additive Model with Unknown Link Function

Peipei Yuan, Xinge You*, Hong Chen, Xuelin Zhang, and Qinmu Peng.

IEEE International Conference on Data Mining 2024 [C]

• In this work, we propose a novel generalized additive model with a flexible link function automatically learned by a bilevel scheme. The proposed model is capable of nonlinear approximation, hidden interaction and feature selection, which also enjoys the theoretical guarantee of algorithmic convergence.

Error Density-dependent Empirical Risk Minimization

Hong Chen*, Xuelin Zhang, Tieliang Gong, Bin Gu, Feng Zheng.

Expert Systems With Applications 2024 [J]

• This paper goes beyond the limitation of error value-dependent learning criterion and proposes the EDERM framework for robust regression against atypical data. The effectiveness of our method is validated by sufficient empirical evaluations. The implementation codes can be found at: https://github.com/zhangxuelincode/EDERM

Fine-grained Analysis of Stability and Generalization for Stochastic Bilevel Optimization

Xuelin Zhang, Hong Chen*, Bin Gu, Tieliang Gong, Feng Zheng.

International Joint Conference on Artificial Intelligence 2024 [C] (Oral)

• In this paper, we provide a systematical generalization analysis of the first-order gradient-based bilevel optimization methods, based on the (on-average argument) algorithmic stability technique. The verification codes are provided at: https://github.com/zhangxuelincode/BilevelOptimization

Improved Concentration Bound for CVaR

Peng Sima, Hao Deng*, Xuelin Zhang, Hong Chen.

International Joint Conference on Neural Networks 2024 [C]

• This paper introduces a novel estimator that relies on an estimator of Value at Risk (VaR) and investigates the concentration inequalities in independent scenarios where the underlying distributions are sub-Gaussian, sub-exponential, or heavy-tailed, where the inequalities we derive are bilateral, exhibit exponential decay, and are not confined to bounded scenarios.

Neural Partially Linear Additive Model

Liangxuan Zhu, Han Li*, Xuelin Zhang, Lingjuan Wu, Hong Chen.

Frontiers of Computer Science 2023 [J]

• This paper proposes a Neural Partially Linear Additive Model (NPLAM), which automatically distinguishes insignificant, linear, and nonlinear features in neural networks, which can realize model-level interpretability.

Stepdown SLOPE for Controlled Feature Selection

Jingxuan Liang, Xuelin Zhang, Hong Chen*, Weifu Li, Xin Tang.

Association for the Advancement of Artificial Intelligence 2023 [C] (Oral)

• This paper goes beyond the previous concern of Sorted L-One Penalized Estimation (SLOPE) limited to the false discovery rate (FDR) control by considering the stepdown-based SLOPE in order to control the probability of k or more false rejections (k-FWER) and the false discovery proportion (FDP). The codes are shared at: https://github.com/zxlml/SLOPE

Robust variable structure discovery based on tilted empirical risk minimization

Xuelin Zhang, Yingjie Wang, Liangxuan Zhu, Hong Chen, Han Li, Lingjuan Wu*.

Applied Intelligence 2023 [J]

• In this paper, we propose a new robust variable structure discovery method for group lasso based on a convergent bilevel optimization framework, where the robust tilted empirical risk minimization is adopted. The implementation codes can be found at: https://github.com/zhangxuelincode/demoTERMGL

Robustness of classifier to adversarial examples under imbalanced data

Wenqian Zhao, Han Li, Lingjuan Wu*, Liangxuan Zhu, Xuelin Zhang, Yizhi Zhao.

International Conference on Computer and Communication Systems 2022 [C]

• In this paper, we provide a theoretical framework to analyze the robustness of classifier to AE under imbalanced dataset from the perspective of AUC (Area under the ROC curve), and derive an interpretable upper bound.

Towards Precise and Explainable Hardware Trojan Localization at LUT Level

Hao Su, Wei Hu, Xuelin Zhang, Dan Zhu, Lingjuan Wu*.

IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems 2025 [J]

• The proposed approach aims to extract the rich structural and behavioral features at look-up-table (LUT) level to train an explainable graph neural network (GNN) model for classifying design nodes in FPGA netlists and identifying the Trojan-infected ones. The implementation codes can be found at: https://github.com/zhangxuelincode/node_label

Pinpointing Hardware Trojans Through Semantic Feature Extraction and Natural Language Processing

Yichen Li, Wei Hu, Hao Su, Xuelin Zhang, Yizhi Zhao, Lingjuan Wu*.

International Test Conference in Asia 2024 [C]

• In this work, we propose a novel hardware Trojan detection method at RTL. Our approach involves the transformation of hardware design into CDFG, followed by path extraction and segmentation.

Automated Hardware Trojan Detection at LUT Using Explainable Graph Neural Networks

Lingjuan Wu, Hao Su, Xuelin Zhang, Yu Tai, Han Li, Wei Hu*.

International Conference on Computer-Aided Design 2023 [C]

• In this work, we propose a novel hardware Trojan detection method based on graph neural networks (GNNs) targeting FPGA netlist. We leverage the rich explicit structural features and behavioral characteristics at LUT, which offers an ideal abstraction level and granularity for Trojan detection. A GNN model with optimized class-balanced focal loss is trained for automated Trojan feature extraction and classification. Model implementation is available at https://github.com/zxlml/XGNN_HT_Detection

Hardware Trojan Detection at LUT: Where Structural Features Meet Behavioral Characteristics

Lingjuan Wu, Xuelin Zhang, Siyi Wang, Wei Hu.

International Symposium on Hardware Oriented Security and Trust 2022 [C]

• This work proposes a novel hardware Trojan detection method that leverages static structural features and behavioral characteristics in field programmable gate array (FPGA) netlists. Mapping of hardware design sources to look-up-table (LUT) networks makes these features explicit, allowing automated feature extraction and further effective Trojan detection through machine learning. The implemented codes are available at: https://github.com/zxlml/HOST22