Generalized Linear Models Fitting with Slab and Shrinkage Estimators

savvy_glm.fit2 is a modified version of glm.fit that enhances the standard iteratively reweighted least squares (IRLS) algorithm by incorporating a set of shrinkage estimator functions. These functions (namely St_ost, DSh_ost, SR_ost, GSR_ost, and optionally Sh_ost) provide alternative coefficient updates based on shrinkage principles. The function allows the user to select one or more of these methods via the model_class argument.

Usage

savvy_glm.fit2(x, y, weights = rep(1, nobs),
                        model_class = c("St", "DSh", "SR", "GSR", "Sh"),
                        start = NULL, etastart = NULL, mustart = NULL,
                        offset = rep(0, nobs), family = gaussian(),
                        control = list(), intercept = TRUE,
                        use_parallel = TRUE)

Arguments

x: A numeric matrix of predictors. As for glm.fit.
y: A numeric vector of responses. As for glm.fit.
weights: An optional vector of weights to be used in the fitting process. As for glm.fit.
model_class: A character vector specifying the shrinkage model(s) to be used. Allowed values are "St", "DSh", "SR", "GSR", and "Sh". If a single value is provided, only that method is run. If multiple values are provided, the function runs the specified methods in parallel (if use_parallel = TRUE) and returns the best one based on Akaike Information Criterion (AIC). When the user does not explicitly supply a value for model_class, the default is c("St", "DSh", "SR", "GSR") (i.e. "Sh" is not considered).
start: Starting values for the parameters. As for glm.fit.
etastart: Starting values for the linear predictor. As for glm.fit.
mustart: Starting values for the mean. As for glm.fit.
offset: An optional offset to be included in the model. As for glm.fit.
family: A description of the error distribution and link function to be used in the model. As for glm.fit.
control: A list of parameters for controlling the fitting process. As for glm.fit.
intercept: A logical value indicating whether an intercept should be included in the model, defalut is TRUE. As for glm.fit.
use_parallel: Logical. If TRUE, enables parallel execution of the fitting process. Defaults to TRUE. Set to FALSE for serial execution.

Value

The value returned by savvy_glm.fit2 has the same structure as the value returned by glm.fit. It includes the following components:

coefficients: the estimated coefficients.
residuals: the working residuals, that is the residuals in the final iteration of the IWLS fit.
fitted.values: the fitted mean values, obtained by transforming the linear predictors by the inverse of the link function.
R: the upper-triangular factor of the QR decomposition of the weighted model matrix.
rank: the numeric rank of the fitted linear model.
qr: the QR decomposition of the weighted model matrix.
family: the family object used.
linear.predictors: the final linear predictors.
deviance: the deviance of the final model.
aic: the AIC of the final model.
null.deviance: the deviance for the null model.
iter: the number of iterations used.
weights: the final weights used in the fitting process.
prior.weights: the weights initially supplied.
df.residual: the residual degrees of freedom.
df.null: the residual degrees of freedom for the null model.
y: the response vector used.
converged: a logical value indicating whether the IRLS iterations converged.
boundary: a logical value indicating whether the algorithm stopped at a boundary value.
time: the time taken for the fitting process.
chosen_fit: the name of the chosen fitting method based on AIC.

Details

savvy_glm.fit2 extends the classical Generalized Linear Model (GLM) fitting procedure by evaluating a collection of shrinkage‐based updates during each iteration of the IRLS algorithm. When multiple shrinkage methods are specified in model_class (the default is c("St", "DSh", "SR", "GSR"), thereby excluding "Sh" unless explicitly provided), if the user explicitly includes "Sh" (for example, model_class = c("St", "DSh", "SR", "GSR", "Sh") or model_class = c("St", "Sh")), then the method Sh_ost is also evaluated.

The function can run these candidate methods in parallel (if use_parallel = TRUE). The final model is selected based on the lowest AIC. In cases where any candidate model returns an NA or non-finite AIC (which may occur when using quasi-likelihood families), the deviance is used uniformly as the model-selection criterion instead.

In essence, the function starts with initial estimates (via start, etastart, or mustart) and iteratively updates the coefficients using the chosen shrinkage method(s). The incorporation of these shrinkage estimators aims to improve convergence properties and to yield more stable or parsimonious models under various data scenarios. Shrinkage Estimators Used in the IRLS Algorithm:

Stein Estimator (St)

Computes a single multiplicative shrinkage factor applied uniformly to all coefficients.
The factor is chosen to minimize the overall mean squared error (MSE) of the OLS estimator.
Best suited when the predictors are similarly scaled so that uniform shrinkage is appropriate.

Diagonal Shrinkage (DSh)

Extends the Stein estimator by calculating an individual shrinkage factor for each coefficient.
Each factor is derived from the magnitude and variance of the corresponding coefficient.
Provides enhanced flexibility when predictor scales or contributions differ.

Simple Slab Regression (SR)

Penalizes the projection of the OLS estimator along a fixed, predefined direction.
Uses a closed-form solution under a slab (fixed-penalty) constraint.
Suitable when prior information suggests shrinking the coefficients along a known direction.

Generalized Slab Regression (GSR)

Generalizes SR by incorporating multiple, data-adaptive shrinkage directions.
Typically employs the leading eigenvectors of the design matrix as the directions.
Provides adaptive regularization that effectively handles multicollinearity.

Shrinkage Estimator (Sh)

Computes a non-diagonal shrinkage matrix by solving a Sylvester equation.
Transforms the OLS estimator to achieve a lower mean squared error.
This estimator is evaluated only when "Sh" is explicitly included in the model_class argument.

References

Marschner, I. C. (2011). glm2: Fitting Generalized Linear Models with Convergence Problems. The R Journal, 3(2), 12–15. doi:10.32614/RJ-2011-012. Available at: https://doi.org/10.32614/RJ-2011-012.

Asimit, V., Cidota, M. A., Chen, Z., & Asimit, J. (2025). Slab and Shrinkage Linear Regression Estimation. Retrieved from https://openaccess.city.ac.uk/id/eprint/35005/.

SavvyGLM article. https://ziwei-chenchen.github.io/savvyGLM.

Author

Ziwei Chen and Vali Asimit
Maintainer: Ziwei Chen <ziwei.chen.3@citystgeorges.ac.uk>