Multidimensional scaling (MDS) and multidimensional unfolding (MDU) are widely used methods for embedding a general square or rectangular proximity matrix into a low dimensional Euclidean space. They can be used both as a preprocessing step for further analysis such as classification or clustering, as well as a visualization tool in its own right (Cox & Cox 2001, Borg & Groenen 2005). Even though MDS and MDU have a relatively long history, new applications with larger and more complex data sets require the development of more powerful statistical methods. As high dimensional data are becoming commonplace in an increasing number of applications, dimensionality reduction and visualization tools such as MDS and MDU are more important than ever.

Regularization methods provide powerful tools to deal with ill-posed problems and/or high dimensional data. The success of regularization lies in the balance between goodness-of-fit and certain model constraints. In this thesis, we develop a general framework for penalized MDS (Chapter 2), penalized supervised MDS (Chapter 3) and penalized MDU (Chapter 4). The proposed methods are very flexible in accommodating features/prior knowledge of the data, handling transformed, noisy and truncated data. We develop efficient numerical methods for computing solutions to the penalized least square and penalized likelihood. We adopt cross validation to estimate the tuning parameter. We investigate a particular form of penalty that shrinks objects towards their centers and accounts for the prior knowledge about clusters among objects. Extensive simulations indicate that penalized estimation provides significant better configurations than traditional unpenalized methods, especially when observations are non-Euclidean and/or errors are large. This novel penalized approach leads to a flexible alternative to the traditional MDS and MDU methods.

This thesis is organized as follows. Chapter 1 provides a review of the MDS and MDU and discusses limitations of some existing methods. In Chapter 2, we present the penalized MDS formulation, numerical methods and simulation results. In Chapter 3, we present the supervised MDS criterion and demonstrate its potential as a tool for data visualization using the simulated and real data. In Chapter 4, we present the penalized MDU framework, numerical methods, simulation results and application to a real data set.