Factor Analysis

Factor analysis is used to find hidden factors that predict the values of observed variables within a set of data.

Factor analysis is commonly used in fields like psychology, finance, and social sciences to identify underlying relationships between observed variables and to reduce the number of variables in a dataset.

Factor analysis works by modeling the observed variables as linear combinations of potential factors plus error terms. The goal is to identify the underlying factors that explain the patterns of correlations among the observed variables. This is achieved through techniques such as maximum likelihood estimation or principal factor analysis.

For example, consider a psychological study where various observed variables such as responses to survey questions are measured. Factor analysis can be used to identify underlying factors such as “anxiety” or “depression” that explain the correlations between the survey responses.

Factor analysis is mathematically similar to principal component analysis (PCA) to the extent that some authors and software suppliers regard the two as synonymous. However, the aims of the two algorithms are different. PCA aims to describe the observed data with a reduced number of dimensions, while factor analysis attempts to explain the relationships between the variables. If the aim is feature discovery and especially trying to understand the relationships between variables rather than merely model them, factor analysis should be preferred to PCA.