ThRee-way methods In Chemistry And Psychology


among other things


A cross-disciplinary conference on three-way analysis


Perle Roi Resort & Spa

Stavros Akrotiriou, ChaniaCrete, Greece



Sun. June 4, 2006 - Fri. June 9, 2006


Three-way chess board

Goal of the meeting:  TRICAP2006 aims to bring together the world’s leading researchers in three-way analysis. In the TRICAP tradition, participants will be invited by the ad-hoc TPC. TRICAP was established by researchers in chemometrics and psychometrics. Starting with TRICAP2000, signal processing researchers with a strong interest in three-way analysis are also invited. Continuing this tradition, TRICAP2006 is open to suggestions for a limited number of three-way researchers from other disciplines. The philosophy of the meeting is to provide a relaxed atmosphere with plenty of opportunities for the exchange and cross-fertilization of technical ideas.


Prof. Nikos Sidiropoulos

Dept. of ECE, Technical University of Crete

731 00 Chania - Crete, Greece

Phone +30-28210-37227; Fax +30-28210-37542

Ad-Hoc Technical Program Committee (TPC):

Rasmus Bro

Richard A. Harshman

Henk Kiers

Lieven DeLathauwer

Age Smilde,

Pieter Kroonenberg,

Barry M.Wise

Eva Ceulemans

Roma Tauler,

Bill Rayens

Nikos Sidiropoulos


Evrim Acar, RPI, U.S.

Andre de Almeida, I3S/CNRS/UNSA, FR

Brett W. Bader, Sandia National Labs, U.S.

J.M.F. ten Berge, Univ. of Groningen, NL

Rasmus Bro, KVL, DK

Heather Bush, Univ. KY, U.S.

Pierre Comon, Univ. Nice, France

Lieven De Lathauwer, KU Leuven, BE

Mohamed Hanafi, ENITIA, FR

Richard A. Harshman, Univ. Western Ontario, CA

Ilgis Ibragimov, Univ. Saarbrucken, DE

Elena Ibragimova, Elegant Mathematics, DE

Henk Kiers, Univ. Groningen, NL

Tamara Kolda, Sandia National Labs, U.S.

Theodora Kourti, McMaster Univ., CA

Pieter Kroonenberg, Leiden Univ., NL

Riccardo Leardi, Univ. Genova, IT

Iven van Mechelen, KU Leuven, BE

Morton Morup, DTU, DK

Pentti Paatero, Univ. Helsinki, FI

Sarah C. Rutan, Virginia Commonwealth Univ., U.S.

Nikos Sidiropoulos, TUC, GR

Alwin Stegeman, Univ. Groningen, NL

Hong Sungjin, Univ. Illinois Urbana-Champaign, U.S.

Paul Taylor, McMaster Univ., CA

Roma Tauler, CSIC, Spain

Marieke Timmerman, Univ. Groningen, NL

Giorgio Tomasi, KVL, DK

Alex O. Vasilescu, New York Univ., U.S.

Barry M.Wise, Eigenvector Research, U.S.

Important dates:

  • Early registration & hotel reservation deadline: Feb. 10, 2006
  • Workshop: Sun. June 4, 2006 - Fri. June 9, 2006

Workshop registration & hotel reservation forms:

When: Sun. June 4, 2006 - Fri. June 9, 2006


Where: Perle Roi Resort, Stavros Akrotiriou, ChaniaCrete, Greece, Here is a view from the top:























The picturesque town of Chania in the island of Crete is one of the prime tourist destinations in Greece. Early June is a particularly good time to visit Chania. Well before the peak tourist season, yet the weather is already very pleasant.


Photo of the old Venetian harbor, Chania - Greece.

Travel: Chania is served by 7 daily flights from/to Athens. The easiest way to get here is direct flight to Athens, then Athens à Chania. During tourist season (April to October) there are many direct charter flights from many European cities to Chania.

Chania airport code is CHQ, Athens is ATH. Aegean airlines

and Olympic airlines offer service. Flight time is 45 minutes from Athens. The town is a short 15 minute taxi ride from the airport, and so is the hotel. 


·        Technical University of Crete 

Links to previous TRICAPs:

TRICAP2003, June 22-27, 2003, Lexington, Kentucky, U.S.A:

TRICAP2000: July 18-21, 2000, Faaborg, Denmark:

TRICAP97: May 4-9, 1997, Lake Chelan, Washington, U.S.A:

Next TRICAP (2009): Contact Roma Tauler,

Links to other 3-way Workshops:

AIM Workshop on Tensor Decompositions,

SIAM-CNRS Workshop on Tensor Decomposition and Applications,

Directions to TRICAP2006 photo gallery, courtesy of Pieter Kroonenberg:

Go to:

Click on: Conference photos (or conference photographs)

Choose Tricap 2006

type password in the space provided; click on submit (do not use return)

Off you go. Thumbnails are clickable. Large high resolution pictures may be requested from:



Technical & Social Program


Sunday, June 4, 2006


·        Optional pre-conference day trip to Knossos Minoan palace & archeological museum

of Herakleion, for those that arrive on Saturday, June 3. Depart around 8am Sunday

morning; return around 6pm Sunday evening.


·        Conference starts:  Welcome reception & dinner @  8:00pm (all activities at Perle Roi resort, unless

          otherwise noted) 


Monday, June 5, 2006


8:00-9:00:                    Registration

9:15-9:30:                    Welcome & opening remarks:

1.      Michael Paterakis, Associate Dean, TUC;

2.      Nikos Sidiropoulos


Session:                Applications & Exploratory Data Analysis (session chair: Rasmus Bro)


  9:30am-10:20am:       Brett W. Bader, Richard A. Harshman, Tamara G. Kolda

Improvements to Three-way DEDICOM for Applications in Social Network Analysis slides

10:20am-10:40am:       Coffee break

10:40am-11:30am:       Barry Wise (+ Dora Kourti)

Re-fitting PCA, MPCA and PARAFAC Models to Incomplete Data Records slides

11:40am-12:30pm:       Sarah E. G. Porter, Sarah C. Rutan, Dwight R. Stoll, Peter W. Carr and Jerry D. Cohen,

Multi-way Analysis of 2D Liquid Chromatographic Metabolomics Data slides



12:45pm-2:00pm:   Lunch


Session:                Models – I (session chair: Lieven de Lathauwer)


  2:00pm-  2:50pm:       Rasmus Bro,

Less Used but Useful slides

 3:00pm-  3:50am:       André L. F. de Almeida, Gérard Favier, João C. M. Mota,

The block-PARAFAC decomposition slides

  3:50pm-  4:10pm:       Coffee break

 4:10pm-  5:00pm:       Iven Van Mechelen, Eva Ceulemans, and Jan Schepers,

The family of hierarchical classes models: A state-of-the-art overview slides

  5:10pm-  6:00pm:       M. Alex O. Vasilescu,

Incremental Multilinear SVD


6:00pm-8:30pm:     Leisure time


8:30pm:                 Dinner


Tuesday, June 6, 2006


Session:                Theory & Algorithms (session chair: Henk Kiers)


  8:30am-  9:20am:       Jos M.F. ten Berge,

Typical rank when the arrays have symmetric slices, and the Carroll-Chang conjecture of equal CP components slides

  9:30am-10:20am:       Alwin Stegeman,

Why do degenerate solutions occur in Candecomp/Parafac? slides

10:20am-10:40am:       Coffee break

10:40am-11:30am:       Giorgio Tomasi,

Recent developments in fast algorithms for fitting the PARAFAC model  slides

11:40am-12:30pm:       Slot open for discussion


12:45pm-2:00pm:   Lunch


Session:                Exploratory Data Analysis (session chair: Richard A. Harshman)


  2:00pm-  2:50pm:       Tamara G. Kolda,

Multilinear algebra for analyzing data with multiple linkages slides

  3:00pm-  3:50am:       Romà Tauler,

Including trilinear and restricted Tucker3 models as a constraint in Multivariate Curve

Resolution Alternating Least Squares method slides

  3:50pm-  4:10pm:       Coffee break

  4:10pm-  5:00pm:       Henk A.L. Kiers & Eva Ceulemans,

Choosing the numbers of components in three-way component models slides

  5:10pm-  6:00pm:       Riccardo Leardi,

How to convince people that 3-way PCA is useful and easy slides


6:00pm-7:30pm:     Leisure time


7:30pm:                 Bus leaves for dinner @ Monastery restaurant at the old Venetian harbor, downtown




(return to hotel around 11:30 pm)


Wednesday, June 7, 2006


Session:                Theory / Uniqueness (session chair: Jos ten Berge)


  8:30am-  9:20am:       Alwin Stegeman & Nikos Sidiropoulos,

Kruskal's uniqueness condition for the Candecomp/Parafac decomposition slides

  9:30am-10:20am:       Pierre Comon, Lek-Heng Lim, Bernard Mourrain

Generic properties of symmetric tensors slides

10:20am-10:40am:       Coffee break

10:40am-11:30am:       Heather Bush, Bill Rayens

Khatri-Rao products and conditions for the uniqueness of PARAFAC

solutions for IxJxK arrays slides

11:40am-12:30pm:       Lieven de Lathauwer,

The decomposition in block terms slides


12:45pm-2:00pm:   Lunch


2:30pm:                 Bus leaves for half-day trip to Elafonisos (“Deer island”) beach on SW corner of Crete;




Dinner @ Elos village, on the way back


(return to the hotel around 11:00pm)


Thursday, June 8, 2006


Session:                Statistical Theory & Method (session chair: Pierre Comon)


  8:30am-  9:20am:       Marieke E. Timmerman, Henk A.L. Kiers, Age K. Smilde, Cajo J.F. ter Braak,

Bootstrap Confidence Intervals in Variants of Component Analysis slides

  9:30am-10:20am:       Pieter M. Kroonenberg,

Multiple imputation in three-mode analysis: A research program slides

10:20am-10:40am:       Coffee break

10:40am-11:30am:       Pentti Paatero,

Estimating and minimizing uncertainty in factor analytic results, both in 2-way and in 3-way models slides


Session:                Models – II (session chair: Barry Wise)


11:40am-12:30am:       Richard A. Harshman,

Generalization of Parafac and Tucker models to Canonical Correlation models slides


12:45pm-2:00pm:   Lunch


  2:00pm-  2:50pm:       Morten Mørup,

Convolutive PARAFAC Models and Applications slides

  3:00pm-  3:50am:       Sungjin Hong

Hidden Parafac2 slides

  3:50pm-  4:10pm:       Coffee break

  4:10pm-  5:00pm:       Mohamed Hanafi,

PLS Path Modelling: Computation of latent variables with the estimation mode B slides


6:00pm-7:30pm:     Leisure time


  7:30pm-  8:15pm:       Special! -- Dinnertime feature by Pieter Kroonenberg:    

Three-mode analysis for Dummies and some history of the Tucker model address

  8:30pm:                      Dinner & discussion: TRICAP2009


Friday, June 9, 2006


Session:                Algorithms (session chair: Pieter Kroonenberg)


  8:30am-  9:20am:       Ilgis Ibragimov,

                                    The parallel decomposition

  9:30am-10:20am:       Elena Ibragimova,

                                    Multidimensional decomposition: Different problems need different solutions

10:20am-10:40am:       Coffee break

10:40am-11:30am:       Chair’s report, and decision regarding TRICAP2009 (… winner is … RomaTauler! -


12:45pm-2:00pm:   Lunch & Farewell: conference ends 




Improvements to Three-way DEDICOM for Applications in Social Network  Analysis


Brett W. Bader, Sandia Labs, U.S.A.

Richard Harshman, Univ. Western Ontario, CA

Tamara G. Kolda, Sandia Labs, U.S.A.


This presentation revisits the DEDICOM (DEcomposition into Directional COMponents) family of models and applies them to new applications in data mining, in particular social network analysis.  The DEDICOM model is a tool developed in the late 1970's for analyzing asymmetric relationships in data analysis and has been revisited in the multiway community over the years.  In this work we present an improved algorithm for computing the 3-way DEDICOM model, including modifications that make it possible to handle large, sparse data matrices.  Then we will demonstrate the capabilities of DEDICOM as a new tool in social network analysis.  For an application we consider the email communications of former Enron employees that were made public, and posted to the web, by the U.S. Federal Energy Regulatory Commission during its investigation of Enron.  We represent the Enron email network as a directed graph with edges labeled by time and construct its corresponding adjacency tensor.  Using the three-way DEDICOM model on this data, we show unique latent relationships that exist between types of employees and study their communication patterns over time.

Re-fitting PCA, MPCA and PARAFAC Models to Incomplete Data Records


Barry M. Wise

Eigenvector Research, Inc.


Many processes generate multivariate or multiway data sequentially. In calibration mode this is not generally a problem, one just waits until all the data is in, then sets about modelling. Often, particularly in process applications, it is desired to know how well the model represents the incoming data before the complete record is available. Several options have been proposed to deal with this problem. Some of the methods are based on in-filling the missing data so that the models may be applied in the usual way. This approach, however, suffers from all the problems associated with missing data. How does one fill in the record, particularly when the missing parts are systematic, not random? An alternative approach is considered here. Models are fit to partial data records by simply truncating the model loadings to coincide with the available data and fitting the partial factors using a classical least squares (CLS) approach. The estimated scores and residuals are found to converge to those of the entire record quite rapidly in the data sets considered. In fault detection applications the implies that it is often possible to detect a bad batch well before its completion.


Multi-way Analysis of 2D Liquid Chromatographic Metabolomics Data


Sarah E. G. Portera, Sarah C. Rutana, Dwight R. Stollb, Peter W. Carrb and Jerry D. Cohenb,

aDepartment of Chemistry, Box 842006, Virginia Commonwealth University, Richmond, VA 23284-2006, USA, bUniversity of Minnesota, Minneapolis, MN, 55455, USA


Multi-way data analysis methods are ideally suited for the study of metabolomics data.  In this work, we have analyzed 4-way data arising from a 2D liquid chromatography analysis of plant extracts.  Diode array detection was used in this instance, but the method is also applicable to mass spectrometric detection.  The data are quadrilinear, with the absorbance in mAU units as the dependent variable, and the independent variables being retention on the first dimension column, retention on the second dimension column, UV-visible wavelength, and sample identity. The overall data set was 84 x 263 x 75 x 6 (first dimension, second dimension, wavelength and sample, respectively). As the chemical rank of this data set is on the order of 90, we subdivided the 2D chromatograms and analyzed the data in smaller 4-way sections. The approach that we used included evaluation of the rank of the matricised data in as a function of wavelength, first column retention times and second column retention times. The maximum rank was used for a PARAFAC analysis, with nonnegativity as a constraint using the program of Bro.  Further iterations were carried out with an alternating least squares algorithm with a flexible unimodality constraint being applied the chromatographic profiles that did not appear to be associated with a background component. The result from this analysis was used as an initial guess for a final PARAFAC analysis. The results provide the relative concentrations of approximately 70 known and unknown signaling compounds in two wild type and two mutant maize seedlings.




Rasmus Bro


This talk describes some methods that are not commonly used in multi-way analysis but which have interesting properties.  Multi-way regression is useful in several application areas and there are many ways to implement regression in higher-order analysis. Here it will be shown how partial least squares regression can be extended to and made useful for  multi-way data. Another interesting multi-way model is PARAFAC2. This model is very useful in situations where the uniqueness properties of the PARAFAC model is needed, but where the PARAFAC model itself is too strict to adequately model the data. We will show how PARAFAC2 can help overcome problems for several different types of data.


The Block-PARAFAC decomposition

André L. F. de Almeida, Gérard Favier, João C. M. Mota

Laboratoire I3S/CNRS/UNSA,

2000, Route des Lucioles,

06903 Sophia Antipolis, France

We present a block-tensor decomposition that mixes the properties of the Parallel Factor (PARAFAC) and Tucker decompositions. Block-PARAFAC is a decomposition of a third-order tensor into a sum of Q structured PARAFAC blocks. Constrained linear combinations or interactions involving columns of different component matrices are permitted and modeled by means of constraint matrices. It is shown that the Block-PARAFAC decomposition is equivalent to a constrained Block-Tucker3 decomposition with orthogonal unfolded cores. Block-PARAFAC enjoys the so-called between-block uniqueness, which stands for the inherent separability of the Q structured PARAFAC blocks. Different levels of within-block partial uniqueness are possible, depending on the particular constrained interaction structure. Applications of Block-PARAFAC to wireless communication signal processing problems such as blind beamforming and multi-antenna coding have been developed, and will be briefly discussed.


The family of hierarchical classes models: A state-of-the-art overview


Iven Van Mechelen, Eva Ceulemans, and Jan Schepers


Hierarchical classes models are decomposition models for N-way data, that can be considered N-way Tucker models, subject to two major modifications: (1) Component scores for all reduced modes are constrained to be binary, (2) in the decomposition the Sum operator is replaced by the Max-operator.  The first modification implies that all reduced modes are subject to an overlapping clustering; the second implies that the models include a representation of quasi-order (implication-type) relations for all reduced modes, which may be defined in quite a natural way (provided the core, and therefore the reconstructed data, take nonnegative values only).  Mathematically speaking, the hierarchical classes family can be considered decomposition models in the exotic semiring (R^+, max, x). In this talk, I will present a state-of-the-art overview of the research on hierarchical classes models and their associated algorithms. On the level of modeling, this overview will include recent extensions of the family to models for rating- and real-valued data, and various types of constrained hierarchical classes models.  On an algorithmic level, simulation results on the performance of a broad range of estimation procedures will be summarized, including most recent results on the performance of local search heuristics.


Typical rank when arrays have symmetric slices, and the Carroll-Chang conjecture of equal CP components

Jos M.F. ten Berge

University of Groningen


The question of how many components are needed to decompose a three-way array by Candecomp/Parafac has led to results on typical rank. These imply that, for arrays sampled randomly from a continuous distribution, certain rank values (numbers of components needed for perfect fit) arise with positive probability. In certain applications, like a scalar product fitting problem that arises in the context of INDSCAL, the slices of the array are symmetric. Ten Berge, Sidiropoulos and Rocci (2004) have shown that such arrays tend to have either the same or smaller typical ranks than their asymmetric counterparts. They also showed that CP, upon convergence, has A and B column-wise proportional in a majority of cases. But exceptions also arise with positive probability. In the present paper, some of these results are discussed and extended. It is shown that 2´J´J arrays and 3´2´2 arrays can almost surely be transformed to have symmetric slices, implying that typical ranks for such arrays formats are the same regardless of symmetry of the slices (Ten Berge & Stegeman, in press). Next, the problem of CP-solutions for the INDSCAL-related case is treated. In such cases, A and B are required to be equal. Carroll and Chang (1970) conjectured that the symmetry of the slices will force component matrices A and B to be equal upon convergence. Indeed, when unconstrained CP is applied, we will have A=B in a vast majority of cases. However, exceptions have also been detected. These cases typically involve low k-rank of the third component matrix C. This will be fully explained for 4´3´3 arrays of rank 5 (Ten Berge & Stegeman, in preparation). It will be shown under what conditions such arrays admit low k-rank for C, and how to obtain alternative CP-solutions which do have A and B equal.


Why do degenerate solutions occur in Candecomp/Parafac ?


Alwin Stegeman


The Candecomp/Parafac (CP) model decomposes a three-way array into a prespecified number R of rank-1 arrays and a residual array, in which the sum of squares of the residual array is minimized. The practical use of CP is sometimes complicated by the occurrence of so-called degenerate solutions. In such cases, convergence of the CP algorithm is extremely slow, some components are highly correlated in all three modes and the elements of these components become arbitrarily large. We consider fitting the real-valued CP model to an I×J×K array X which is randomly sampled from an IJK-dimensional continuous distribution. For K = 2, we show for which combinations of I, J and R degenerate solutions may occur and offer a mathematical explanation for this. For K = 3, we consider all known cases in which the random array X has rank q with positive probability, and rank q+1 with positive probability (i.e. X has a two-valued typical rank). We show that if X has rank q+1 and R = q, then degenerate solutions may occur. Again, a mathematical explanation is offered. For the presented cases, degenerate CP solutions occur when the CP objective function does not have a minimum but an infimum. If this holds, then any sequence of feasible CP solutions of which the objective value approaches the infimum, will become degenerate. The underlying “cause” of degeneracy in the presented cases, is the occurrence of a two-valued typical rank. Also, it is shown that CP uniqueness is not necessary for degenerate solutions to occur.


Recent developments in fast algorithms for fitting the PARAFAC model


Giorgio Tomasi


The PARAFAC model is commonly used in chemometrics because of its uniqueness properties and because it corresponds to the physical model of several types of data (e.g. EEM fluorescence or chromatographic data). In spite of its conceptual simplicity, this model has proved to be particularly difficult to fit on real as well as simulated data sets. Several algorithms have been proposed for this purpose, with uneven results. In particular, none of the methods proposed thus far operates well in all conditions. For example the Alternating Least Squares algorithm is slow in case of high collinearity and large fractions of missing values whereas the Levenberg Marquadt algorithm may be too expensive for large problems. In this talk, a number of possibilities based on the Khatri-Rao product are shown that can greatly improve the efficiency of the ALS and all other methods and some possibilities are outlined based on QR decomposition that can be used to compress the array and reduce the computational expense. In particular, a compression will be outlined that reduce the computational complexity to O((I + J + K)F) for a three -way array and is compatible with nonnegativity constraints.


Multilinear algebra for analyzing data with multiple linkages


Tamara G. Kolda, Sandia National Labs


Abstract: Link analysis typically focuses on a single type of  connection, e.g., two journal papers are linked because they are written  by the same author. However, often we want to analyze data that has  multiple linkages between objects, e.g., two papers may have the same keywords and one may cite the other. The goal of this paper is to show  that multilinear algebra provides a tool for multi-link analysis. We analyze five years of publication data from journals published by the Society for Industrial and Applied Mathematics. We explore how papers can be grouped in the context of multiple link types using a tensor to represent all the links between them. A PARAFAC decomposition on the resulting tensor yields information similar to the SVD decomposition of a standard adjacency matrix. We show how the PARAFAC decomposition can be used to understand the structure of the document space and define paper-paper similarities based on multiple linkages. Examples are presented where the decomposed tensor data is used to find papers similar to a body of work (e.g., related by topic or similar to a particular author’s papers), find related authors using linkages other than explicit co-authorship or citations, distinguish between papers written by different authors with the same name, and predict the journal in which a paper was published. (This is joint work with Daniel M. Dunlavy and W. Philip Kegelmeyer.)


Including trilinear and restricted Tucker3 models as a constraint in Multivariate Curve Resolution Alternating Least Squares method


Romà Tauler

Department of Environmental Chemistry, IIQAB-CSIC, Jordi Girona 18-26, Spain



Constrained bilinear models implemented in Multivariate Curve Resolution Alternating Least Squares (MCR-ALS) method have been revealed extremely useful in solving rotational and intensity ambiguities associated to two-way factor analysis problems in different chemistry fields and in spectroscopy. Constraints applied to resolved profiles have included non-negativity, unimodality, closure, selectivity, local rank and physical and chemical laws. These models are easily extended to three-way and multiway data sets arranged in appropriate matrix augmentation (concatenation) schemes (matricization). The bilinear model assumed in MCR-ALS may include additional constraints concerning three-way structures like those implied in PARAFAC and Tucker models. These models may be implemented in the ALS algorithm in a very flexible way covering a wide range of possible situations, including mixed modeling situations where multilinear constraints are only partly fulfilled by the physical nature of the investigated data. Detailed explanation of different implemented algorithms and of possible application will be described and discussed.


Choosing the numbers of components in three-way component models


Henk A.L. Kiers & Eva Ceulemans


One of the main choices to be made in using three-mode component models is how many components to use. In the last five years, various new procedures have been proposed for choosing the number(s) of components for CANDECOMP/PARAFAC, Tucker3, Tucker2, and Tucker1 models. Here an overview will be given of these methods, and some experience with them in practice will be shown. Notably, the procedures proposed by Bro and Kiers (2003), Timmerman and Kiers (2000) and Kiers and der Kinderen (2003) will be described. In addition to choosing the numbers of components, the data analyst has to choose the particular type of model to use (e.g., CANDECOMP/ PARAFAC, Tucker3, or Tucker2). Ceulemans & Kiers proposed a procedure that combines these choices, i.e., that helps choosing among all types of models, and all sensible numbers of components to be used with these models. This procedure is an elaboration and variation of the procedure proposed by Kroonenberg & van der Voort (1987). Simulation results will be described, as well as some practical experiences, and problems.




Riccardo Leardi


Department of Chemistry and Food and Pharmaceutical Technologies- University of Genoa - ITALY


At present, the application of three-way methods for exploratory analysis and data display is still not as common as it should be. The main reasons for that are a widespread ignorance of the multi-way techniques and the generalized feeling that they can be used only by experts in the field. In order to make them common tools for every people using chemometrics a big effort should be performed to show how powerful they can be, even when used at their simplest level. The talk will present a set of easily understandable examples, taken from different fields, from which also the non experts in three-way methods can appreciate how powerful they are. Furthermore, it will be shown how the application of a simple algorithm removing the sign ambiguity makes the joint interpretation of the three loading plots absolutely straightforward.


Kruskal's uniqueness condition for the Candecomp/Parafac decomposition


Alwin Stegeman, Nikos Sidiropoulos


Let X be a real-valued three-way array of order I×J×K. The real-valued Candecomp/Parafac (CP) decomposition is written as X = Y(1) + … + Y(R) + E , where Y(r) are real-valued rank-1 arrays and E is a real-valued rest term. Each rank-1 array Y(r) is defined by the outer product of three vectors a(r) , b(r) and c(r), i.e. . These vectors make up the R columns of the component matrices A (I×R), B (J×R) and C (K×R). If 2R+2 is less than or equal to the sum of the k-ranks of A, B and C, then the fitted part of the decomposition is unique up to a change in the order of the rank-1 arrays Y(r) and rescaling/counterscaling of each triplet of vectors (a(r),b(r),c(r)) forming a rank-1 array. This classical result was shown by Kruskal in 1977. His proof is, however, rather inaccessible and does not seem intuitive. In order to contribute to a better understanding of CP uniqueness, this talk presents an accessible and intuitive proof of Kruskal's condition. The proof is both self-contained and compact and can easily be adapted to the complex-valued CP decomposition.


Generic properties of Symmetric Tensors

Pierre Comon, Lek-Heng Lim, Bernard Mourrain

Symmetric tensors are arrays whose entries do not change under any permutation of indices. Note that with this definition, symmetric arrays enjoy more symmetries than in the INDSCAL model where only matrix slices are requested to be symmetric. On the other hand, the advantage is that Symmetric Tensors of order d and dimension N can be bijectively associated with homogeneous polynomials of degree d in N variables. Yet, working with polynomials permits to resort to Algebraic Geometry results. In accordance with PARAFAC/CANDECOMP decomposition framework, a rank-one tensor is defined as the outer product of vectors, and the tensor rank is usually defined as the smallest number of rank-one tensors necessary to achieve a perfect fit. These definitions are assumed throughout the presentation. First, a precise definition of generic and typical ranks is given. This definition is compatible with the terminology previously admitted in the literature (eg. ten Berge). Limiting our discussion to the complex field, we then show that the closure of the sets of tensors of rank r can be ordered by inclusion (cf. Theorem 1). A sub-product of this Theorem, among others, is that the set of tensors of rank at most r is generally not closed, unless r=1. This is quite important, because it means that the problem of approximating a tensor by another of lower rank is an ill-posed problem. It is also pointed out that the generic rank is now known for any pair (d,N). This result can be actually obtained from two rather complicated results published by Alexander and Hirschowitz. One can imagine decomposing symmetric tensors into a sum of non symmetric rank-one terms. Therefore, if each rank-one tensor is constrained to be itself symmetric, the obtained rank could be larger; this rank is called the symmetric rank. We show that in fact the symmetric rank of any symmetric tensor equals its tensor rank (cf. Theorem 2). Lastly, examples are given in order to illustrate the previous results. In particular, a series of rank-2 order-3 tensors converging to a rank-3 limit is exhibited. This kind of example cannot exist for matrices.





Heather Bush, Univ. of KY

Bill Rayens, Univ. of KY

Utilizing Khatri-Rao products to represent the PARAFAC decomposition, ten Berge and Sidiropoulos’ seminal paper (2002) provided the first evidence for countering the necessity claim of Kruskal's uniqueness condition, identifying PARAFAC decompositions that were unique when Kruskal's condition was not met. Moreover, ten Berge and Sidiropoulos conjectured that Kruskal’s condition would have to be supplemented with additional k-rank restrictions if a set of necessary and sufficient conditions were to be had.   In our research, we adopt a different linear algebraic perspective on this problem and are able to provide: (1) the answers to lingering questions about the occurrence of uniqueness for a specific number of columns, (2) evidence that necessity would require more than a restriction on k-rank, (3) an explanation of uniqueness beyond k-rank, and (4) an approach that could be extended to cases beyond those investigated by ten Berge and Sidiropoulos.


The decomposition in block terms

Lieven De Lathauwer1,2

1Department of Electrical Engineering, Division ESAT-SCD, Katholieke Universiteit Leuven, Leuven, Belgium

2ETIS (CNRS, ENSEA, UCP), Centre National de la Recherche Scientifique, Cergy-Pontoise, France


In this talk we introduce a new tensor decomposition. The given tensor 'block' is decomposed in 'blocks of smaller dimensionality'. Several variants of this new decomposition will be presented. PARAFAC and the Tucker model can be considered as special cases. The result sheds new light on very fundamental aspects of tensor algebra, including tensor rank.


Bootstrap Confidence Intervals in Variants of Component Analysis


Marieke E. Timmerman, University of Groningen1

Henk A.L. Kiers, University of Groningen

Age K. Smilde, University of Amsterdam

Cajo J.F. ter Braak, Wageningen University


1Heymans Institute for Psychology (DPMG),Grote Kruisstraat 2/1, 9712TS Groningen, The Netherlands,


The bootstrap methodology can be used to estimate confidence intervals (CI’s) for the statistics in a component analysis. We discuss bootstrap CIs in three types of component analysis, namely for independent two-way data (Principal Component Analysis (PCA)), for time-dependent data obtained from replicates within experimental conditions (Principal Response Curve (PRC) model) and for hierarchically ordered data (Multilevel Component Analysis (MLCA)). We present an overview of possible strategies for bootstrap confidence intervals, and pay attention to the resampling scheme, methods for estimating the CI’s, and possible non-uniqueness of the estimated parameters. CI estimates using different methods in PCA may diverge highly. We explain that this results from differences in quality, and in perspective on the rotational freedom of the population parameters. A comparative simulation study shows that the bias-corrected and accelerated (BCa) method for bootstrap CI’s generally yields CI’s with good coverage properties. Moreover, the BCa method is more flexible and yields better CIs than its counterparts based on asymptotic standard errors. The PRC model analyzes multivariate data resulting from experiments involving repeated sampling in time. The time-dependent treatment effects are represented by Principal Response Curves (PRCs). A comparative simulation study shows that confidence bands for PRCs with good coverage can be obtained with BCa intervals using a nonparametric bootstrap, except for the case of exactly zero population PRCs for all conditions. Then, the behaviour is irregular. As will be explained, this is caused by the sign indeterminacy of the PRCs. MLCA analyses multilevel multivariate data. Such hierarchically ordered data require some particularities in the bootstrap scheme. The results of a comparative simulation study will be discussed.


Timmerman, M.E. (in press). Multilevel Component Analysis. British Journal of Mathematical and Statistical Psychology, xx, xxx-xxx.

Timmerman, M.E., Kiers, H.A.L. & Smilde, A.K. (in press). Estimating Confidence Intervals in Principal Component Analysis: A comparison between the Bootstrap and Asymptotic Results. British Journal of Mathematical and Statistical Psychology, xx, xxx-xxx.

Timmerman, M.E. & Ter Braak, C.J.F. (2006). Bootstrap Confidence Intervals for Principal Response Curves. Submitted.


Multiple imputation in three-mode analysis: A research program

Pieter M. Kroonenberg,

Department of Education and Child Studies, Leiden University

One of the ways to handle missing data is via multiple imputation, i.e.generating several data sets with differently imputed values for the missing data. The basic idea is to use generalised procrustes analysis to evaluate the stability of the parameters from the multiply imputed data sets. Attention is paid to the possibilities and impossibilities of using multiple imputation in three-mode analysis. Much of the research is still in its infancy and primarily ideas are presented rather than firm results.

Estimating and minimizing uncertainty in factor analytic results, both in 2-way and in 3-way models


Pentti Paatero

Using Bootstrap (BS) for estimating uncertainty of PCA is discussed by Timmerman et al in TRICAP2006. For non-negatively constrained 2-way factor analysis, such as PMF, BS is not a fully satisfactory method because there is usually rotational ambiguity in the results and customary BS cannot determine rotational ambiguity.  Recent (so far, unpublished) results by Paatero, Eberly and Hopke have shown that a combination of BS and random rotational “forcing” of the computed factors produces reliable confidence intervals (CI’s) for PMF factor elements. These two-way results are summarized in this talk. Generalization of this approach to PARAFAC is discussed: simplified, BS of a 3-way array means that randomly chosen subsets of slices of the array are analyzed and the variation of results between different subsets is used for deriving CI’s. Rotational forcing is effected so that randomly chosen elements of one (or of all) of factor matrices A, B, or C are “pulled up” (and “pulled down”) by auxiliary equations that are introduced in the Least-Squares (LS) model.

Paatero and Hopke (2003) have demonstrated qualitatively that in 2-way factor analysis, such as PCA or PMF, columns or rows with exceptionally low signal-to-noise (S/N) ratio should be omitted from the matrix in order to determine weak factors with best possible resolution. In this talk, quantitative criteria are reported for deciding when a given low-S/N row or column should be omitted (or downweighted) or kept in the matrix. These results are new. The criteria are extended to the Tucker T3 model.

In applications, all rows (columns) of matrices often possess similar S/N. However, transformations can be applied so that in the transformed matrix, some rows(columns) have extremely low S/N and hence should be omitted. Application to factor analysis of spectroscopy data is discussed as an example: in many spectra, there are smooth regions between sharp peaks. It is shown that in a smooth region, a group of adjacent spectral channels should be replaced by the average of these channels in order to enhance S/N of computed results. This result is also valid for 3-way models: then a group of adjacent slices is to be replaced by the average of those slices.

Reference: Pentti Paatero and Philip K. Hopke, Discarding or downweighting high-noise variables in factor analytic models. Analytica Chimica Acta  490 (2003) 277-289.


Generalization of Parafac and Tucker models to Canonical Correlation models

Richard A. Harshman 
University of Western Ontario, London, Ontario, Canada

Tucker and Parafac are methods of  internal relationship analysis”: they find patterns of relations within a single dataset. However, statistical data analysis also uses “external relationship analysis”. External analysis methods find relationships between patterns in one dataset and those in another, or between patterns in a dataset and those in a theoretical logical structure or ``design matrix''.  Bro and others have introduced external analysis into the three-way domain by deriving n-way extensions of PLS regression.  I have stumbled upon an alternative approach, one that extends the symmetric relation of (canonical) correlation. Canonical correlation finds weights for an optimal linear combination of the columns of one matrix, and weights for an optimal linear combination of the columns of another matrix  (the “other side” of the relation)  such that the two  vectors they define have maximal correlation. This is then repeated for vectors orthogonal to the first set, to get added dimensions of correlation between the two matrices.

     In statistics, canonical correlation is used to define the General Linear Model (GLM). By choice of arguments supplied to the GLM, one can obtain special cases that encompass a wide range of standard statistical techniques, including Univariate and Multivariate  Analysis of Variance and Covariance, Discriminant Analysis, Multiple Regression, analysis of counts and contingency tables, and even the simplest “special case”, group comparisons via the t-test.
     I will describe an “external analysis” analog of Tucker T3, called TUCCON, and a Parafac-like special case of it called PARACCON, which provide multilinear generalizations of canonical correlation and thus of the GLM. This directly leads to multi-way versions of all the GLM’s ``special-cases'', often providing the added benefits for these methods that multiway generalization gives to Factor Analysis. For example: when, on either side of the canonical relation, there is a higher-way array in which the relevant patterns have a unique Parafac decomposition, then there also be a unique decomposition of the dimensions defining  the “external relations'' between the two sides (i.e., of the “canonical space” on each side).  Also, the source for patterns on each side of a canonical relation can now involve more than one dataset per side, allowing an outer-product synthesis of patterns from several sources on one side to be related to, say, the outer-product patterns in a single higher-way array on the other. Estimation of the parameters for TUCCON and PARACCON is often possible using familiar methods like Alternating Least Squares or Paatero’s Multilinear Engine. [ I will sometimes need to use the array notation AIN described in an earlier TRICAP meeting and published in the ``Index Formalism...'' article in Journal of Chemometrics. Interested colleagues are encouraged to gain some advance familiarity with this notation —downloadable from]



Convolutive PARAFAC Models and Applications


Morten Mørup


The regular factor analysis only considers instantaneous mixing, i.e. . However, this analysis can be extended to a non-instantaneous mixing, i.e. forming the convolutive factor analysis. Lately, much attention has been given to convolutive factor analysis models since these models can account for echo effects, i.e. various forms of time delays from source to sensors. Recently, we’ve proposed to extend the convolutive factor analysis model to a model convolutive in both modalities, i.e. the double convolutive model . Furthermore, the convolutive factor analysis models also generalize to higher orders forming convolutive PARAFAC models, i.e. CPARAFAC. It will be demonstrated how CPARAFAC might be useful in the analysis of a wide range of signals.


Hidden Parafac2


Sungjin Hong

University of Illinois, Urbana-Champaign


Parafac is a three- and higher-way component model which yields a set of uniquely determined factors under mild conditions. Parafac2, as a variant of Parafac, relaxes the invariance condition of component weights for one of three data modes such that cross-products of component weights in the relaxed mode are invariant instead of the components themselves, also yielding a unique decomposition of n-way data. Parafac2 is suited well to the multiple-groups factor-analytic problems since the relaxed model condition allows for fitting multiple groups of distinctive observational units by reweighted, invariant component loadings. In general, multiple-groups analysis assumes known grouping of observation units. However, it is sometimes very desirable to consider the grouping itself as unknown model parameters to optimize (as in k-means clustering). Unlike a consecutive modeling of data reduction and clustering (e.g., k-means clustering on reduced-dimensional component space), Hidden Parafac2 (HIDPAR2) simultaneously estimates component weights for dimensional reduction and grouping memberships for classification, minimizing a unified fit function. From the mixture-modeling perspective, HIDPAR2 can be considered as a constrained component mixture modeling of two-way data by an (reweighted) invariant component loading pattern and hidden membership. An alternating least squares algorithm is developed for parameter estimation in which group membership is iteratively estimated by a fuzzy k-means step, given all other model parameters. Simulated data are used to assess performance of the algorithm.



PLS Path Modelling: Computation of latent variables with the estimation mode B


Mohamed Hanafi

ENITIAA-INRA, Unité de Sensometrie et Chimiometrie - Rue de la Géraudière BP 82225, 44322 Nantes Cedex


PLS Path Modelling shows a number of interesting advantages over others existing approaches traditionally used for structural modelling. However, the lack of convergence properties of the existing iterative procedures for the computation of the latent variables, has always been considered as one of the major drawbacks. The convergence is stated only in practice. The present paper shows that when the estimation mode B is chosen for all blocks, the iterative procedure for the computation of latent variables proposed by Wold (1985) is monotonically convergent.