Not-Positive-Definite-Matrices--Causes-and-Cures

Not PositiveDefinite Matrices-Causesand CuresTheseminal workon dealingwith not positive definitematrices isWothke

1993.Thechapter isboth readableand comprehensive.This pageuses ideasfrom Wothke,fromSEMNET messages,and frommy ownexperience.The ProblemThereare foursituations inwhich aresearcher may get a message abouta matrix being Hnotpositive definite.The foursituations canbe verydifferent interms of their causesandcures.First,the researchermay geta messagesaying that the inputcovariance or correlationmatrix being analyzed isnnot positive definite.Generalized leastsquares GLSestimationrequires that the covariance or correlation matrix analyzedmust be positive definite,andmaximum likelihoodML estimationwill alsoperform poorlyin suchsituations.If thematrix to be analyzed isfound to be not positive definite,many programswill simplyissuean errormessage andquit.Second,the messagemay referto the asymptotic covariance matrix.This is not thecovariance matrixbeinganalyzed,but rathera weight matrixto be usedwith asymptoticallydistribution-free/weighted leastsquares ADF/WLS estimation.Third,the researchermaygetamessagesaying thatits estimateof Sigma2,themodel-implied covariance matrix,is not positive definite.LISREL,for example,willsimply quitif it issues thismessage.Fourth,the programmay indicatethat someparameter matrix within the model is notpositive definite.This attributeis onlyrelevant toparameter matrices that arevariance/covariance matrices.In thelanguage of the LISRELprogram,these includethematrices Theta-delta,Theta-epsilon,Phi中and Psi.Here,however,this nerrormessage0can resultfrom correctspecification of the model,so theonly problem is convincingtheprogram tostop worryingabout it.ft“Not PositiveDefinite-What DoesIt MeanStrictlyspeaking,a matrix is“positive definiteif allof itseigenvalues arepositive.Eigenvalues arethe elements of avector,e,which resultsfrom thedecomposition of asquare matrixS as:S=eMeTo anextent,however,we candiscuss positive definiteness interms of the sign of the“determinant of the matrix.The determinant is ascalar functionof the matrix.In thecaseof symmetricmatrices,such ascovariance or correlation matrices,positive definitenesswilonly holdif the matrix andevery principalsubmatrix1has a positive determinant.Principal submatricesare formedby removingrow-column pairsfrom theoriginalsymmetric matrix.A matrixwhich failsthis testis Hnot positive definite.If thedeterminant of thematrix isexactly zero,then thematrix isnsingular.n Thanksto MikeNeale,Werner Wothkeand MikeMiller forrefining thedetails here.Why doesthis matterWell,for onething,using GLSestimation methodsinvolvesinverting the input matrix.Any textonmatrixalgebra willshow thatinverting a matrixinvolves dividing by thematrix determinant.So ifthematrix is singular,then invertingthematrix involvesdividingbyzero,which isundefined.Using MLestimation involvesinvertingSigma,but sincethe aimto maximizethe similaritybetween the input matrixandSigma,the prognosisis notgood ifthe input matrix isnot positive definite.Now,someprograms includethe optionof proceedingwith analysiseven iftheinputmatrix isnotpositive definite—with Amos,for example,this isdone byinvoking the$nonpositivecommand—but it is unwiseto proceed without anunderstanding of the reasonwhy thematrix isnotpositive definite.If the problem relates to theasymptotic weightmatrix,theresearcher may not beable toproceed withADFAVLS estimation,unless the problem canberesolved.In addition,one interpretationof the determinantofa covariance or correlation matrix is asa measureof ngeneralizedvariance.n Sincenegative variancesare undefined,and sincezerovariances applyonly toconstants,itistroubling when a covariance orcorrelation matrixfails tohave apositive determinant.Another reasonto carecomes frommathematical statistics.Sample covariancematrices aresupposedtobepositive definite.For thatmatter,so shouldPearson andpolychoriccorrelation matrices.That isbecause thepopulation matricesthey aresupposedlyapproximating*are*positive definite,except undercertain conditions.So thefailure ofamatrix tobepositive definite mayindicate a problem with theinputmatrix.Why isMy MatrixNot PositiveDefinite,and WhatCanI DoAbout ItProperly,the questionis,why doesthematrixcontain zeroor negative eigenvalues.However,it may be easierfor manyresearchers tothink aboutwhy thedeterminantiszeroor negativeEither way,there aremany possibilities,and there are differentpossiblesolutions thatgo with each possiblecause.Further,there areother solutionswhich sidestepthe problem without reallyaddressing itscause.These optionscarry potentiallysteep cost.They arediscussed separately,below.Linear DependencyAnotpositivedefinite inputcovariance matrixmay signala perfectlinear dependencyofone variableon another.For example,if aplant researcherhad dataon cornmaize stalks,and twoof the variables in the covariance matrix wereplant height1and plantweight/thelinear correlationbetween thetwo wouldbe nearlyperfect,and thecovariance matrixwouldbe notpositivedefinitewithin samplingerror.It may be easierto detectsuchrelationships bysight ina correlation matrix rather than a covariance matrix,but oftentheserelationships arelogically obvious.Multivariate dependencies,where severalvariablestogether perfectlypredict anothervariable,maynot be visuallyobvious.In thosecases,sequential analysis ofthecovariance matrix,adding onevariable ata timeand computingthedeterminant,should helpto isolatethe problem.I woulduse aspreadsheet programforthis,like MicrosoftTM ExcelTM,for convenience.Dealing withthis kindof probleminvolves changing the set of variables included inthecovariance matrix.If twovariables areperfectly correlatedwitheachother,then onemaybe deleted.Alternatively,principal componentsmay beused toreplace asetofcollinearvariables withone or more orthogonalcomponents.In regardto theasymptotic weightmatrix,the lineardependency existsnot betweenvariables,but betweenelementsofthe momentsthe meansand variancesand covariancesorthe correlationswhich arebeinganalyzed.This canoccur inconnection withmodelingmultiplicative interactionrelationships betweenlatent variables.Jdreskog andYang1996show howmoments ofthe interactionconstruct arelinear functionsof momentsof themaineffect11constructs.Their articleexplores alternativeapproaches forestimating thesemodelsErrorReading theDataIf the problemiswith yourinputmatrixin particular,first make sure that the programhasread yourdata correctly.Remember,an emptycovariancematrix with novariablesinit isalwaysnotpositivedefinite.Try readingthe datausing anotherprogram,which willallowyou tovalidate thecovariancematrixestimated bythe SEMprogram.If yougenerated thecovariancematrixwithone program,and areanalyzing itwith another,makesure that thecovariancematrix wasread correctly.This canbe particularlyproblematic when theasymptotic weightmatrixis the focusofthe problem.Typographical ErrorWhenevera covariancematrixistranscribed,there isa chanceof error.So ifyou justhavethe matrixsay,from apublished article,but notthe dataitself,double-check fortranscriptionerrors.Also rememberthat journalsare notperfect,so acovariancematrixinan articlemay alsocontain an error.In arecent case,for example,it appearedthatthesignof asingle relativelylarge coefficientwas reversedat somepoint,and thisreversal madethematrix notpositivedefinite.In thatcase,changingthesignofthat onecoefficienteliminated theproblem.Starting ValuesThemodel-implied matrixSigma iscomputed fromthe modefsparameter estimates.Especially beforeiterations begin,those estimatesmaybesuch thatSigma isnot positivedefinite.So iftheproblemrelatestoSigma,first makesurethatthe modelhas beenspecifiedcorrectly,with nosyntax errors.If theproposed modelis unusualJ then thestarting valueroutines thatare incorporatedinto mostSEM programsmay fail.Then itis upto the researcherto supplylikely startingvalues.Sampling VariationWhensample sizeis small,a samplecovarianceorcorrelation matrixmaybe not positivedefinitedue tomere samplingfluctuation.As mostmatrices rapidlyconverge on thepopulation matrix,however,this initself isunlikely tobeaproblem.Anderson andGerbing1984documented howparameter matricesTheta-Delta,Theta-Epsilon,Psi andpossiblyPhi maybe notpositivedefinitethrough meresampling fluctation.Most often,such casesinvolvenimproper solutions/where somevariance parametersare estimatedas negative.Insuch cases,Gerbing andAnderson1987suggested thatthe offendingestimates could befixed tozero withminimal harmto theprogram.Estimators oftheasymptotic weightmatrixconverge muchmore slowly,so problemsdueto samplingvariation canoccur atmuch largersample sizesMuth6nKaplan,1985,

1992.Using anasymptotic weightmatrixwithpolychoric correlationsappears tocompoundtheproblem.Where samplingvariation isthe issue,Yung andBentler1994have proposeda bootstrappingapproach toestimating theasymptoticweightmatrix,whichmay avoidtheproblem.Missing DataLargeamounts ofmissing datacan lead to acovarianceorcorrelation matrixnot positivedefinite.With simplereplacement schemes,the replacementvalue maybe atfault.Withpairwise deletion,theproblemmay ariseprecisely becauseeach elementofthecovariancematrix iscomputed froma differentsubset ofthe casesArbuckle,

1996.To checkwhetherthis isthe cause,use a different missing data technique,such asa differentreplacementvalue,listswise deletionor perhapsideally amaximum likelihood/EMCOV simultaneousestimationmethod.My VariableisaConstant!Sometimes,either throughanerrorreading dataor throughthe processof deletingcasesthat includemissingdata,it happensthat somevariable ina dataset takeson only a singlevalue.In otherwords,one ofthe variablesis actuallya constant.This variablewill thenhavezero variance,and thecovariancematrix will benotpositivedefinite.Simpletabulation ofthe datawill providea forewarningof this.If thisistheproblem,either theresearchermust chooseadifferentmissing-data strategy,or elsethevariablemust bedeleted.Polychoric CorrelationsProgramsthat estimatepolychoric correlationson apairwise basis-one correlationat atime—may yieldinput correlationmatricesthatare notpositivedefinite.Here theproblemoccurs becausethe wholecorrelationmatrixisnotestimated simultaneously.It appearsthatthis ismost likelytobeaproblemwhenthe correlationmatrixcontains largenumbers ofvariables.Try computingamatrixof Pearsoncorrelations andsee whetherthe problempersists.If theproblem lieswiththepolychoric correlations,there maynotbea goodsolution.Oneapproach isto usea program,like EQS,that includesthe optionof derivingall polychoriccorrelationssimultaneously,ratherthanone ata timecf.，Lee,PoonBender,

1992.Butbe warned—Joop Hoxreports thatthe computationalburden isenormous,and itincreasesexponentially withthe numberof variables.Ed Cookhas experimentedwith aneigenvalue/eigenvector decompositionapproach.If acovarianceorcorrelationmatrixisnotpositivedefinite,then oneormoreof itseigenvalueswill benegative.After decomposingthecorrelationmatrix intoeigenvalues andeigenvectors,Ed Cookreplaced thenegativeeigenvalueswith small.05positive values,used thenew valuesto computeacovariancematrix,then standardizedthe resultingmatrixdiving bythe squareroot ofthe diagonalvalues sothattheresult wasagain wasacorrelation matrix.Ed reportedthatthebias resultingfrom thisprocess appearedto besmall.No ErrorVarianceSometimes researchersspecify zeroelements on the diagonalsof Theta-delta orTheta-epsilon.A zerohere impliesno measurementerror.While itmay seemunlikely,onreflection,that anylatent variablecouldbemeasured withouterror,nevertheless thepracticeis common,whenaconstruct hasonlyasingle measure.Single measuresoften leadtoidentification problems,and analystsmay leavethe parameterfixed atzero bydefault.Ifa diagonalelement isfixed tozero,thenthematrixwillbenotpositivedefinite.However,since thisis preciselywhat the researcher intendedto do,there isno causefor alarm.Theonly problemis thatthese valuesmay causethe solutionto failan admissibility check,which mayleadtopremature terminationoftheiterative estimationprocess.In suchcases,itismerely amatter ofdisabling theadmissibilitycheck.In LISREL,for example,this isdoneby addingAD=OFF to the OUtputline.Negative ErrorVarianceNegative valuesonthediagonal areanother matter.Since thediagonal elementsof thesematricesare varianceterms,negative valuesare unacceptable.Further,since theseerrorvariances representthe nleft-overn partof somevariable,a negativeerror variancesuggeststhat theregression hassomehow explainedmore than100percent ofthe variance.In myownexperience,these valuesare symptomsofaserious fitproblem.Comprehensive fitassessmentwill helptheresearcherto isolatethe specificproblem.Sidestepping theProblemAs withmany problems,thereareways tosidestep thisproblemwithoutactually tryingtodiscern itscause.Besides simplycompelling theprogram toproceedwithits analysis,researchers canmake aridge adjustmenttothecovarianceorcorrelationmatrix.Thisinvolves addingsome quantitytothediagonal elementsofthematrix.This additionhas theeffectof attenuatingthe estimatedrelations betweenvariables.A largeenough additionissure toresult inapositivedefinite matrix.The priceof thisadjustment,however,is biasinthe parameterestimates,standard errors,and fit indices.Partial leastsquares methodsmayalso proceedwith noregard forthedeterminantofthematrix,but thisinvolves anentirelydifferent methodology.ReferencesAnderson,J.C.，Gerbing,D.W.

1984.The effectof samplingerror onconvergence,improper solutions,and goodness-of-fitindicesfor maximumlikelihood confirmatoryfactoranalysis.Psychometrika,492—June,155-

73.Arbuckle,J.L.

1996.Full informationestimation inthe presenceof incompletedata.In G.A.MarcoulidesR.E.Schumacker Eds.,Advanced structuralequation modeling:Issuesand techniquespp.243-

78.Mahwah,NJ:Lawrence Erlbaum.Gerbing,D.W・,Anderson,J.C.

1987.Improper solutionsintheanalysisofcovariancestructures:Their interpretabilityand acomparison ofalternate respecifications.Psychometrika,521—March,99-

111.Joreskog,K.G.,Yang F.[now FanYang Jonsson]

1996.Nonlinear structuralequationmodels:The Kenny-Judd modelwith interactioneffects.In G.A.MarcoulidesR.E.Schumacker Eds.,Advanced structuralequation modeling:Issuesand techniquespp.57-

88.Mahwah,NJ:Lawrence Erlbaum.Lee,S.-Y.,Poon,W.-Y.,Rentier,P.M.

1992.Structural equation models withcontinuousand polytomousvariables.Psychometrika,571—March,89-

105.Muthen,B.Kaplan,D.

1985.A comparisonof somemethodologies forthe factoranalysisof non-normal Likertvariables.British Journalof Mathematicaland StatisticalPsychology,38,171-

89.Muthen,B.Kaplan,D.

1992.A comparisonof somemethodologies forthe factoranalysisof non-normal Likertvariables:A noteonthesize ofthemodel.British JoumalofMathematical and Statistical Psychology,45,19-

30.Wothke,W.

1993.Nonpositive definitematrices instructural modeling.In K.A.BollenJ,S.Long Eds.,Testing structuralequationmodelspp.256-

93.Newbury Park,CA:Sage.Yung,Y.-F.，Bentler,P.M.

1994.Bootstrap-corrected ADFtest statisticsin covariancestructureanalysis.British Journalof MathematicalandStatisticalPsychology,47,63-

84.。