20 Chasing the Wind: Assessing Philippine Democracy democracy. In a particular case using a single index for measuring political democracy and stability, Bollen (1990: 12) notes that much confusion results because one is unable to establish "whether democracy or stability is responsible for any relations found." His curt advice to errant colleagues: "Distinct concepts should be treated distinctly." Munck and Verkuilen (2002: 14-31), reviewing measures designed by other prominent researchers, also call attention to the conflation in several democratic attributes for which indices had been designed. In practically all of these cases, they point out that not enough rigor was exercised in thinking through the nuances of concepts that the indices would measure. The theoretical relationships among the various democratic attributes as well as their operationalizing indicators not being explicitly delineated, much conflation could not be avoided. Leading researchers, academic institutions and quasi-academic groups undertaking the measurement of democracyrelated concerns (e.g. Freedom House and Transparency International) have invited mostly constructive criticisms triggered by inadequate or poor conceptualization work. Beyond conceptual inadequacies, other shortcomings are also noted by recent assessments of democracy-measurement work. The following is a list of commonly encountered flaws in the design and operation of several democracy measures: level-of-measures confusion, conflation induced by using multiple indicators where one or a few would suffice, or, on the other hand, ill-advised singular-indicator designs where several are actually needed, inappropriate choice of indicators, arbitrary, non-theoretically justified, "rule-of-thumb" thresholds for establishing the substantive quality or level of democracy or democratization, subjective data coding, unclear rationale for employed weighting procedures, questionable validity and problematic reliability and replicability concerns as well as the use of inappropriate statistical techniques. Many of these infirmities are encountered even in the most comprehensive and the most popularly used measures of democracy and related topics; the measures and datasets employed by Freedom House, Polity IV and Transparency International have been among the targets of academic criticism in recent years. Individual academics' measures of democracy are also often found wanting by reviewers—often long-time, well-meaning and constructive professional colleagues— assessing the metrics. (Vanhanen 2000; Munck and Verkuilen 2002; Munck 2009; Bogaards 2007 and 2010; Bayer and Bernard 2010; Knutsen 2010). In their quest for competent democracy metrics, academics have found certain guidelines helpful. Bohlen (1990: 19) notes that most of the problems identified above could be avoided if those involved in measuring work tried to: