Defending Dignity A Manual for National Human Rights Institutions on Monitoring Economic, Social and Cultural Rights 6.4. HOW CAN WE JUDGE SECONDARY DATA? With the proliferation of data in today’s “information age”, it is important to be able to judge how appropriate secondary data is for what you want to do with it. Secondary data should always be interpreted cautiously and there are a number of issues to think about in this regard. For example, what happens if there is a difference between data provided by global databases and national databases? This is not uncommon due to different methodologies of data collection, statistical analysis and interpretation or computational method. Different data sources should be compared with attention to these details. Concept Explanation Potential issues Validity Data must reflect what it is trying to measure (i.e. the fulfilment of a right) as closely and as accurately as possible. Most secondary data is collected for uses other than human rights monitoring and so must be repurposed. This reinterpretation can be challenging. Reliability Refers to the consistency or dependability of the data. In other words, data collected multiple times in the same way should deliver similar results. Ambiguities or biases in the way data is collected (e.g. how survey questions are framed or how a population is sampled) might make data unreliable. Impartiality Data must be collected in a way that respects scientific independence and in an objective, professional and transparent manner. A national statistical office that is not independent may “spin” numbers so that a situation appears better than it is. Ability to be disaggregated To be most useful, data should be disaggregated (or able to be disaggregated) by, for example, sex, ethnicity, age, economic status, disability and geography. The original dataset is often needed in order to disaggregate the data, which is not always easy to obtain. These criteria are influenced by the following questions: • • • • How are the questions for the data collection framed? How broadly is it collected (i.e. what is the sample size)? Who is the data collected by? How frequently is the data collected? Reviewing “meta-data”, which is essentially data about the data, can help to answer these questions. It provides information such as the computation methods used, the measures of variability (discussed further in Chapter 8) and the description of errors in the data, such as bias and sampling errors. The following table considers these questions for common types of secondary data: Characteristics Administrative data Surveys Censuses Breadth All events registered All sampled units All units (100%) Frequency On-going 3-5 years 10 years Bias Can be incomplete Usually biased, but good design can minimize this Theoretically no bias Potential for rights monitoring Very good Good Good, but not in the short-term Source: OHCHR, HRBA Training Materials (not dated) 66