This is the inaugural attempt to systematically evaluate the quality and completeness of homicide data in Mexico at a sub-national level.
IEP’s Índice de Datos sobre Homicidios (IDH) is the inaugural attempt to quantify the Bogota Protocol on Quality of Homicide Data in Latin America and the Caribbean and systematically evaluate the quality and completeness of state-level homicide data in Mexico. The index, and this first report, is designed to be a tool for identifying opportunities for improvement in state-level public security information systems.
Why data matters
As the world becomes increasingly data based, the quality of data matters ever more. Evidence-based policy has gained global momentum over the last 25 years, as modern information systems allow us to better evaluate policies and programs, identify risks and unintended consequences, and draw connections between policy interventions and desired outcomes. However, the efficacy of evidence-based policy strongly depends upon the quality of the evidence. Good data enriches the work of the human analyst, be they detective, social scientist or policy maker. Bad data reinforces the shortcomings in existing systems.
Two years ago, IEP’s annual Mexico Peace Index (MPI) reported that the state of peacefulness in Mexico was returning to pre-drug war levels. Long-running justice and policing reforms appeared to be proving effective and violence was subsiding; it seemed as if the country was on track to recover. However, 2017 saw a severe escalation of violence, with the national homicide rate rising 25% and surpassing its 2011 peak. With the drug war now in its twelfth year, Mexico urgently needs a viable and effective plan to end the crisis of violence. And yet, accurate information about what has transpired – and what might stop it – remains difficult to obtain and analyse. Higher quality homicide data is vital for evidence-based violence reduction policy that will effectively address homicide rates in Mexico. The IDH is designed to provide a reference for state governments, provide an accountability tool for civil society, and ultimately improve the development of evidence-based public policy.
The report finds that homicide data quality is poor throughout Mexico, with no state scoring above 6.5 out of 10 on the index. On average, states achieved 14.8 of the index’s 43 indicators and no state completed more than 29. The index is broken into five domains that measure different aspects of homicide data quality and completeness. Of the five domains, the average score of states was above 50 percent in only two—convergencia entre fuentes (convergence between sources) and calidad de datos (data quality), which measures the percentage of missing information in the data. Nuevo León, Querétaro, Aguascalientes, Baja California and Coahuila scored the highest on the index, while Ciudad de México, Yucatán, Nayarit, Tabasco and Hidalgo scored the lowest. On average, states do best in the domain of convergencia entre fuentes (convergence between sources) and worst in that of definición (definition), as many incorrectly include events such as suicides, accidents and attempted homicides in the definition. The top five states significantly outperform the rest in detalle de las informaciones (detail of the information), which measures the data’s inclusion of several details about the homicide.
Only three states were able to provide data at the victim level—the preferred structure that allows for in-depth analysis, and only San Luis Potosí provided it in the accessible database format. No state has a validation body to compare victim data between sources, missing out on the opportunity for outside experts to contribute to data collection processes. Additionally, none of Mexico’s official datasets records whether a homicide is believed to be related to organized crime—a pressing issue in the country.
The case for high quality data
High-quality data benefits both the institutions that generate it and wider society. In the case of law enforcement and public security, homicide data should improve the ability of police and justice officials to investigate and prosecute cases. Collaboration with researchers should support the development of effective techniques, while the release of complete and accurate public information builds trust amongst the population. Strong institutions develop legitimacy in the eyes of the civil society when they are transparent and accountable. However, all of this depends largely on the quality of the data itself—the quality of quantitative insights is only as good as the quality of the data they are based on.