Strength in numbers: some provisional metrics for the sector

Visualization created by IBM of Wikipedia edits. (Photo: Viegas)

Rumour has it that 2013 is the international year of statistics (and quinoa and water cooperation, but we have less to say about that).

Given the globe’s current obsession with big data  (Wikipedia: a collection of data sets that are so big and complex they are difficult to process) and open data (often associated with big data released by governments) it seems an appropriate choice of year.

Big, open data is exciting a lot of people. Since information is power, handing over giga-bytes of information to people to mash up, manipulate and analyze, feels quite empowering. It popularizes the capacity of people to generate their own original data and evidence, and use it to shape policy.



But frankly, given all the hype, we have to admit we are kind of disappointed.

We were expecting all of this big data to tell us a lot more about what we actually wanted to know (and to get government to do more of what we wanted them to).

Over the past eight months, CCIC has been dabbling with some big data (Canada Revenue Agency data sets  on Canada’s 80,000 plus charitable organizations and CIDA’s historical data sets ) to generate some metrics (a fancy word for “useful numbers”) for the international development sector.

For example, how many organizations are there in Canada working on international development? How much money collectively are they generating from the Federal and Provincial government, and private individuals? How many individuals do these groups employ? And where are these groups working and on what?

It seemed a simple enough task.

We couldn’t have been more wrong.

What they don’t tell you about big data (“data sets that are so big and complex…”) is that they are, in fact, really big and complex (you’ll need a super-advanced database programmer and a Post-Doc statistician on your team). And despite the size and complexity, the data is limited, of course, to the ‘interests’ of whoever produced it. So the data can maybe tell you answers to really big and complex issues, but not necessarily the questions you want answers to. ‘What can the data tell us?’ is perhaps the only reliable question.

(As an aside, Development Initiatives in the UK is trying to develop some reliable and robust aggregate data for private giving for development across all OECD countries. They have met with similar challenges in the data they have tried to generate. And conversations they have had with other national platforms trying to develop their own metrics to feed into this project have echoed many of the challenges we outline here. Comfort in numbers!)

Case in point: all of the above questions.   

How many organizations work on international development? Using in part the CIDA data set, we came up with a figure of approximately 865 organizations.  This is already only a part answer – since it’s only the organizations which get funding from CIDA. The further challenge here was figuring out who we wanted to count and how we would go about counting them. In the end, we established a definition for the sector that included members of CCIC and members of the Provincial and Regional Councils from the past five years (minus overlap) and who had received funding from CIDA as civil society over the past five years (minus any consortia of groups). And in doing so, we also came up with our own new (albeit small) set of data. It likely generates a close, but still approximate, number, since some of the organizations we included may have dissolved since then, or the definition we developed to catch the field of actors may still not be broad enough – for example, we may still need to include organizations who have only gotten provincial funding but may not be members of any Council. Filling some of these gaps will be part of our next step.

How much Federal, Provincial and private money are they generating? From the CRA figures, in 2010, we can generate some very preliminary and approximate estimates: the total aggregrate revenue that organizations generate is about $2.6 billion, of which approximately $450,000 from the Federal Government, $350,000 from CIDA, $180,000 from the Provinces, $175,000 from private foundations and $1 billion from individuals, among other sources. And this is where things get really tricky (and even more approximate).

While the CRA (and the T3010 information they provide) is a mine of information, it is of course only good for registered charities. If we compare the number of organizations defined above, only 430 of them are Federally Registered. Furthermore, of those that are Federally registered, the numbers include 55 Universities and Colleges, whom have got contracts to do various development work, but including their (for example for revenue and employees) completely skews the broader results. The Red Cross and YMCA-YWCA similarly do a lot of development work but also have large domestic programs. And then what about the other 435 organizations that aren’t captured (because they are only registered provincially, are housed within another organization, or simply don’t need to issue charitable receipts?). A second phase of work will help refine these unwieldy figures.

And how many individuals do these organizations employ? Again, we think around 8,000; but some of the same reservations we have identified above also apply here. Equally in terms of where groups work and on what, the CIDA data sets can tell us a lot of information (which, nostra culpa, we still have to extract), but only for those organizations that get CIDA funding. It is a subset, albeit a large one, but of a bigger pie.
Looking forward, CCIC plans to work in collaboration with the Provincial and Regional Councils to fill some of these gaps and get a more comprehensive picture of the sector – check your inboxes soon for a short, but extremely important, survey.

The results to date have been encouraging (albeit generating more questions than answers). But the findings are clearly extremely important. More and more individuals and organizations are asking us for this data. And we are already getting to use some of it.

What better project for the international year of statistics!

This blog was written by Fraser Reilly-King, Policy Analyst (Aid), CCIC, and Al Webb, Volunteer and consultant. The views expressed are their own, and do not necessarily represent the views of CCIC or its members.