trudeau-laptop_thumb-1.png This article is more than 7 years old

8 signs the data from the Trudeau government’s online electoral reform survey is useless

Is the data from Trudeau's "fun little questionnaire" reliable enough to make decisions on a new electoral system?


Can anyone trust the data from the Liberal government’s new online electoral reform survey?

Prime Minister Justin Trudeau seems to think so – last week, Trudeau suggested the “fun little questionnaire” at will help “drive a solution for electoral reform.”

And Democratic Institutions Minister Maryam Monsef said the online survey will “make sure that the values that (Canadians) feel are most important in a voting system” are “represented in the final outcome,” adding this is something she’ll consider “before we introduce legislation.”

Plus Mark Holland, parliamentary secretary for democratic reform, argued the fun quiz is actually a “scientific study done by professional scientists,” designed with “the integrity of the data” top of mind.

The quiz might have value as a fun educational tool – but can Trudeau’s Liberals seriously use it to justify their decision on a new electoral system?

At the same time, Monsef has been busy trashing an all-party parliamentary committee and her own electoral reform town halls, both of which endorsed proportional representation. Other Liberal MPs trashed those conclusions on the grounds that their own town halls are “not a representative sample of the Canadian population.”

So, how much integrity can the data from this fun little quiz really have? Here are several things to consider:


1. Bad questions

When the online survey first went live this week, many Canadians felt there was something fishy about the survey’s questions: 

One question framed proportional systems as a choice between wanting parliament to “represent the views of all Canadians” and opening the floodgates to “radical or extreme” parties:


Other questions follow a similar pattern, framing the electoral reform debate as a series of exaggerated and unrealistic choices.

Makes you wonder if this fun little quiz is really an attempt to sabotage proportional representation?


2. The explanation for their bad questions makes no sense

Defending the survey’s questions, Monsef claims these false dilemmas are crucial to understanding the underlying “values” Canadians have about democracy, that’s why they don’t directly ask which system they want.

Except that point is undercut by the fact the survey asks several direct questions about non-electoral system related issues like voting age, mandatory voting and the length of campaigns:


Apparently a simple yes or no is good enough for some issues, but not for others – namely issues where the Liberals have a different preference in mind

Not everyone strongly agrees with that approach, though:


3. We can’t get a straight story about who came up with the bad questions

Who wrote the questions?

Here’s a short list of the wide-range of answers we heard over this last week:

Asked earlier this week who designed the survey’s questions, Vox Pop Labs CEO Clifton van der Linden (whose company built the survey) would only tell PressProgress that the survey was “reviewed and approved by the government” and “an independent academic advisory panel.”

 Monsef told the House of Commons the survey is “a Canadian made application, formed by Canadian academic experts to engage Canadians.” 

• However, Monsef repeatedly refused to disclose the names of the members of the mysterious expert panel. 

 Vox Pop Labs eventually released the names of the expert panel. The Ottawa Citizen reported one panelist “confirmed that he was involved in the review process, although he noted that he didn’t actually see the final draft of the survey until he took it himself” and added that “they had no role in actually coming up with the questions.”

 The expert panel later released a statement stating they had, in fact, “proposed survey questions” and said “questions written by Vox Pop Labs” too – a detail the company didn’t initially volunteer.

• In the meantime, the government’s own hotline was apparently telling Canadians that an all-party parliamentary committee approved the questions in the survey – a claim that is entirely false.

One thing’s for sure: no one seems eager to take credit for this survey.

4. Polling experts say the data is “unusable”

Here’s how one Canadian pollster put it: 


5. One expert defending the survey admits the data has no “significance”

In an interview with Maclean’s titled “In defence of the Liberals’ electoral-reform survey,” Carleton University professor Philippe Lagassé argued the survey has value as an “educational tool.”

But if you read past the headline, Lagassé doesn’t actually sound like he thinks anyone should “compile this data and present it as some kind of decision point”:

“If the purpose is at the end of this to produce a report and say, ‘This is what people thought,’ then I agree, you can’t do that … I think everybody would readily attack them if they claimed this was a survey of some kind of significance if you’re able to re-take it over and over again.”

And yet, that is exactly what Trudeau, Monsef and Holland have all suggested they want to do.

6. No one will explain how they stop people from filling-out the survey multiple times

Another problem: the survey doesn’t ask for your name or attempt to verify your identity in any way.

How do they prevent anonymous Internet users from filling-out the survey multiple times? If you check out the FAQ page at, you’ll see a vague blurb claiming the survey has built-in “controls” that prevent “repeat entries or malicious use” of the survey.

Thing is, there’s no transparency about how these “controls” actually work.

Asked to elaborate on their “controls,” Vox Pop Labs would only tell PressProgress they have a “series of proprietary techniques” that “screen for multiple entries from the same user” – unfortunately, they say they can’t explain what they are or how they work because “disclosure of these techniques would embolden attempts to circumvent them.”

That explanation doesn’t quite square up with other details on

The same FAQ page also encourages rural and low-income Canadians who don’t have Internet access to go use the computer at “your MP’s office, a local library, friend or family member’s house” – in other words, explicitly encourages multiple people to fill-out the quiz using the same computer, something they supposedly screen against.


In an interview with Maclean’s this week, van der Linden said that “when we’ve identified observations that are assumed to be from the same user, we take the first set of responses.”

But asked if the survey is able to tell the difference between one person voting multiple times from home versus multiple persons voting from the same MP’s office, Vox Pop Labs said it wouldn’t be a problem because “in most cases it would be possible to tell the difference.”

The takeaway appears to be that you’re encouraged to fill-out the survey multiple times from the same computer, except they’ll block your responses if you do, and they’ll use techniques that must remain secret in order to discourage you from filling-out the survey multiple times from the same computer.

Sound like we’re going around in circles? Maybe greater transparency would avoid that problem.


7. The survey “excludes” over one million Canadian expat voters

Over one million Canadian voters live and work in other countries, and Monsef says she’s committed to meeting “the needs of highly mobile Canadian citizens who live in today’s increasingly interconnected world.”

However, it turns out those voters are actually blocked from submitting survey responses.

Vox Pop Labs told PressProgress they have “a series of proprietary techniques” that “screen for multiple entries from the same user – both at the point of collection and when screening the data – as well as to identify non-Canadian respondents.”

“It is likely that many Canadian expats will be excluded from our results,” van der Linden told PressProgress. “We do not have a control variable in the survey to identify them as such.”


8. Oh, and they may have excluded your results too

And a million expats might not be the only ones surprised to learn their survey results didn’t count.

Take, for example, anyone who follows the survey’s instructions.

When you get to the final page of the online survey, it asks you to fill out a few general demographic questions about where you live and how much you earn. A blurb on the page reads:

“This portion is completely optional. Even if you choose not to complete the questions in this section, you will still be presented with the full results.”


Well, it turns out that’s not true.

Tucked away in the privacy page of is a note that says the following:

“While answering the profile questions is optional, not answering these questions will result in your input not being included as part of the overall results of the study.”


For her part, Monsef says she “can assure all Canadians that providing personal information is completely optional” and “can assure all Canadians that we will incorporate all feedback into the final result,” assurances contradicted by what it literally says on her own website. 

What a fun little mess!

Photo: PMO.

Our journalism is powered by readers like you.

We’re an award-winning non-profit news organization that covers topics like social and economic inequality, big business and labour, and right-wing extremism.

Help us build so we can bring to light stories that don’t get the attention they deserve from Canada’s big corporate media outlets.


PressProgress is an award-winning non-profit news organization focused on uncovering and unpacking the news through original investigative and explanatory journalism.

Most Shared

thumb-2024-05-07-rainbow-rally-vancouver News

Take Back Alberta Leaders are Training ‘Scrutineers’ to Infiltrate Campaigns and Act as ‘Security’ on Voting Day

Related Stories


Hundreds of People Rallied in Vancouver against Anti-LGBTQ+ Hate. Here’s What They Had to Say.

View the post

International Students are Coming to Canada Dreaming of a Better Life. Many Are Leaving Disillusioned and In Debt.

View the post

After the assassination of a Sikh on Canadian soil, what did (or didn’t) the foreign interference commission reveal about India?

View the post
Our free email newsletter delivers award-winning journalism directly to your inbox.
Get Canadian Investigative News You Won't Find in Corporate Newspapers.
Our free email newsletter delivers award-winning journalism to your inbox.
Get Canadian Investigative News You Won't Find in Corporate Newspapers.