Public opinion research to provide evidence for the interpretation of the participation criteria for the leaders’ debates

Prepared for: Leaders’ Debates Commission, 155 Queen Street, Suite 301, Ottawa, Ontario
Supplier name: EKOS Research Associates, 359 Kent Street, Suite 300, Ottawa Ontario

Sample Design

Four bilingual surveys were conducted in the electoral districts of Etobicoke North, Ontario; Nipissing—Timiskaming, Ontario; Charleswood—St. James—Assiniboia—Headingley, Manitoba; and Pickering—Uxbridge, Ontario. Each survey asked respondents their likelihood of voting for the People’s Party of Canada candidate in their electoral district. Respondents were not asked their specific partisan vote intention, as the Commission believed it was important to minimize the degree to which the potential polling could be perceived as influencing an electoral outcome. Surveys were conducted with 409 respondents in Etobicoke North, 502 in Nipissing—Timiskaming, 508 in Charleswood—St. James—Assiniboia—Headingley and 504 in Pickering—Uxbridge. The surveys were conducted with residents of those four ridings who were 18 years of age or older. Each survey instrument included a sample of randomly selected respondents from within the targeted electoral districts.

Each survey was conducted using EKOS Research Associate’s interactive voice response (IVR) platform, wherein respondents enter their preferences by punching the keypad on their phone, rather than by speaking to an operator. Where possible, in an effort to reduce the coverage bias of landline only random digit dialing (RDD), a dual landline/cell phone RDD sampling frame used for this research.

Methodology

The survey was completed in the ridings of Etobicoke North, Nipissing—Timiskaming, in Charleswood—St. James—Assiniboia—Headingley and Pickering—Uxbridge. The survey instrument asked respondents in each of the four ridings “how likely are you to vote for [name of candidate], the People’s Party of Canada candidate in your riding in the next federal election?” and four demographic questions, namely their gender, age group, highest level of education, and whether or not they consider themselves to be a visible minority, Indigenous or both. Additionally, cell phone respondents and respondents with ambiguous postal codes were asked if they planned on voting in the riding in question. Each survey instrument included a sample of randomly selected respondents from each of the four ridings. The average time it took respondents to complete the questionnaire was 2.0 minutes.

In each riding, respondents to the surveys were 18 years of age and older, and were randomly selected. The survey was available to all respondents in either English or French. The survey sample relied on a dual landline/cell phone RDD sampling frame of listed and unlisted landlines in the four electoral districts, and cell phone numbers in nearby rate centres. Each survey is therefore considered to be a probability sample that is representative of the wider universe of each riding. The survey in Etobicoke North had a sample of 409 with an associated margin of error of up to plus or minus 4.9%, at a .05 confidence interval (i.e., 19 times out of 20). The survey in Nipissing—Timiskaming had a sample of 502 with a margin of error of 4.4, the survey in Charleswood—St. James—Assiniboia—Headingley had a sample of 508 with a margin of error of 4.4 and the survey in Pickering—Uxbridge had a sample of 504 and also a margin of error of 4.4. Data collection was conducted between August 26 and September 8, 2019 for the ridings of Nipissing—Timiskaming and Charleswood—St. James—Assiniboia—Headingly and between August 26 and September 9, 2019 for the ridings of Etobicoke North and Pickering—Uxbridge.

All public opinion research undertaken on behalf of the Commission conformed to the definition of non-partisan communications in the Policy on Communications and Federal Identity, the Public Opinion Research Contracting Regulations, as well as all other relevant laws and policies.

Respondents were informed of the sponsor (the Leaders’ Debates Commission) and the research supplier (EKOS Research). If desired, respondents were also provided with contact information for EKOS and the Debates Commission as well as more background about the supplier.

Once the survey data was collected, each database was reviewed for data quality. Survey results were weighted based on Statistics Canada data according to age, gender and education to ensure the sample was representative of the general public aged 18 years and older in each electoral district.

Data tables were created for each survey to isolate results for major subgroups to be used in the analysis (e.g., results for each age segment, gender, education segment, and minority status).

Response Rates and Non-Response Bias

The combined response rate for all four surveys was 2.3 per cent. The response rate is calculated by dividing the in-scope responding (2,048) by the total (functional) sample (90,714) – see table below.

	Total Numbers Attempted		111,887
	Invalid numbers		21,173
	Invalid	21,173
U	Unresolved		51,932
	No answer	21,740
	Callbacks	30,192
IS	In-scope – Not responding		36,734
	Respondent refusal	13
	Selected respondent not available	36,350
	Qualified respondent break-off	371
R	In-scope – Responding		2,048
	Quota filled	0
	Other disqualify	125
	Completed interviews	1,923
	Total (U + IS + R)	90,714
	Response rate (R divided by Total)		2.3%

Discussion of Non-Response Bias

A comparison of each unweighted sample with 2016 Census figures from Statistics Canada suggests that there are similar sources of systematic sample bias in each survey, following patterns typically found in most general public surveys. These include an under-representation of youth under 35 (five per cent versus 32 per cent in the general population in Etobicoke North; seven per cent versus 24 per cent in Nipissing—Timiskaming; five per cent versus 27 per cent in Pickering—Uxbridge; and three per cent versus 25 per cent in Charleswood—St. James—Assiniboia—Headingley), and an over-representation of older Canadians (51 per cent over 65 versus 18 per cent in the adult population in Etobicoke North; 51 per cent versus 25 per cent in Nipissing—Timiskaming; 54 per cent versus 19 per cent in Pickering—Uxbridge; and 62 per cent versus 25 per cent in Charleswood—St. James—Assiniboia—Headingley). There is a more educated sample in the survey than found in the general population (38 per cent reporting university degrees compared with 21 per cent in the population in Etobicoke North; 43 per cent versus 16 per cent in Nipissing—Timiskaming; 47 per cent versus 27 per cent in Pickering—Uxbridge; and 49 per cent versus 25 per cent in Charleswood—St. James—Assiniboia—Headingley). The sample under-represents Canadians with a high school education or less (26 per cent compared with 52 per cent in the population in Etobicoke North; 23 per cent versus 43 per cent in Nipissing—Timiskaming; 15 per cent versus 38 per cent in Pickering—Uxbridge; and 21 per cent versus 42 per cent in Charleswood—St. James—Assiniboia—Headingley). In certain ridings, visible minorities and Indigenous Canadians are also under-represented in the sample. In Etobicoke North, 25 per cent of the sample indicated that they were a visible minority compared to 76 per cent of the general population. In Pickering—Uxbridge, 15 per cent of the sample were visible minorities compared to 36 per cent of population. Meanwhile, in Nipissing—Timiskaming, eight per cent of the sample indicated they were Indigenous compared to 14 per cent of the population. Both visible minorities and Indigenous were under-represented in Charleswood—Assiniboia—Headingley, with seven per cent and four per cent respectively, compared to 10 per cent and 11 per cent of the riding’s population. As previously described, the sample was weighted by age and education.

Part of the non-response bias can be blamed on lower than desired cell phone coverage in the sample, as cell phone only households tend to be younger than the population as a whole. Best efforts were made to include as much cell phone sample as possible. However, cell phone sample cannot be geo-coded at the electoral district level (which is why cell phone respondents were asked the additional question of which riding they were voting in), making cell phone sample limited for this exercise.

Compared to the other three ridings, Etobicoke North had a much lower number of completed cases. This owes to the fact that the riding is home to a much younger, more diverse and more high school educated population than the other ridings. These are all groups that tend to respond less to public opinion surveys (including this one), and contributed to the lower response rate in this riding (in addition to lower available cell sample).

Questionnaire

INTRO

Hello, this is an automated survey from EKOS Research Associates on behalf of the Leaders’ Debates Commission and it will take a few minutes of your time.

CONTACT_EKOS

You may reach EKOS anytime by phone at 1-888-688-0709.

OFFER_INFO

Please listen to the following instructions. Press * to hear more information about EKOS, including how we can be contacted. Press 0 to skip a question. Press 9 to repeat a question.

INFO_EKOS (if star is pressed at anytime during the call)

EKOS Research is a firm with over 30 years of experience in the research field. We are based out of Ottawa. Our business is research, not sales. You can call EKOS anytime at 1-888-688-0709, visit us online at ekos.com, that is, E K O S .com, or email us at I V R @ ekos.com. You may reach the Leaders’ Debates Commission by email at info@debates-debats.ca.

QCELLCODE

Cell phone/incomplete postal code only.

Will you be voting in the federal electoral district of ____ in the upcoming federal election scheduled for October 21^st?

Yes
No (thank and terminate)
Skip (thank and terminate)

INELIGIBLE (if no/skip to Qcellcode)

Based on your responses, you are ineligible to complete the remainder of this survey. EKOS thanks you very much for your time. You may reach EKOS anytime by phone at 1-888-688-0709. Good-bye.

Q1

How likely are you to vote for_______, the People’s Party of Canada candidate in your riding in the next federal election?

Certain to vote for that candidate
Likely to vote that candidate
Possibly will vote for that candidate
Will not vote for that candidate
Skip

DEMO

We now have a few questions left which will be used to help us make better sense of the results. Please be assured that your answers will remain completely confidential and anonymous.

QGENDR

What is your gender?

Male
Female
Other
Skip

QAGE

In which age group do you belong?

Under 35 years of age
35-49
50-64
65 years of age or older
Skip

QEDUC

What is the highest level of education you have completed?

High school or less
College or CEGEP
University
Skip

QMIN

Do you consider yourself to belong to any of the following groups?

A member of a visible minority
An indigenous person
Both
None of these
Skip

THANK

EKOS Research thanks you very much for taking the time to complete this survey. You may reach EKOS anytime by phone at 1-888-688-0709. Good-bye.