Participation Criterion (ii) Approach for the Next Leaders' Debates

I. Introduction

On January 14, 2025, the Leaders' Debates Commission ("the Commission") set the participation criteria for the next federal leaders' debates.

In order to qualify under criterion (ii), a leader of a registered political party must meet the following:

The Commission has decided how it will apply criterion (ii), and this document provides the rationale for this determination. The Commission is driven by its guiding principles for debate participation criteria, and in particular that they be simple, clear, objective, and measurable.

The Commission made this decision following its request for and receipt of advice from the academics conducting the Canadian Elections Study. Convened by Professor Peter Loewen, the experts in this Polling Advisory Group are André Blais, Claire Durand, Allison Harell, Richard Johnston, Daniel Rubenson, and Laura Stephenson, in addition to Dr. Loewen. The Polling Advisory Group consists of the same group of polling experts who provided advice to the Commission for the 44th general election.

In making its determination for a political party under criterion (ii), the Commission will select polls that:

• Were completed within a 10-day window ending at 4 p.m. Pacific Daylight Time (PDT) 28 days before the date of the general election (the day of the week four weeks prior to the date of the general election) and released by 5:00 p.m. Pacific Daylight Time (PDT) 28 days before the date of the general election (the day of the week four weeks prior to the date of the general election);
• were conducted by public opinion polling organizations that are either members of the Canadian Research Insights Council or are providing public opinion data to major media organizations, using the polling organizations' most recent publicly-reported result;
• report explicit information about the support level for the party;
• were conducted on a nationally representative sample of at least 1,000 respondents; and
• publicly release methodological information sufficient for verification of the integrity of the poll, including question wording for the vote choice question(s), fielding dates, and details on sample size, weighting criteria, and survey mode.

The Commission will not consider internal party polling or polls commissioned by a political party.

In order to average the selected polls and determine a party's level of support, the polls will be averaged using a simple mean for each party included in at least two polls.

The Commission will make its decision with respect to which political parties meet the debate participation criteria no later than 27 days before Election Day.

The Commission's rationale for these determinations, as well as the advice received from the Polling Advisory Group, is provided below.

II. Criterion (ii)

In order to qualify under criterion (ii), a leader of a registered political party must meet the following:

In its January 14, 2025, decision setting the participation criteria for the next leaders’ debates and amended on February 6, 2025, the Commission stated that it would select the latest published public opinion polls based on the quality of the methodology employed, the reputation of the polling organizations, and the frequency and timeliness of the polling conducted.

The Commission also stated it may take professional advice to assist in selecting the leading national public opinion polls to be used in applying the criterion and will identify the selected polling organizations at the time the criterion is applied.

To this end, the Commission sought for the 45th general election the advice of the academics conducting the Canadian Elections Study (CES), a university research project initiated in 1965 to examine various aspects of federal elections. The Commission's point of contact was Professor Peter Loewen, who in addition to co-leading the CES study in both 2019 and 2021, is the former director of the Munk School of Global Affairs and Public Policy at the University of Toronto and the Harold Tanner Dean of the College of Arts and Sciences and a Professor of Government at Cornell University.

Dr. Loewen convened a group of academics to provide advice to the Commission on the application of criterion (ii). The academics convened in this Polling Advisory Group are Peter Loewen, André Blais, Claire Durand, Allison Harell, Richard Johnston, Daniel Rubenson, and Laura Stephenson. Each of these academics is an expert in the field.

The Commission asked the Polling Advisory Group to provide advice to the Commission in order to make a determination under criterion (ii). In particular, the Commission asked the Polling Advisory Group to provide its advice on the following:

1. The timeframe during which to consider polling results;
2. Which national public opinion polling organizations to select; and
3. How to average the polling organizations most recently publicly-reported results.

The Commission received a submission from the Polling Advisory Group, included in full in Annex A. The Commission also obtained an absence of conflict declaration from each member of the group.

The group's recommendations for each of the three items, and the Commission's resulting analysis and conclusions will be considered in turn.

In its consideration of each of these elements, the Commission is driven by its guiding principles for debate participation criteria, and in particular that they be simple, clear, objective and measurable.

Timeframe

The Polling Advisory Group recommended that a 10-day window for eligible polls be included provided that the poll results be made public by 5:00 p.m. Pacific Daylight Time (PDT) 28 days before Election Day (the day of the week four weeks prior to the date of the general election) and the polling fieldwork be completed within a 10-day window ending at 4:00 p.m. Pacific Daylight Time (PDT) 28 days before Election Day (the day of the week four weeks prior to the date of the general election). This amounts to a timeframe of 10 days in which to consider polling. The Polling Advisory Group further explained that this timeframe contributes to the generation of the largest number of possible polls and the use of the most recent polls.

The Commission has stated that it will select the latest published opinion polls based on the timeliness of the polling conducted and tie the polling window to Election Day, rather than the date the general election is called, to ensure that the polling data is as close to the debates as possible. The Commission is satisfied that the recommendation of the Polling Advisory Group achieves this objective.

The Commission concludes that a timeframe of 10 days is reasonable because it will provide the Commission with access to the latest polls possible and to as broad a range of polls as possible for the application of criterion (ii), while still ensuring that the polls used remain in close proximity to the date of determination, which is the period ending on the 28th day before (the day of the week four weeks prior to) Election Day.

Some public opinion polling organizations may release multiple polls issued during this period; in these instances, criterion (ii) states that the Commission will consider the organizations' most recent publicly-reported result.

Selection of national public opinion polling organizations

The Polling Advisory Group recommended that, in applying criterion (ii), the Commission select polls that:

The Polling Advisory Group's rationale for these criteria states that, in their view, they satisfy the “guiding values articulated by the Acting Commissioner” in that they are clear and simple while still requiring polls to pass a "credibility test."

The Commission has stated that it would select the latest published public opinion polls based on the quality of the methodology employed and the reputation of the polling organizations. The Commission is satisfied that the recommendation of the Polling Advisory Group takes this position into consideration.

In considering the recommendation made by the Polling Advisory Group, the Commission has reviewed CRIC's Public Opinion Research Standards and Disclosure Requirements that its members are required to follow. Its objectives are:

1. To support sound and ethical practices in the disclosure of research;
2. To ensure research is unbiased and supports decision-making in the public, private and not-for-profit sectors;
3. To enhance public trust and improve the public's understanding of the use of research; and
4. To ensure the appropriate transparency and disclosure of research results and methods of studies.

The Commission concludes that CRIC's requirements of its members align with the Commission's mandate and its guiding principles for setting participation criteria, and that therefore polls conducted by CRIC members within the 10-day timeframe are appropriate for inclusion in a determination of average voting intention.

The Commission recognizes that CRIC does not represent the entirety of the Canadian public opinion polling industry. However, the Commission agrees with the Polling Advisory Group that CRIC's membership includes a significant proportion of Canada's national public opinion organizations.

The Commission also agrees with the Polling Advisory Group's recommendation to supplement the pool of available polls by including those organizations who have been contracted by the major Canadian media organizations for the purposes of conducting polls to measure Canadians' voting intentions, as long as these polls satisfy methodological and disclosure requirements. This ensures that CRIC membership is not a necessary condition for inclusion in the Commission's decision-making process while maintaining a focus on the reputation and methodology of the organizations to be considered.

The Commission also concludes that it will not consider internal party polling, as recommended by the Polling Advisory Group, because internal party polling would not satisfy the above requirements that polls be publicly reported and undertaken by a CRIC member or a major news organization. The Commission also will not consider polling commissioned by a political party in order to ensure that the data available for the Commission's consideration is objective.

The Commission agrees with the Polling Advisory Group's recommendation that the measurement of a political party's level of support shall be done using polls where that party receives an explicit level of support. This means that respondents directly indicate their intention to vote for that party.

The Commission assessed this recommendation in light of two related considerations. First, not all polls will provide the option to respondents to indicate support for all political parties. Second, some polls will provide respondents with the option to select "Other" as their voting intention, after a subset of the full list of registered and eligible political parties is presented as a choice. While it may be argued that these two considerations mean that the Commission does not have a complete picture of voting intention for each political party, the Commission concludes that there are several factors that weigh against this argument. The Commission explains these factors in the following paragraphs.

The Commission is of the view that public opinion polling organizations have a professional and reputational interest in presenting to respondents as options those parties that have a meaningful presence in daily political conversation at the time. This means that it is likely that parties that would measure 4% or higher in voting intention are presented as options in their polls.

The Commission further notes that some current polls present respondents with the option to select political parties that have historically registered much less than 4% in past electoral results. Some polls also allow respondents to select newly emerging political parties and/or change the list of available responses over time. All of this evidence indicates that national public opinion polling organizations are likely to present respondents with the opportunity to indicate explicit support for parties that are likely to receive a measured voting intention of 4% or higher.

With regards to polls that have responses for "Other" parties, the Commission concludes that it is not in keeping with the guiding principles to undertake any method that would attempt to assign these voting intentions to particular political parties or to otherwise infer what respondents wished to indicate with a "Other" response. The Commission therefore concludes that adopting the approach of requiring an explicit level of support is reasonable and ensures the criteria and their application are simple, clear, objective and measurable.

The Commission asked the Polling Advisory Group to consider whether a minimum sample size for polls should be considered, and the Commission agrees with the Polling Advisory Group's recommendation to only include those polls that have a sample size of 1,000 or greater. The Commission has reviewed the recent history of polls undertaken in Canada and notes that almost all polls conducted by leading national public opinion polling organizations use a sample size of 1,000 or more, and so it is unlikely that credible polls will be excluded as a result of this requirement. Polls with a sample size less than 1,000 are likely to have a greater margin of error and may not be comparable to polls with more respondents, and as a result establishing a minimum size of 1,000 will allow for a better comparison amongst the polls selected to determine the average.

The Commission also asked the Polling Advisory Group to provide expert advice on how to assess the credibility of all potential polls that could be used by the Commission. The Polling Advisory Group recommended to the Commission that it provide this expert advice in two parts.

In order to be clear prior to the issuance of the writs on what the Commission will be looking for, the group recommended that polls be included so as long as its release "is accompanied by the following publicly accessible information: exact wording on questions about vote intention; dates of fieldwork, sample size; weighting criteria, if any; and survey mode." The group also recommended that it provide advice to the Commission after the election is called explaining "the inclusion or exclusion of an individual firms' results" by assessing each poll against the pre-identified requirements.

The Commission is satisfied that the Polling Advisory Group's recommendations on the specific components required of each poll to be included in the Commission's analysis, in terms of method, transparency, and disclosure, will help ensure a credible process that aligns with the Commission's guiding principles.

The Commission recognizes that a fully predictable process would seek to name the specific polling organizations to be used in advance of the issuance of the writs but has concluded that in a minority context with an unknown election call date; such an approach may limit the polls available for use by the Commission. The Commission concludes that that the two-step process recommended by the Polling Advisory Group, which first identifies ahead of time the requirements for a poll to be included and then after the issuance of the writs provide specific advice assessing each potential poll against these requirements meets the Commission's guiding principles.

The Polling Advisory Group's submission recommends that polls be included subject to a number of factors discussed above which assess that poll based on organizational reputation, methodological soundness, and transparency and disclosure standards. The Commission agrees with the rationale of the group and concludes that the Commission's selection of national public opinion polling organizations will be based on these factors. No polling organizations will be excluded on the grounds that, in the view of one or more political party, they have previously demonstrated a bias towards a political party.

Averaging the selected polls

The Polling Advisory Group has recommended that, for each political party included in at least two polls, the mean value of that party's level of support as measured by voting intention will be determined using a simple mean across the polls in which it receives an explicit level of support.

The Commission has stated it will use an average of polls in order to make its determination of whether a party receives a level of national support of at least 4%, determined by voting intention. The Commission has considered the Polling Advisory Group's recommendations for how to determine the average of the selected polls. In so doing, the Commission is aware that there are differences amongst the methodologies and sample sizes of the above polls; that each poll involves statistical uncertainty or a 'margin of error'; and that each was conducted at slightly different periods.

The Commission is of the view that taking a simple mean is reasonable given that polls will only be included if they have a representative sample of at least 1,000 respondents and were all conducted in close proximity to the date of determination.

The Commission also concludes that it is reasonable to require a political party to appear in at least two polls that report explicit information about the support level for that party. As stated previously, the Commission is of the view that public opinion polling organizations are likely to present as options to respondents those parties that could measure 4% or higher in voting intention. Criterion (ii) also states that a party's level of national support shall be measured by polling organizations, meaning more than one organization.

With regards to statistical uncertainty and margin of error, the Commission concludes that it has attempted to mitigate the impact of this unavoidable uncertainty by incorporating multiple polls within its analysis. Additionally, the Commission is aware that polls typically report the maximum margin of error, which is the margin of error when the reported value is 50%. For values less than 10%, as is the case for applying criterion (ii), the margin of error is likely to be smaller.^{Footnote 1}

The Commission therefore concludes that each political party's mean value across the polls in which it receives an explicit level of support, as measured by voting intention, will be used to determine whether it receives a level of support of at least 4%.

III. Conclusion

In making its determination for a political party under criterion (ii), the Commission will select polls that:

• Were completed within a 10-day window ending at 4 p.m. Pacific Daylight Time (PDT) 28 days before the date of the general election (the day of the week four weeks prior to the date of the general election) and released by 5:00 p.m. Pacific Daylight Time (PDT) 28 days before the date of the general election (the day of the week four weeks prior to the date of the general election);
• Were conducted by public opinion polling organizations that are either members of the Canadian Research Insights Council (CRIC) or are providing public opinion data to major media organizations, using the organizations' most recent publicly-reported result;
• report explicit information about the support level for the party;
• were conducted on a nationally representative sample of at least 1,000 respondents; and
• publicly release methodological information sufficient for verification of the integrity of the poll, including question wording for the vote choice question(s), fielding dates, and details on sample size, weighting criteria, and survey mode.

The Commission will not consider internal party polling or polls commissioned by a political party.

In order to average the selected polls and determine a party's level of support, the polls will be averaged using a simple mean for each party included in at least two polls.

IV. Next Steps

The Commission will make its decision with respect to which political parties meet the debate participation criteria no later than 27 days before Election Day. The decision date is tied to Election Day to ensure that the length of the next general election campaign, whether long or short, is taken into account. The Commission will then issue invitations to leaders of those political parties that meet debate participation criteria. A party not invited will have an opportunity to ask the Commission to reconsider within two days of the invitation to leaders. The Commission will make its final decision with respect to any such party within three days of that party's application for review. The tight timelines are being set to ensure that the debates producer has sufficient time to produce a debate of high quality, as required by the OIC, and that the political parties can properly prepare for the debates in order to ensure they are informative for Canadians.

Annex A – Expert Group process for determining party inclusion in the Leaders' Debates (received on February 18, 2025)

Prepared by Peter John Loewen, André Blais, Claire Durand, Allison Harell, Richard Johnston, Daniel Rubenson, and Laura Stephenson (the Expert Group)

Summary

The purpose of this report is to review the second criterion (ii) for inclusion set out by the Commission, as well as the principles for determining included parties. We propose a decision rule for inclusion, justify it according to the principles articulated by the Commission, and then lay out the exact method for determining inclusion given this decision rule.

Criteria for Inclusion

On January 14, 2025, the Executive Director and Acting Commissioner, Michel Cormier, released criteria for inclusion in the next leaders' debate. In order to be invited by the Commission to participate in the leaders' debates, a leader of a registered political party must meet two of the following three criteria:

(i): on the date the general election is called, the party is represented in the House of Commons by a Member of Parliament who was elected as a member of that party.

(ii): 28 days before the date of the general election, the party receives a level of national support of at least 4%, determined by voting intentions, and as measured by leading national public opinion polling organizations, using the average of those organizations' most recently publicly-reported results.

(iii): 28 days before the date of the general election, the party has endorsed candidates in at least 90% of federal ridings.

Our purpose is to specify a method to determine which parties satisfy the second (ii) criterion.

In addition to the criteria specified, the Acting Commissioner further states that “In administering criterion (ii), the Commission will select the latest published public opinion polls based on the quality of the methodology employed, the reputation of the polling organizations, and the frequency and timeliness of the polling conducted. The Commission may take professional advice to assist in selecting the leading national public opinion polls to be used in applying the criterion and will identify the selected polling organizations at the time the criterion is applied.”

Decision Rule

The Commission has set a clear standard for inclusion in the Debates: an average level of national support of 4% of decided voters, as measured by leading national public opinion polling organizations, using the average of those organizations' most recently publicly-reported results within the inclusion period ending on the 28th day before (the day of the week four weeks prior to) the date of the general election.

The Acting Commissioner notes that “tying the polling window to Election Day, rather than the date the general election is called, ensures that the polling data is as close to the debate as possible.”

The decision rule(s) we recommend for the averaging of polls for each party are the following:

1. Include polls from all polling organizations who are members of the Canadian Research Insights Council (CRIC) and/or from polling organizations which are providing public opinion data to major media organizations.
2. For any polling firm or organization, the most recent poll satisfying the following criteria will be included:
1. The polls report explicit information about the support level for the various parties;
2. The polling fieldwork must be completed within a 10-day window ending at 4:00 p.m. Pacific Time (PDT) the 28th day before the date of the general election;
3. The results of those polls were publicly available by 5:00 p.m. PDT on the date 28 days prior to the election;
4. The polls were conducted on a nationally representative sample of at least 1000 respondents (unweighted);
5. For each poll, its release is accompanied by the following publicly accessible information: exact wording of questions about vote intention; dates of fieldwork; sample size; weighting criteria; and survey mode(s);
3. Internal party polls will not be included.
4. The polls will be averaged using a simple mean for each relevant party included in at least two polls.

The expert group will report which parties have an average support level equal to or exceeding 4% (unrounded) of decided voters and which parties fall below this threshold.

Justification

The decision rules above satisfy the specifics of the second criterion of inclusion, as well as the guiding values articulated by the Acting Commissioner. First, we believe that condition (1) of membership in CRIC - the leading accreditation body of Canadian public opinion research firms - or partnership with a major media organization satisfies the Commission's standard of leading national organizations. Moreover, it is a clear and simple rule, satisfying the guiding principles. Second, rule 2 (sections a through e) is articulated to generate the largest number of possible polls, conditional upon their passing a credibility test (rule 3). Finally, the use of simple weighting (rule 4) satisfies the guiding principles of clarity and simplicity.

Process

We articulate a process to implement the above decision rules.

1. Beginning as soon as possible, the polling partners of major media organizations for the purposes of the election should be identified. Combining these firms with those who are in CRIC will generate the complete set of polling organizations to be considered by the Expert Group.
2. A definitive list of these organizations that identifies where their polls and accompanying information are posted will be created.
3. The Expert Group will develop a spreadsheet for calculating average party support.
4. The Expert Group will determine the inclusion period for eligible polls (the 10 days before the 28th day prior to the general election). After the start of the inclusion period, the Expert Group will review polls released by all listed polling organizations on a daily basis.
5. Accepted results from the most recent poll conducted by a firm will be entered into the spreadsheet, with polling results updated up to and including the polls released by 5:00 p.m. PDT on the date 28 days prior to the election.
6. The Expert Group will compile all background information for each included poll in a single document.
7. The spreadsheet and identification of which parties equal or exceed 4% and a memo explaining the results and any inclusion or exclusion of an individual firms' results will be presented to the Acting Commissioner 27 days before the general election.
8. The final decision will rest with the Acting Commissioner.

Expert Group

Peter Loewen is the Harold Tanner Dean of the College of Arts & Sciences and a Professor of Government at Cornell University, and the former director of the Munk School of Global Affairs and Public Policy at the University of Toronto. He is a former co-investigator of the Canadian Election Study.

André Blais is a Professor Emeritus at the Université de Montréal. He is a former co-investigator of the Canadian Election Study.

Claire Durand is a Professor at l'Université de Montréal. She is the former President of the World Association for Public Opinion Research.

Allison Harell is a Professor at the Université du Québec à Montréal. She is a co-investigator of the Canadian Election Study.

Richard Johnston is a Professor Emeritus at the University of British Columbia. He is a former co-investigator of the Canadian Election Study.

Daniel Rubenson is a Professor at the University of Toronto. He is a co-investigator of the Canadian Election Study and the Executive Director of Evidence in Governance and Politics (EGAP).

Laura Stephenson is a Professor & Chair of the Department of Political Science at the University of Western University. She is a co-investigator of the Canadian Election Study.