Evaluation of the client services and lobbyists registration system (final report)

November 28, 2018

Table of contents

• List of abbreviations
• Executive summary
• 1.0 Introduction
  • 1.1 Scope and objectives
  • 1.2 Structure of the report
• 2.0 Overview
  • 2.1 The Office of the Commissioner of Lobbying of Canada
  • 2.2 Organizational structure
  • 2.3 The role of the RCSD
• 3.0 Methodology
  • 3.1 Document, data, and literature review
  • 3.2 Key informant interviews
  • 3.3 Online survey of registrants and representatives
  • 3.4 Limitations
• 4.0 Findings
  • 4.1 Use of client services advice and support
  • 4.2 Outreach services
  • 4.3 Online tools and resources
  • 4.4 Accessibility of the LRS
  • 4.5 Measures of success
  • 4.6 Efficiency of services
• 5.0 Conclusions and recommendations
• Appendix A – References
• Appendix B – Survey methodology

List of abbreviations

CAT

Compliance Advisory Team

CIO

Chief Information Officer

DPOH

Designated Public Officer Holders

EPAS

Enhanced per Agent Service

FAQ

Frequently Asked Questions

GEDS

Government Electronic Directory Services

LRS

Lobbyists Registration System

MCR

Monthly communication reports

OCL

Office of the Commissioner of Lobbying of Canada

POH

Public office holders

RCSD

Registration and Client Services Directorate

RUA

Registrant User Agreement

Executive summary

The Evaluation of Client Services and Lobbyists Registration System conducted for the Office of the Commissioner of Lobbying of Canada (OCL) focusses on the activities of the Registration and Client Services Directorate (RCSD) related to:

1. providing advice, support services, and resources to lobbyists to help them ensure that they are complying with the Lobbying Act and the Lobbyists Code of Conduct, and helping them to navigate the Lobbyists Registration System (LRS);
2. developing and maintaining the LRS; and
3. offering outreach services to lobbyists and potential lobbyists.

The evaluation's methodology included three main lines of evidence: a document, data, and literature review; key informant interviews; and an online survey of registrants and representatives.

An overview of the findings and recommendations is presented below.

The evaluation evidence supports the conclusion that client services and the LRS are accessible.

Assistance from Registration Advisors is experiencing increasing use. The evaluation found that the assistance provided to registrants and representatives by the RCSD receives substantial use, which demonstrates the need to offer email and telephone assistance. Over the two years covered by the evaluation (2016-17 and 2017-18), the client services provided by the RCSD have seen an increase of 27% in the number of total contacts (email and telephone) and a 38% increase when considering only contacts from registered lobbyists and representatives. Both methods of reaching the RCSD to request assistance—telephone and email—are used by clients, but based on administrative data, telephone is most used and interview results indicate that this is because registrants and representatives believe that they get a faster response by telephone.

Outreach services are offered to registrants and representatives. The RCSD offers webinar or in-person outreach sessions to new registrants and representatives as part of the orientation email sent by their Registration Advisor. Over the time period covered by the evaluation, the RCSD provided 87 outreach sessions to registrants and representatives.

The LRS is complex, but is considered accessible. Those registrant and representative interviewees who are more frequent users of the LRS reported that the system is accessible, but acknowledged that this ease of use comes from experience.

In general, most of the registrants and representatives surveyed reported that the registration process was clear or very clear (from 69% to 75%, depending on the dimension of the process enquired about) and were satisfied with the ease of completing the steps in the registration process (62%). While a minority view, 14% of registrants and representatives were unsatisfied with the ease of completing all of the steps in the registration process and 11% considered the Registrant User Agreement (RUA) process to be unclear. The RCSD is moving to a process where an RUA will not have to be separately submitted, which should improve this feature of the LRS. Aspects of registration that are less used, such as procedures related to change of registrant, recertification, and reactivation of an account, were considered to be less clear based on survey responses. This finding could point to areas where the RCSD might want to target education efforts or tools.

Based on interview and survey results, the LRS procedures related to monthly communication reports (MCRs) are generally clear and reasonable, although two thirds (66%) of survey respondents considered the requirements regarding what information should be reported to be clear.

The evaluation found general satisfaction with the tools available to lobbyists.

A wide variety of tools are available on the OCL website, as well as through links in the LRS. These tools cover a range of topics and include many formats (e.g., text, video, interactive, diagrams). Most registrants and representatives surveyed (86%) had used at least one of the available tools and three quarters (75%) responded that they were satisfied with the available tools. When registrants and representatives surveyed were asked what tools need improvement, no tool was identified by more than 25% of respondents. Based on use, as well as satisfaction with the individual tools, registration help tools, with the exception of the worksheets, tend to do better than the interpretation and advice tools.

The RCSD is providing effective services based on all of the available measures of success.

The service standards currently tracked are being met. Four service standards are met or largely met. Those that are met include answering 80% of telephone calls received during business hours within 30 seconds; responding to less complicated emails within two business days; and approving registrations or sending them back for corrections within three business days. The service standard that is largely met is responding to more complicated emails within 14 calendar days.

The evaluation found a high level of satisfaction with the assistance from Registration Advisors. Overall, most (86%) survey respondents indicated that they were highly satisfied or satisfied with the support they received from Registration Advisors. This level of satisfaction existed across various dimensions of service (respectfulness, responsiveness, ability to answer questions and/or provide advice needed; and the clarity of the advice or instructions given), with only a small difference based on whether the service was provided by email or telephone. Telephone assistance scored only slightly higher in terms of the level of satisfaction. The assignment of Registration Advisors was important to clients, and internal key informants also noted the benefits of assigning Advisors, including building rapport and ensuring continuity of service.

Clients' level of knowledge of their obligations under the Act and the Code is an indirect indication of success. The majority of registrants or representatives surveyed and interviewed believe that they have a good understanding of their obligations. Interviewees attributed this, in part, to the assistance they received from Registration Advisors and from the use of the available tools and resources.

Clients' level of compliance also provides an indirect measure of success. The level of compliance cannot be directly attributed to the effectiveness of client services, but is an indirect measure. Based on the available statistics, registrants' and representatives' compliance for meeting MCR reporting deadlines is high, as is the accuracy of the reports. The one compliance issue identified was the proportion of newly created accounts that become pending and then are deactivated. Strictly speaking, this may not be a compliance issue as much as registering either prematurely or when not necessary.

The RCSD services are efficient and cost-effective.

Cost per unit service remained steady over the two years. Even with the expanding demand for its services, the cost of RCSD staff per unit of service (enquiries and registration activities) has remained essentially the same, which is an indication of efficiency in service delivery.

The RCSD has taken steps to continuously improve. The evaluation found that staff felt they had the tools, training, and resources to support them in doing their work. The RCSD has developed templates, "cheat sheets" to simplify processes, and a wiki to assist Registration Advisors. The LRS has continued to be improved with automation of some activities that used to require staff time. A new call routing system is credited with improving approval time for registrations.

Areas for improvement

Outreach services. Comparing the total number of new registrations over the last two fiscal years (4,258) to the number of outreach sessions offered (87) during approximately the same time period indicates a potential underutilization of RCSD outreach. In addition, the outreach sessions were found to be too long and cover too many topics. The evaluation also found that the RCSD has ended its process for systematically gathering participant feedback.

The LRS. Detailed suggestions were made to reduce the complexity of the system. One suggestion made by registrants, representatives, and internal key informants was to have an overview page in order to reduce the navigation issues experienced by users, and/or enable Registration Advisors to share their computer screens with users so they can see the correct location in the LRS. More assistance with inputting information was also desired, such as pop-up texts and logic checks, in order to decrease errors. Finally, while there is some interest in the LRS becoming mobile-friendly, just over one quarter (26%) of survey respondents said this was very important to them.

Recommendations: The RCSD should explore ways to improve the navigation of the LRS and reduce user errors, such as through the use of an overview page and more pop-ups and logic checks.

Tools. The evaluation findings indicate that there is room for better promotion of tools and ensuring their visibility/accessibility. Based on survey results, only one tool (Guidance on the Lobbyists' Code of Conduct) was used by a majority (55%) of respondents. A tool often referred to as helpful by registrants, representatives, and internal OCL stakeholders interviewed was the Frequently Asked Questions (FAQ) page, which was also considered "buried" on the website. Also noteworthy is that many of the common questions received by Registration Advisors or mentioned by registrants or representatives (e.g., identifying Designated Public Office Holders [DPOHs], interpreting the 20% rule) have tools that address them, but either clients are not locating the tools they need or the content is not sufficient to meet their needs. Improving the website should help address some of these issues, and the OCL could use its social media to promote tools and push information to its stakeholders. Other registries use social media to do so.

Other suggestions were to simplify tools—in terms of language used, number of tools, and amount of information—and to offer more visual tools/graphics. Best practices from other registries were a "quick tips" document, flow charts, and video tutorials. Many of these practices are in use or under way by the RCSD, but based on comments related to being overwhelmed by the quantity of tools, it will be important not to continue to add tools without considering the overall toolkit. One method of ensuring that the tools are relevant to stakeholders is to devise tools based on types of common enquiries or compliance issues. It was pointed out that the RCSD does not currently track the type/topic of questions with sufficient detail to use that information to inform tool development. One item that was raised repeatedly in interviews with registrants and representatives, as well as by some survey respondents, was the difficulty encountered in identifying DPOHs.

Service standards. Service standards should be transparent, which means they should be monitored and reported to stakeholders. Not all service standards are reported upon in the OCL annual reports or reported on internally. In addition, the service standards focus on timeliness and do not include other dimensions of service, such as accuracy and access. Service standards should also include operational targets to ensure internal accountability (e.g., the percentage of the activity that should meet the standard). Some service standards would benefit from more clarity (e.g., the definition of a complex enquiry).

Collaboration within the OCL. Internal key informants believe that collaboration within the OCL has improved. Specific suggestions for further improvements included having the policy unit work with the RCSD on drafting policies in order to include a frontline perspective; and (most often mentioned) enabling the sharing of information between the Investigations Directorate and the RCSD. As to the latter suggestion, there are other registries that engage in sharing information between both areas and one that enables investigators to access client services information, but not vice versa. A key consideration in fully sharing information is providing the proper training for Registration Advisors and ensuring that there is a client service rationale for Registration Advisors having access to information on investigations.

1.0 Introduction

This report presents the findings of the Evaluation of Client Services and Lobbyists Registration System conducted for the Office of the Commissioner of Lobbying of Canada (OCL).

1.1 Scope and objectives

The evaluation focuses on the activities of the Registration and Client Services Directorate (RCSD), including its work in developing and maintaining the online Lobbyists Registration System (LRS).^{Footnote 1} The evaluation covers the last two fiscal years — 2016/17 and 2017/18 — and focuses on the services that the RCSD provides to lobbyists. Therefore, public office holders, parliamentarians, the media, other jurisdictions, and the public will not be included as "clients" for the purpose of the evaluation.

The evaluation's objectives are to determine the extent to which:

• current client services and the LRS are accessible and effective in supporting lobbyists' awareness/understanding of their obligations and facilitating their ability to meet those obligations;^{Footnote 2}
• the tools and other resources available to lobbyists are adequate, or there are gaps that should be addressed;
• the RCSD has the policies, procedures, supports, and other resources needed to provide the expected level of client services; and
• the information-sharing and collaboration within the OCL supports the delivery of client services.^{Footnote 3}

1.2 Structure of the report

This report contains five sections, including this introduction. Section 2.0 provides a description of the OCL and the RCSD, more specifically. Section 3.0 describes the methodology used to address the set of evaluation issues and questions. Section 4.0 summarizes the key findings that have emerged from the data collection process, and Section 5.0 provides the overall evaluation conclusions.

2.0 Overview

2.1 The Office of the Commissioner of Lobbying of Canada

In July 2008, the Lobbying Act (the Act) and its related regulations came into force with the aim to "ensure transparency and accountability in the lobbying of public office holders (POHs) in order to contribute to confidence in the integrity of government decision-making" (OCL, 2015b). The Act established the Office of the Commissioner of Lobbying of Canada (OCL) to support the activities of the Commissioner of Lobbying (the Commissioner). Appointed by Parliament as an independent Agent of Parliament for a seven year term, the Commissioner is responsible for administering the Act and the Lobbyists' Code of Conduct (the Code) (OCL, 2015b). The Code outlines the high ethical standards that are expected of lobbyists when they conduct their activities (OCL, 2015c). The Commissioner's mandate includes three main responsibilities: maintaining a public registry of lobbyists; using education and outreach initiatives to raise awareness about the Act and its requirements and the Code; and ensuring compliance with the Act and the Code (OCL, 2012).

2.2 Organizational structure

The OCL's organizational structure has undergone recent changes, although the directorates discussed in this evaluation report, the RCSD and the Investigations Directorate, remain. What is referred to in this report as the "policy group" is now part of the newly created Policy, Public Affairs and Outreach Directorate. As of May 1, 2018, the OCL is divided into the following units:

• the Office of the Commissioner, which directly supports the Commissioner by providing legal and strategic advice, as well as administrative support;
• the Finance Directorate/Office of the Chief Financial Officer (CFO);
• the RCSD (and Corporate Services) Directorate;
• the Investigations Directorate, which supports the Commissioner in fulfilling the mandate to ensure compliance with the Lobbying Act and Lobbyists' Code of Conduct by monitoring lobbying activities, verifying the accuracy of monthly communication reports (MCRs), investigating allegations of non-compliance, and reviewing applications for exemptions to the five-year prohibition on lobbying for former Designated Public Officer Holders (DPOHs); and
• the Policy, Public Affairs and Outreach Directorate (job descriptions and classifications for this Directorate are currently being reviewed by the OCL).

For purposes of this evaluation, the activities of the RCSD are the main focus. The RCSD is headed by the Director of Registration and Client Services, who reports directly to the OCL Commissioner (as do the Director of Investigations and the Deputy Commissioner and Chief Financial Officer). At the time of the evaluation, other key roles within the RCSD included a Business and Systems Analyst, which is a role that was not captured in this evaluation.

Registration Advisors are responsible for a variety of tasks including:

• responding to requests for information on, and providing advice and interpretation related to, the Act, the Lobbyists Registration Regulations, and the registration process and system;
• reviewing lobbyist registrations and liaising with registrants to resolve registration issues;
• analyzing updates to existing registrations and rejecting registrations that are not compliant with the Act;
• conducting quality testing of, and identifying issues related to, the LRS; and
• conducting outreach sessions/presentations to lobbyists and registrants on the Act registration requirements, and the LRS.

There are currently four Registration Advisors in the RCSD.

Manager, Registration Service oversees the Registration Advisors and is responsible for a variety of tasks, including:

• coordinating day-to-day operations of the OCL Registration Service;
• developing and implementing operating procedures; and
• ensuring that the registrations are completed pursuant to the requirements set out in the Lobbying Act and regulations.

The Manager, Registration Service reports to the RCSD Director of Registration and Client Services.

As of May 2018, the RCSD staffing complement also includes:

• One Chief Information Officer (CIO)
• One Information Management/Information Technology Support Technician
• One Administrative Services and Access to Information and Privacy Officer
• Two Consultants (who are not full-time equivalents) for the LRS

2.3 The role of the RCSD

As noted in Section 1.1, the focus of this evaluation is on the RCSD's activities related to

1. providing advice, support services, and resources to lobbyists to help them ensure they comply with the Act and the Code, and help them to navigate the LRS;
2. developing and maintaining the LRS; and
3. offering outreach services to lobbyists and potential lobbyists.

Under the Act, there are two types of lobbyists:

Consultant lobbyists are individuals who are paid to communicate directly with a federal POH on behalf of a client (i.e., another individual, a company, or an organization) or who arranges a meeting between a POH and another person. Consultant lobbyists are required to register each of their lobbying undertakings no later than 10 days after entering into an undertaking.

In-house lobbyists (corporations or organizations) are employees of a corporation or an organization who communicate with federal POHs on behalf of their employer. The most senior paid employee is responsible for filing a registration for a corporation or organization. In-house lobbyists are required to register within two months of when lobbying activities constitute a significant part of the duties of one full-time employee or would constitute a significant part of the duties of one employee if they were performed by only one employee (the 20% rule).^{Footnote 4}

Providing advice and support services: Registration Advisors within the RCSD provide telephone and email support to lobbyists. They assist in a number of ways depending on the needs of the individual. The assistance can include answering questions related to such topics as:

• whether or not to register
• obligations under the Act
• how to create an account
• help with the submission of a completed Registrant User Agreement (RUA)
• guidance through the registration process
• guidance with MCRs
• various deadlines for submissions
• how to navigate the website
• technical assistance related to their accounts and the LRS (with referrals to IT if the Registration Advisor cannot resolve the issue, although the Registration Advisor remains as the intermediary between the IT team and the registrant or representative)

Developing and maintaining the LRS: The RCSD develops and maintains the LRS, which is the application through which lobbyists report on their lobbying activities. The LRS, with its searchable registry, is the main mechanism used to ensure that lobbying activities at the federal level are transparent and compliant.

Lobbyists must disclose information on "oral and arranged communications" with DPOHs.^{Footnote 5} As cited closely from (OCL, 2017a, pp. 7–8), this includes information on:

• Who is lobbying;
  • the names of lobbyists and whether they previously held federal public offices;
• Who benefits from the lobbying;
  • the clients, organizations, or corporations, including the parent and subsidiary corporations, that benefit from the lobbying activities;
  • organizations that are members of coalition groups represented;
  • the government funding received by those represented;
• The names of the federal institutions lobbied;
• The subject matters of lobbying activities; and
• The communication techniques used.

Once registered, lobbyists are also required to submit monthly reports detailing oral and arranged communication they have had with DPOHs (OCL, 2017a, p. 12).

Outreach services: The RCSD offers webinar or in-person outreach sessions to those who use the LRS (registrants and representatives, defined in Section 3.3) and are new registrants or representatives, or are current registrants or representatives who desire a refresher. These services are described in more detail in Section 4.2.

3.0 Methodology

The methodology included three main lines of evidence: a document, data, and literature review; key informant interviews; and an online survey of registrants and representatives.

3.1 Document, data, and literature review

Internal program documents were reviewed (e.g., internal procedures and manuals, templates, web analytics) as well as publicly available information on the OCL website. PRA also conducted a targeted review of grey literature on other jurisdictions' lobbyist registry systems for potential best practices or lessons learned (e.g., procedures, standards, methods of providing client services). The targeted literature review focused on the lobbyist registration systems for Ontario, Québec, and British Columbia, and the City of Toronto.

3.2 Key informant interviews

The evaluation included 26 semi-structured interviews with key informants from the following groups of stakeholders.

• RCSD supervisors (Director and Manager) and Registration Advisors for a total of six interviews
• Other stakeholders within the OCL that are knowledgeable about RCSD activities and could speak to issues covered by the evaluation for a total of four interviews
• Representatives of lobbyist registry systems for Ontario, Québec, and British Columbia, and the City of Toronto (five interviews)
• Lobbyists and representatives^{Footnote 6} (11)

The following scale has been used to report on interview findings:

3.3 Online survey of registrants and representatives

The evaluation included an anonymous and confidential web-based survey with stakeholders who access the LRS. The stakeholders surveyed had any one of three types of active accounts in the LRS:

• Registrant: Individuals who must file a return are referred to as "registrants." Registrant Accounts are used by individuals who must register personally as consultant lobbyists under Section 5 of the Act, and/or by persons having to register In-house Lobbyists (Corporation) or In-house Lobbyists (Organization), as per Section 7 of the Act.
• Representative This type of account is used by individuals (representatives) who have been authorized by a registrant to manage his or her account on his or her behalf. However, only registrants have the right to certify registrations or MCRs with their account passwords.
• Registrant/representative This account is used by persons who 1) are registrants and 2) who also act as a representative by managing the accounts of other registrants.

The survey was online for a total of three weeks — from June 5 to June 27, 2018. To encourage responses, two reminder emails were sent to those who had not yet completed the survey. In addition, the survey was extended from the original June 22 end-date until June 27.

Invitations were sent to 2,451 active emails. In total, 402 respondents completed the survey for a response rate of 16%. As shown in Table 1, most respondents were consultant lobbyists, followed by in-house organization representatives.

Once the survey was finished, open-ended questions were coded and the survey data were analyzed using SPSS, a statistical software package. More details on the methodology for the survey are in Appendix B.

3.4 Limitations

The evaluation faced a few methodological limitations, which are listed below by line of evidence.

Interviews. The interviews with key informants have the possibility of self-reported response bias and strategic response bias. Self-reported response bias occurs when individuals are reporting on their own activities and so may want to portray themselves in the best light. Strategic response bias occurs when the participants answer questions with the desire to affect outcomes. Interviews within the OCL included all of the managers, supervisors, and Registration Advisors of the RCSD. The other areas of the OCL were represented in interviews, but not as extensively (e.g., one representative). As a result, other areas of the OCL were not consulted to the same extent as the RCSD, which is particularly relevant for findings on issues such as collaboration across the OCL.

Survey of registrants and representatives. The survey used a census approach (see Appendix B) rather than a random sample and the responses cannot be considered generalizable to the population of registrants and representatives. As with the key informant interviews, the survey also has the possibility of self-reported response bias and strategic response bias.

Mitigation strategy. The mitigation strategy for the methodological limitations above was to use multiple lines of evidence. In addition, the evaluation used both quantitative and qualitative data collection methods to answer evaluation questions. By triangulating the findings from these different sources, the evaluation was able to strengthen its conclusions despite the limitations.

4.0 Findings

4.1 Use of client services advice and support

The evaluation found that the assistance provided to registrants and representatives by the RCSD receives substantial use, which demonstrates the need to offer email and telephone assistance. Over the two years covered by the evaluation (2016/17 and 2017/18), the client services provided by the RCSD have increased 27% in the total number of contacts (email and telephone) from 4,889 to 6,189. When considering only contacts from registered lobbyists, representatives, and Board members (consultant lobbyists), the increase is 38%, from 3,662 to 5,036 contacts. There are several possible reasons for the increase in the number of contacts. After remaining largely untouched since the late 1980s, a new Lobbyists' Code of Conduct came into force in December 2015, which likely resulted in more questions for OCL. In addition, the 2015 federal election and the change in government also likely had an effect. The increase in enquiries is also partially accounted for by a 5% increase in the number of active lobbyists between 2016/17 and 2017/18 (8,653 to 9,084) and a 7% increase in registration activities (33,045 to 35,077).^{Footnote 7} Figure 1 shows the increase in contacts with the RCSD since 2015/16.

Figure 1 - Text version

Contacts (telephone and email) with RCSD over time

	2014-15	2015-16	2016-17	2017-18
Contacts from registrants and representatives	2,605	2,688	3,662	5,036
Total contacts	3,464	3,816	4,889	6,189

Based on survey results, registrants and representatives have regular but not frequent contact with Registration Advisors. Most active registrants or representatives (87%) reported having had contact with a Registration Advisor at some point in time. Of those reporting contact in the last year by email or telephone, almost all (98%) had contact either a few times during the year or only once. Only 2% reported more frequent contact. Similarly, most of the registrant and representative key informants had used their Registration Advisor for assistance in the last year, with a few reporting that they did not have any questions or issues arise for which they needed assistance.

Available administrative data indicate that most contacts are by telephone (72% in 2016/17),^{Footnote 8} although survey respondents reported using the two modes of communication with Registration Advisors about equally.^{Footnote 9} Most registrants or representatives interviewed tended to communicate with their Registration Advisor by telephone as they believe that they get a faster response than when they use email.

According to administrative data, interviews, and survey responses, the most common types of assistance/enquiry involved enquiries for registration support. Based on administrative data, about half as many contacts involved enquiries about advice on the Act and/or Code. A point raised by many internal key informants during their interviews was that tracking of the subject matter of enquiries with greater detail would enable the RCSD to better identify areas where more education and assistance are needed (see also Section 4.3.3).

4.2 Outreach services

RCSD offers webinar or in-person outreach sessions to new registrants and representatives or those who desire a refresher. New registrants and representatives receive an offer of an outreach session as part of the orientation email sent by their assigned Registration Advisor. The outreach offered by the RCSD has a different purpose than the outreach offered by the Outreach and Education Program, which focuses on federal POHs and lobbyists group associations. The RCSD outreach focuses on registrants and representatives and presents information about the Act and the Code as well as information on creating an account and registering in the LRS. Given these differences in the types of outreach offered, internal key informants reported that the responsibilities related to outreach between the RCSD and the Outreach and Education Program were clear and there was no duplication of efforts.

Between February 1, 2016 and March 31, 2018, the RCSD conducted 87 outreach sessions. The total number of participants is not available as internal key informants noted that for webinars this information is not typically gathered. However, given that there were approximately, 4,258 new registrations during the same time period (FY 2016/17 and 2017/18), it appears that the outreach sessions may be underutilized. That being said, some internal key informants pointed out that each outreach session requires approximately two hours of staff time, so capacity to provide substantially more sessions may be an issue. Each outreach session involves a lead/presenter who liaises with the client, prepares the presentation materials, and presents and responds to questions; and an observer who prepares a summary of the questions raised during the session.

The RCSD takes steps to ensure that all new registrants and representatives, including those who have not requested webinar or in-person outreach sessions, have access to the information. The RCSD has a PowerPoint presentation posted online that outlines the requirements of the Act and regulations, provides an overview of the role of the RCSD, and provides information about lobbying and the registration process. This presentation has been customized for different lobbyist types (corporations, organizations, and consultants) and is available in both official languages. Analytics show that the online versions of these presentations are being accessed. From February 10, 2016 to March 7, 2018, the presentations for corporations, organizations, and consultants had 1,994, 1,711, and 2,973 unique page views, respectively. These totals include both English and French versions of the presentations. Web analytics show that the average time spent on each presentation's webpage was a little over two minutes, which is a short period of time given that each presentation is about 31 slides long. Viewing the material online is clearly not comparable to the amount of learning that would be expected to occur through a 45-minute webinar or in-person outreach session.

Based in interviews with internal key informants and representatives of other registries, a number of potential improvements for the RCSD outreach sessions were identified:

• Streamline the outreach sessions — Internal key informants noted that they are too long and unfocused (e.g., they cover a number of topics, such as the Act and regulations, the role of the RCSD, the registration process, what information should be reported, MCRs and due dates, updating registrations). For new registrants and representatives, it was suggested that the sessions should have a greater focus on the "nuts and bolts" of the registration process.
• Offer different types of outreach sessions from which registrants and representatives can choose — Some sessions could focus on registration, while other sessions could address common questions related to obligations under the Act and the Code as well as common reporting issues.
• Provide more "open" sessions by video or webinar — Currently, the RCSD provides in-person or webinar sessions to individual consultants, firms, or organizations. Internal key informants were concerned that offering webinars to groups of registrants and representatives that are not affiliated with the same organization or firm would hinder asking questions that are specific to a lobbyist's or firm's practice. However, other registries have found that these more general sessions work well and are more efficient than in-person sessions or webinars with more limited participation (Ontario Lobbyists Registry). The RCSD could consider the use of more "open" sessions for general topics, such as the "nuts and bolts" of registering.
• Refresh the process for evaluating RCSD outreach activities — The RCSD no longer has an online survey for its outreach sessions and is not tabulating the results of paper-based surveys for the in-person sessions. To better understand what is working with the outreach sessions, what can be improved, and what other topics would be helpful to be included in outreach sessions, gathering feedback and analyzing it is essential.
• Provide more clarity on the expectations for outreach sessions — Some internal key informants said they would benefit from having a clearer idea on what is expected when they are conducting outreach sessions.

Internal key informants noted that improvements to the outreach sessions are under way, including streamlining the content and developing more videos.

4.3 Online tools and resources

A wide variety of tools are available on the OCL website. In addition to making available full texts of the Act, the Code, and associated regulations (the Lobbyists Registration Regulations, and the DPOH Regulations), the OCL website includes a variety of tools that aid in the interpretation and understanding of these legislative documents. In particular, the OCL has created an annotated version of both the Act and the Code, which include notes on the Commissioner's interpretations of specific terms and statements, as well as links to related interpretation bulletins,^{Footnote 10} and advisory opinions.^{Footnote 11} Also included on the website are reports on investigations of alleged breaches of the Act and the Code, such as unregistered lobbying. In addition, there are online tools to aid in the interpretation of the Code, such as documents providing guidance and advice from the Commissioner regarding application of several Code rules, including Rule 6 (Conflict of Interest), Rules 7 and 8 (Preferential Access), Rule 9 (Political Activities), and Rule 10 (Gifts).

The OCL has also developed a series of help tools to assist in the registration process, including the following:

• a PDF, as well as video tutorials, outlining the Steps to Register
• presentations outlining the requirements of the Act and regulations, the role of the RCSD, and registration and reporting processes, each tailored for the different lobbyist types
• a tool that guides users through a series of questions to help them determine whether or not they are required to register in the federal Registry of Lobbyists, as well as a newly-added video entitled "Do I need to Register?" which is listed as a "quick link" on the OCL main webpage
• a series of worksheets (tailored as necessary to different lobbyist types) detailing information required by the LRS to assist in the registration process and submission of registration updates and MCRs
• a Frequently Asked Questions (FAQ) page providing answers to questions frequently received by the OCL from lobbyists, organized according to their applicability to different lobbyist types
• a Guide to Registration, which provides a comprehensive overview of the lobbyist registration process
• a series of registration tutorials

4.3.1 Awareness and use

The RCSD refers registrants and representatives to the OCL's online tools and resources when Registration Advisors respond to enquiries and during outreach activities. Based on interviews and survey results, most registrants and representatives have used at least one of the available tools and supports (86% of survey respondents had used at least one tool), which indicates a fairly high level of awareness among the lobbying community of at least some of the tools.

The most used tools, based on available web analytics (for the period February 10, 2016 to March 7, 2018), survey results, and interviews are listed below.

• Of the listed tools and documents, the most often used, and the only one used by a majority of survey respondents (55%), was the Guidance on the Lobbyists' Code of Conduct.^{Footnote 12}
• The Steps to Register (.pdf) is the second most used tool based on the survey results (48% of respondents) and had 1,551 unique page views.
• The FAQ page is the most accessed of the help tools, based on web analytics, with 6,730 unique page views and an average time spent on the page of just under four minutes. Confirming its high usage among the available tools, 42% of survey respondents reported using the FAQ page, making it the third most used tool. Internal key informants also reported that they use the FAQ page themselves as well as refer clients to it.
• The Guide to Registration is the fourth most accessed tool based on the survey results (40% of respondents) and had 1,331 unique page views.

As shown in Table 2, web analytics are not available for all of the tools enquired about through the survey. There is no precise alignment between the web analytics and survey responses. In particular, just over a quarter (27%) of respondents reported using one of the presentations for registrants, while this had the second highest number of unique page views (6,678).

Worksheets appear to be underutilized based on interview and survey results. Between approximately one tenth and one quarter of survey respondents had used a worksheet, depending on the category of registrant and the type of worksheet. Internal key informants who mentioned the worksheets reported that they typically do not refer clients to them. Web analytics reflect the reported lower use of the worksheets, as the web page for the worksheets had 1,401 unique page views (the third lowest amount of page views among the tools for which information was available).

The Steps to Register video also receives less use, based on the survey results, as less than one fifth (17%) of respondents reported using it. Internal key informants considered the video misleading or confusing, as it focuses more on account creation rather than registration.

Overall, the registration help tools (with the exception of the worksheets and the Steps to Register video) were used by more respondents than were the advice and interpretation tools, with the Guidance on the Lobbyists' Code of Conduct noted above as the main exception. In terms of advice and interpretation tools, over one third of respondents had used interpretation bulletins (37%), and about one quarter had used the Annotated Lobbying Act (27%), the Annotated Lobbyists' Code of Conduct (26%), and advisory opinions (24%).

4.3.2 Satisfaction with online tools and resources

Most registrants and representatives are satisfied with the online tools and resources. Three quarters (75%) of survey respondents were satisfied/very satisfied (see Table 3).

Reflecting the general level of satisfaction with the tools and resources, a minority of respondents indicated the need to improve them (see Figure 2, next page). The online documents or tools with the highest proportion of respondents indicating that improvements were needed were the following:

• Advisory opinions (25%)
• Interpretation bulletins (24%)
• FAQ (23%) — this was one of the tools most used by respondents, with 42% of respondents reporting using this tool
• In-house Corporation Details Worksheet (22%)
• Steps to Register (video) (22%)
• Monthly Communication Report Worksheet (21%)

Other online documents or tools are also worth noting, as they were among the most often used resources by respondents.

• The Guidance on the Lobbyists' Code of Conduct was the most used tool (by 55% of respondents), and 17% of those who used it indicated that it needs improvement.
• The Steps to Register (.pdf) was used by 48% of respondents, and 18% of those respondents believed that it needs improvement.
• The Guide to Registration was used by 40% of respondents, and approximately one fifth of respondents who had used it (19%) reported that it needed improvement.

Figure 2 - Text version

Proportion of respondents who believe the online document or tool needs improvement*
Thinking of the online documents and tools that you have used, please indicate which ones need improvement.

	Proportion of respondents
Advisory opinions	25%
Interpretation bulletins	24%
Frequently Asked Questions	23%
In-House Corporation Details Worksheet	22%
Steps to Register (video)	22%
Monthly Communication Report Worksheet	21%
A Guide to Registration	19%
Presentation for In-house Organisation Lobbyists	19%
Client Details Worksheet	18%
Are You Required to Register?	18%
Presentation for Consultant Lobbyists	18%
Steps to Register (.pdf)	18%
Guidance on the Lobbyists' Code of Conduct	17%
Lobbying Information Worksheet	16%
In-House Organization Details Worksheet	15%
Registration Worksheet	15%
Status of officials during an election campaign	14%
Presentation for In-house Corporation Lobbyists	13%
Annotated Lobbying Act	13%
Annotated Lobbyists' Code of Conduct	11%
Lobbyists' Details Worksheet	7%

*Base: Varies based on the number of respondents who used document or tool.

4.3.3 Suggestions for improvement

Most internal key informants thought the current online documents and tools were sufficient, and there were no gaps in terms of the types of tools or their content. The survey results above also confirm the general satisfaction with the available tools. That being said, based on interviews, a few suggestions for improvement were indicated.

Streamline website and improve the ease of finding tools: The registrant and representative key informants who had used the online resources generally found them helpful, although a few noted that they were not easy to locate. In particular, they noted the FAQ document and the Guide to Registration as very useful tools that are "buried" in the website. All of the internal key informants also believe that the website needs improvement, noting that it is difficult to navigate and the online tools are not easy to find.

Simplify tools and offer more graphics/visual tools: A few internal key informants also mentioned that the language used in interpretation bulletins and other tools could be simplified. Survey respondents commented that "There are too many documents and too much info. Tools should be simpler. You've made the whole thing too complicated" and "All of the tools would benefit by being made more direct and clear." As one survey respondent suggested, "Less bureaucratic; more common language and basic outlining of procedures." Some internal key informants suggested that more visual tools that use graphics or video could help simplify the information provided to registrants and representatives. Since the interviews and survey, the OCL has added another video on "Do I need to register?"

Other registries identified their best practices in addressing the need to simplify materials for users. The British Columbia Office of the Registrar of Lobbyists has a "quick tips" document available online that is credited with reducing the number of telephone enquiries about registration.^{Footnote 13} The British Columbia Office of the Registrar of Lobbyists has a flow chart for determining whether one should register, which was noted as a best practice by a registrant and representative key informant, as well as the BC registry key informant. A best practice identified by the Office of the Lobbyist Registrar for the City of Toronto was to offer video tutorials, flow charts, and an interactive online tool to help individuals determine if they should register. In development is a counterpart to that tool, which will be a flow chart for the public service and all POHs called, "Am I being lobbied?" which is intended to help them know when to direct someone to the Office of the Lobbyist Registrar.

Offer newsletters: Internal key informants suggested that quarterly emails or newsletters would be useful to keep registrants and representatives apprised of changes or developments. In fact, most of the other registries interviewed had e-newsletters that they considered to be best practices. A specific example is the British Columbia Office of the Registrar newsletter, which has a section on "lessons learned" that highlights errors made by registrants so that other lobbyists can learn from them.

Consider more/better use of social media The Office of the Lobbyist Registrar for the City of Toronto has expanded the use of internet tools like apps, as well as its use of social media. The use of multiple social media platforms (Twitter, Facebook, LinkedIn, and Instagram) was credited with reducing the number of open accounts and encouraging registrants to be more proactive with respect to compliance. For example, the Twitter feed of the Office of the Lobbyist Registrar pushes information and tips to its audience. The OCL may want to review its Twitter feed and compare it to the Office of the Lobbyist Registrar. For example, the new OCL video "Do I need to register?" was published on August 24, 2018 but does not appear in the OCL's Twitter feed.

Develop tools to address identified needs Some internal key informants noted that the RCSD cannot currently track the types/topics of questions it receives, which is considered a barrier to understanding where to direct tool development and education efforts. In addition, they believe that more emphasis could be given to timelines in materials, as missing deadlines is the most common compliance issue. While the RCSD sends reminder emails with timelines, they appear not to be read, so finding other ways to flag upcoming timelines would be helpful. Flagging upcoming deadlines is an area where some registries use social media.

4.4 Accessibility of the LRS

The evaluation considered the accessibility of the LRS primarily in terms of the registration process and MCRs.

4.4.1 Registration process

The evaluation findings indicate that registrants and representatives find the registration process to be accessible. Survey respondents generally consider LRS registration procedures to be clear (see Table 4).

• About three quarters of respondents consider the procedures for creating an account (75%), and the procedures for completing and certifying a registration (72%), to be clear or very clear.
• Approximately two thirds of respondents consider the steps required to complete a registration (70%), and the process for completing and submitting the RUA (69%), to be clear or very clear.

Overall, about two thirds of survey respondents were satisfied or very satisfied with the ease of completing all of the steps of the registration process (62%) and the ease of inputting information into the system (69%). In terms of timeliness of the process, almost three quarters (72%) of respondents were satisfied or very satisfied with the length of time between the start of the registration process and a registration being published. Details are in Table 5.

While the results in Tables 4 and 5 are generally positive, it is worth noting that 14% of respondents are dissatisfied with the ease of completing all of the steps of the registration process, and the procedures considered the least clear are the steps required to complete a registration and completing and submitting the RUA (considered unclear or very unclear by 13% and 11%, respectively). The current process for opening an account is a two-step process whereby the registrant first creates a profile and then must complete and submit by email or mail a signed RUA. Once the RUA is reviewed by a Registration Advisor and accepted, the registrant's account is activated, typically within 24 hours. However, the registrant is still not registered with the OCL until a registration is completed. According to internal key informants, the requirement of the RUA creates a few issues:

• the RUA is long (five pages) and uses a lot of technical terms, which means it is often completed incorrectly;
• as a result, there can be substantial back and forth between the Registration Advisor and the registrant before the RUA is accepted;
• it is not infrequent that registrants believe that once the RUA is accepted, they are registered, despite receiving an email with further instructions on the need to complete a registration (as a result their accounts remain pending); and
• the additional step of submitting the RUA is thought to hinder timely registration for consultant lobbyists who have less time to register than in-house corporations and organizations (10 calendar days as opposed to two months).

In recognition of this confusing and awkward process, the RCSD is changing its procedures so that the RUA is not required and the information required can be submitted online when creating an account.

For the procedures related to maintaining an account, including recertification (five-month), change of registrant, and reactivation of an account, a number of respondents (about one third) indicated that they could not provide an opinion, which is likely due to their never having to use the LRS for those purposes. When removing the "don't know" responses, for those three activities, a majority considered the procedures for how to undertake them to be clear or very clear (56%-58%). However, about one fifth of respondents considered the procedures to be unclear or very unclear, which indicates potential areas for improvement (see Table 6).

Interviews with registrants and representatives confirmed the generally positive survey findings, with almost all of those interviewed reporting that they found the registration process to be relatively clear and intuitive, although some noted that it can be intimidating at first and that there is a learning curve. According to these key informants, once users have some experience with the system, it is easy to use. The link at the top of each page for contacting the OCL, should the registrant or representative have a question, was considered sufficient to ensure accessibility of the LRS to its users. For those registrant and representative key informants who had used other lobbyist registries (provincial or municipal), they generally found the federal registry to be more accessible and easier to use.

4.4.2 Reporting

Based on interview and survey results, the LRS procedures related to MCRs are considered to be clear and reasonable.

• Almost all registrant and representative key informants reported that, in general, the requirements for MCRs are clear and straightforward. Those who found the requirements related to MCRs unclear were those registrant and representative key informants who had not been in their position as long (e.g., less than one year).
• About three quarters (76%) of survey respondents consider the timelines for MCRs to be clear or very clear, as did the registrants and representatives key informants who commented.
• Two thirds of respondents (66%) consider the requirements on what information to include in an MCR to be clear. However, 16% of respondents consider the information required for MCRs to be unclear.

4.4.3 Potential improvements

While the main evaluation finding is that the LRS is an accessible system, survey and interview results pointed to potential areas for improvement. Each of the below areas were identified by a minority of key informants and survey respondents.

Reducing the complexity of the system While many of the following suggestions for improvement are aspects of addressing the complexity of the system, this was a point raised most often by survey respondents when asked to provide suggestions for improvement.^{Footnote 14} Many of these respondents noted that they were not frequent users of the system. Some more detailed requests included: provide more targeted outreach to individuals and organizations that are not full-time lobbyists; allow multiple meetings to be inputted, including uploading them in an Excel format, rather than having to save and certify each meeting individually; and consider allowing "umbrella" registrations for Board members of an organization, as requiring individual registrations for individuals not regularly engaged in lobbying is onerous. Other examples of the feedback from survey respondents follows:

"This is far too complicated (there are a half dozen videos on how to fill forms in — that should tell you something)."

"Provide new users with a 'plain language' guide (with representative examples) for each of the main elements within the Lobbyists Registration System. The current process implies a level of familiarity and experience with the process and the body of knowledge related to lobbying (which was an invalid assumption for me and for a number of members of my organization who were all registering as lobbyists for the first time). Keep it simple; use plain language; provide examples."

Streamlining the registration process: As discussed above, the RCSD is already removing the need to send in a RUA and instead that information will be gathered online as part of the account creation process. In addition, based on internal key informant interviews, the RCSD should consider reducing the amount of information currently required by the RUA as well as using less technical language.

Addressing issues with navigating the LRS: According to registrant and representative key informants, survey respondents, and internal key informants, navigation of the LRS could be improved.^{Footnote 15} In particular, it was suggested that an overview page would be helpful where the user can choose which sections to edit; currently, the user must edit each section separately in a way that was described as having to "jump in and out of sections." Other examples of navigation issues from comments made on the survey include:

"Website is not user-friendly, particularly for monthly communications reports. Too much scrolling is required and there are too many windows, each of which take time to load."

"Website is confusing and too complex in moving from one screen to another. It is not clear what is required and how to get to the appropriate screens both upon initial registration, reporting contacts and updating information."

"More user-friendly platform for submitting reports. It is cumbersome to navigate if you don't use it on a regular basis."

According to internal key informants, assisting LRS users could be improved if the computer screen of the Registration Advisor could be shared with registrants and representatives. Because navigation of the site can be difficult (e.g., scrolling too far or not enough), being able to show the correct location in LRS to the registrant and representative would reduce frustration and confusion.

Providing more assistance with identifying DPOHs: While links to DPOH instructions and the definition of a DPOH are embedded within, and accessible from, each step for creating an MCR within the LRS, based on interviews and survey responses, identifying DPOHs remains an issue. Registrant and representative key informants reported having difficulties with determining the identity of DPOHs, particularly when it comes to determining "comparable positions" to those listed as well as what are "oral and arranged communications" with DPOHs. Some survey respondents^{Footnote 16} also reported difficulties identifying DPOHs either because they found the definition of a DPOH to be unclear or they had difficulty confirming if someone was a DPOH, as the Government Electronic Directory Services (GEDS) is not up to date. According to internal key informants, identifying DPOHs is a common enquiry.

Providing more assistance with inputting information: Internal key informants suggested that pop-up texts and more logic checks in LRS could potentially reduce the number of errors as well as the number of questions received by Registration Advisors. Pop-ups would address registrant and representative key informants' suggestion of providing more assistance with definitions. Registrant and representative key informants also noted difficulties with classifying activities based on the options listed in the system, finding the choices provided to be too limited. A few also mentioned that the Québec lobbyist registry allows the user to indicate multiple government departments in one drop-down list, which was considered easier to use than the LRS, where federal departments must be added one by one and it's not entirely clear if the additional federal departments are saved.

Focusing on methods to support and improve compliance: The LRS does have methods to support timely and accurate reporting by sending automatic emails to prompt clients to upload documents in order to complete their application, to remind them of deadlines for MCRs, and to notify them that they have not logged in for five months and will need to do so to keep their account active. However, it was suggested by internal key informants that perhaps the LRS could have registrants indicate if they want more information/more frequent reminders. Other suggestions to improve compliance included:

• prominently displaying reporting deadlines on social media, the website, as well as when logging in (e.g., an "upcoming dates" notice);
• explaining the difference between a registration and MCRs (the main filings required) when logging into LRS and on the website; and
• having the RCSD contact information more visible on each screen to encourage users to contact them if they are having difficulties completing the forms.

Offering a mobile-friendly platform: Currently, the LRS is not mobile-friendly and RCSD is planning to make the application more mobile-friendly. Data on LRS usage by device shows that the LRS website and application are being accessed primarily by desktop computers, as opposed to mobile devices or tablets; between February 22, 2016 and November 16, 2017, 80.55% of site usage was from desktop computers. Mobile device and tablet usage accounted for 14.80% and 4.65% of site usage over this period, respectively. Further, the data provides some indication that users may experience more difficulties in accessing the LRS from mobile devices and tablets than from desktop computers. From February 22, 2016 to November 16, 2017, the bounce rate^{Footnote 17} was higher for mobile devices (75.28%) and tablets (69.34%) than for desktop computers (57.50%). In addition, the average number of pages viewed per session was lower for mobile devices (2.28) and tablets (2.88) than for desktop computers (6.0).

A few internal key informants raised the need for the LRS to be mobile-friendly so that users can input information from their phone or tablet. According to some of the registrants and representatives interviewed and a few survey respondents, a mobile app or platform would be helpful, particularly for MCRs. However, overall survey results showed less interest in making the LRS mobile-friendly. More respondents rated accessibility on a mobile device as unimportant (38% of respondents rated importance as 1-3) than important (26% of respondents rated as 8-10) (see Table 8). According to the Ontario Lobbyist Registry, despite the site being mobile-friendly, most people do not access the registry on their mobile devices due to the complexity of the system in terms of the number of forms. Based on that experience, it is critical to have mobile-friendly content and structure as, without that, users who wish to access the site on their mobile devices will not be able to do so.

Ensuring a fully bilingual system: It was noted that the system used by Registration Advisors for entering notes is not bilingual and will make "corrections" to notes entered in French, creating mistakes. This is an accessibility issue, as notes can now be shared with registrants and representatives.

4.5 Measures of success

4.5.1 Compliance with service standards

The OCL has developed client service standards for the RCSD. The OCL does not report on its compliance with all of its service standards in its annual reports, so for some standards, the information had to be calculated for this evaluation. As shown in Table 9, those standards for which information is available are largely being met.

The OCL may want to review its service standards to align them better with the established characteristics of good service standards established by the Treasury Board of Canada (Treasury Board, 2012).

• The current set of service standards focus exclusively on timeliness; the other dimensions of service standards, namely accuracy and access, are not reflected. Internal key informants provided suggestions for other measures of success that could help address this issue, such as

  • tracking number and type of questions so the RCSD can determine if the number of questions overall or in specific areas are declining;
  • tracking the proportion of registration and communication reports that require corrections overall and by type of correction; and
  • conducting client surveys so satisfaction with quality and accuracy of responses are measured.

• The service standards are generally client-focused but should be paired with an operational performance target. Answering telephone calls pairs these two components: calls must be answered within 30 seconds (client-focused target) 80% of the time (operational target for internal accountability).

• Service standards should be transparent to clients. The issue with the 30 second service standard mentioned above is that it only applies to a minority of the calls received during business hours. The service standard only applies to calls that go to the Registration Advisors' queues, and does not apply to calls that go to the general queue, which is most of the calls received. That distinction is not made clear when reporting on the service standard. Given that the service standard is met even when all calls during business hours are considered (regardless of the queue), there is no reason not to apply the service standard to all calls received during business hours.

• Service standards should be transparent, which means they should be monitored and reported to stakeholders. Currently, the OCL annual reports do not include information on all of the service standards. If OCL is not tracking information and reporting on it for all of these service standards, perhaps those standards not being tracked and reported on should be dropped.

• Service standards should be clearly communicated so that staff and other stakeholders understand how they are measured. Some RCSD staff reported that they do not know when the "clock" begins to run for service standards (i.e., whether the day of receipt counts as one of the business days). This was particularly the case for responding to less complicated emails or approving or sending back registrations.

• Service standards should be reliable and valid. Currently, no definition exists for what a "complex" enquiry is. Given the lack of an agreed upon definition, there is the potential that this service standard is not measured consistently and may not, in fact, measure responsiveness to "complex" enquiries.

• Service standards should be ambitious but realistic. Several internal key informants queried the complex enquiry standard. They considered 14 calendar days to be too long, and also questioned the use of calendar days when other service standards are based on business days. In addition, the high rate of compliance with some standards, coupled with internal key informant views that they are reasonable and attainable, even for new employees, may indicate they are not ambitious enough.

• There is a different approach for email and telephone enquiries. Voicemails are "responded to" within 24 hours versus emails which are "acknowledged" within 24 hours and "addressed" within two business days if the enquiries are not complex, and within 14 calendar days if they are complex.

Internal key informants consider the current standards to strike the right balance between good client service and efficient use of OCL resources. In any review of service standards, that balance should be kept in mind, as well as the consideration of whether the standards are the most appropriate by which to measure success and high quality client service.

4.5.2 Satisfaction with client services

Overall satisfaction

The evaluation found a high level of satisfaction based on interviews and the survey with registrants and representatives. Overall, most (86%) survey respondents indicated that they were highly satisfied or satisfied with the support they received from Registration Advisors. Only 3% reported being dissatisfied, with the remainder being neither satisfied nor dissatisfied (10%), or responding that they don't know (1%). Registrant and representative key informants who had communicated with their Registration Advisors in the last year echoed the survey findings, as almost all were satisfied in terms of responsiveness, respectfulness, clarity of the advice or instructions given, and the ability to answer questions. They described their communication with the Registration Advisors to be "very straightforward," and "excellent service," and that the Registration Advisors "were doing everything they could to provide us with the information." One registrant and representative key informant was dissatisfied with the service provided in terms of its respectfulness and helpfulness.

Survey responses indicate only a small difference in the level of satisfaction with assistance received based on whether it was telephone or email assistance. As shown in Figure 3 (next page), respondents tended to be slightly more satisfied with telephone assistance than they were with email assistance across the dimensions explored in the survey. Respondents were most satisfied with the respectfulness of the communications (for telephone assistance, 95% were satisfied or very satisfied, and for email assistance, 90% were satisfied or very satisfied). The proportion of respondents who were satisfied or very satisfied with the other dimensions are listed below:

• Responsiveness to enquiries: 90% telephone and 84% email
• Ability to answer questions and/or provide the advice needed: 83% telephone and 77% email
• Clarity of advice or instructions given: 81% telephone and 78% email

Most of the remaining respondents were neither satisfied nor dissatisfied. Respondents who were dissatisfied were in single digits.

Figure 3 - Text version

Satisfaction with client services assistance
Telephone (n=263)
Email (n=275)
How satisfied were you regarding the assistance you received from the Registration Advisor in the following areas?

Area	Communication tool	Very dissatisfied/Dissatisfied	Neutral	Satisfied/Very satisfied
Respectfulness of communications	Telephone	1%	4%	95%
	Email	1%	7%	92%
Responsiveness to enquiries	Telephone	3%	7%	90%
	Email	3%	12%	85%
Ability to answer questions and/or provide advice needed	Telephone	6%	11%	83%
	Email	5%	16%	79%
Clarity of advice or instructions given	Telephone	8%	12%	81%
	Email	6%	14%	80%

Satisfaction with the ability to answer questions and the clarity of advice or instructions given are substantially lower than respectfulness and responsiveness. The reason for this could be related to Registration Advisors not being able to address questions related to the Code for the period of time covered by the evaluation. The OCL desired for enquiries about the Code to be handled by the policy unit after the Code was revised in 2015. Internal key informants thought that Registration Advisors can now respond to more of these common Code-related questions as there exist templates for how to respond to these questions.^{Footnote 18} Other indications for the lower satisfaction rating include the comments made by some registrant and representative interviewees and survey respondents, who noted that it can be frustrating when the Registration Advisors redirect them to the website or restate the Act or the Code.

Based on key informant interviews with registrants and representatives, there may be some benefit to addressing the limitations related to the assistance that Registration Advisors can provide. As one key informant noted, Registration Advisors are limited in the assistance that they can provide, as decisions on whether to register, how to report, and so forth are the responsibility of the registrant. That being said, it was noted that more specific advice in writing would be helpful. It was noted that small non-profits often do not have the resources to consult with legal counsel and may find it difficult to get the advice needed to make decisions. It was suggested that the OCL may want to do more outreach work with small non-profits, particularly so that paid Board of Directors are aware of the lobbying rules.

Assignment of registration advisors

The evaluation found that the assignment of Registration Advisors is important to clients. When asked on a scale of 1 to 10 the importance of having an assigned Registration Advisor, the average response was 7.55. Almost one third of respondents (31%) gave the importance of having an assigned Registration Advisor a 10 (very important). It is worth noting that 8% of respondents indicated that they did not know they had an assigned Registration Advisor, which did not vary based on type of respondent.

Internal key informants also believe that the assignment of Registration Advisors is working well from the perspective of client service, as well as workload management. The Manager of Client Services assigns first-time registrants to a Registration Advisor and an orientation email is sent to the new registrant with the name of their Advisor. The assignment is made with an eye to balancing the workload of Registration Advisors, but also keeping registrants from the same firm together. Assignments also consider the distribution of the type of registrants, and effort is made to ensure that Registration Advisors assist a variety of registrants (e.g., banks are not all assigned to one Advisor). Internal key informants thought that the assignment process had many advantages:

• Assigning Registration Advisors to a registrant helps build a rapport and ensures continuity of service. The Registration Advisor gains a good understanding of the registrant's business and the terminology of their work, which results in better client service.
• Assigning a firm to the same Registration Advisor creates efficiencies for both the OCL and the client. Registrants can copy information from a colleague's approved registration, which saves them time and requires less intensive review by the Registration Advisor. Similarly, responding to a question for one registration will then resolve the question for the other registrations for that firm and ensure that there is a consistent approach.
• Having each Registration Advisor work with a variety of types of registrants distributes experience. In so doing, in situations where the assigned Registration Advisor is not available, the person assisting may still have experience with that type of organization/sector and understand its issues.

Internal key informants noted that, when the assigned Registration Advisor is not available, another Registration Advisor can still assist the registrant or representative, as detailed notes are kept in each file. That being said, it was reported that most callers prefer to speak to their assigned Registration Advisor and prefer to be called back if they are not available.

4.5.3 Client level of knowledge

Success of client services can be measured, in part, by the clients' knowledge of their obligations under the Act and the Code. Most of the registrants and representatives interviewed attributed their knowledge of their obligations in part to the resources and assistance provided by the OCL. They also noted that the rules related to lobbying activities are complicated and pointed out that their years of experience also have contributed to their level of understanding. According to these key informants, the desire to ensure that they follow the law is strong, as being investigated for lobbying activities would adversely impact their reputation and/or that of the organization for which they are lobbying.

Based on survey and interview results, registrants and representatives consider their understanding of their obligations related to registering and reporting lobbying activities to be generally good, although there is a room for improvement. The majority of survey respondents (56%) assessed their level of understanding of the obligations related to registering and reporting lobbying activities under the Act and the Code to be high or very high. Four fifths of respondents (39%) assessed their understanding as moderate, and those considering their understanding to be low or very low was 5%.

As mentioned in Section 0 under "suggestions for improvement," internal key informants emphasized the need to better track the subject matter of enquiries so that RCSD can better identify areas where more education and assistance are needed. In that regard, evaluation findings might prove of assistance in identifying areas to target. About half of registrant and representative key informants provided examples of areas where they believe that their or their organization's understanding could be improved. Each example was mentioned by one or two key informants and indicate areas where the OCL may want to target education or other efforts (tools, etc.).

• The 20% threshold is confusing to some and they noted it can be hard to know whether their organization or corporation has reached the threshold. Internal key informants also identified this as a relatively frequent subject of enquiries.
• Understanding who is a DPOH was noted as an area where more education or better tools could be useful. Related to this, what constitutes "oral and arranged communications" with DPOHs was considered unclear.^{Footnote 19} Internal key informants also mentioned DPOH issues as a frequent enquiry.
• Improving registrants' and representatives' understanding of what constitutes activities that must be registered as lobbying was also suggested as an area that the OCL could focus its efforts on.
• The section where the registrant/representative lists government funding could have directions that explicitly state the funding listed must include municipal, provincial, federal, and foreign funding. Currently, it was thought that registrants/representatives may interpret this section to only require a listing of federal funding. This section in the LRS does explicitly refer to all orders of government in Canada, as well as foreign governments. However, as internal key informants raised this issue too, the issue may be about how to better highlight this information, such as using pop-ups or an "i" icon that users can click on for more information.
• According to both internal key informants and registrants and representatives, more/ clearer information on the "nuts and bolts" of the system is needed: how to register, what is a registration, what is an MCR, deadlines, how to update the name of the most highly paid official (CEO or Executive Director) for in-house organizations or corporations, how to correct subject-matter details, how to link accounts, and how to set up consulting firm profiles.

4.5.4 Level of compliance

Another indirect measure of the effectiveness of client services is the level of compliance. In terms of reporting requirements, compliance is high. In the last two fiscal years (2016/17 and 2017/18), the vast majority (94%) of MCRs were submitted in a timely manner (i.e., by the 15th of the month following the occurrence of a reportable communication) (OCL, 2017a, 2018). In addition, there appears to be increasing compliance with submission deadlines for these reports over time, as the percentage of timely MCRs increased from 82.6% in 2009/10 to 94.0% in both 2016/17 and 2017/18. This percentage rose steadily each year between 2009/10 and 2016/17, with the exception of a slight decrease between 2011/12 and 2012/13 (OCL, 2017a).

In addition to evidence that MCR deadlines are being met, there is evidence that the MCRs submitted by lobbyists/representatives are accurate. The OCL verifies a 5% sample of MCRs each month. In this process, DPOHs identified in the reports that are part of the sample are asked to validate the information provided. In 2017/18, the OCL verified a total of 1,112 MCRs with 215 DPOHs. The vast majority (93%) of these reports were confirmed to be accurate. Of the 75 reports that were found to be inaccurate, the inaccuracies identified were relatively minor; five reports had errors such as missing names or subject matters, and the rest had minor clerical errors or listed names of individuals who were not DPOHs (Office of the Commissioner of Lobbying of Canada, 2018a).

An area where compliance appears to be an issue is the existence of pending accounts. As described earlier, after creating an account in the LRS, registrants/registrant-representatives/representatives are required to submit an RUA before their account is activated. If the RUA is not submitted, the account is "pending" and the account holder cannot register lobbying activities. Over the past two fiscal years, just under one fifth of registrant and registrant-representative accounts created received pending account emails, indicating a delay between account creation and submission of the RUA.

• In 2017/18, a total of 843 registrant and registrant-representative accounts were created. Of these, 139 accounts (around 16%) received pending account emails.
• Similarly, in 2016/17, a total of 920 registrant and registrant-representative accounts were created. Of these, 174 accounts (around 19%) received pending account emails.

In 2017/18, 103 registrant/registrant-representative accounts received deactivation emails (representing almost three quarters [74%] of the accounts that received pending account emails during the fiscal year). Similarly, in 2016/17, 130 registrant/registrant-representative accounts received deactivation emails — again, representing three quarters 75% of the accounts that were pending over the fiscal year. It is not clear from available data what proportion of these accounts were deactivated as a result of registrants' failure to comply with RUA requirement, versus the proportion of accounts that were deactivated at the request of registrants/registrant-representatives. However, given the issue raised by internal key informants that some registrants and representatives are unclear about the steps in the registration process or whether to register (i.e., have they reached the 20% rule), the proportion of pending accounts may be an appropriate area for the RCSD to focus its efforts.

4.6 Efficiency of services

4.6.1 Cost per unit of service

For purposes of the evaluation, the cost per unit of service was used as an estimate of efficiency in providing services. The analysis used available financial data and statistics and, because calculating the cost per unit of service has not been undertaken before, is intended to provide a benchmark estimate of the RCSD's efficiency.

• The unit of service consists of the sum of the number of enquiries received by the RCSD and the number of registration activities.^{Footnote 20} It does not include other activities of the RCSD (e.g., outreach activities, internal projects).
• The cost of the RCSD is based solely on the human resource costs for the Director, Manager, four Registration Advisors, and the Business Systems Analysis. It does not include other costs (e.g., IT costs).

Because the analysis does not include all activities or all costs of the RCSD, it is only intended to be used as an estimate of efficiency/productivity over the two years covered by the evaluation. The analysis should not be used for other purposes (e.g., budgetary estimates).

As shown on the following page, the cost of the RCSD, based solely on the human resource costs for the positions listed above, increased by about $150,000 over the two fiscal years under consideration, but the number of enquiries and registration activities increased as well. The result is that the cost per unit of service increased by 9% from $22.26 to $24.24 between 2016/17 and 2017/18. The difference is primarily due to the RCSD having less than a full staffing complement for part of 2016/17.

* Source: OCL financial and administrative data

4.6.2 Tools, training, and processes

The evaluation found that the RCSD has taken several steps to improve its practices and the efficiency of its service delivery. In particular, evidence from documents and internal key informant interviews highlighted the following improvements:

• the development of templates, "cheat sheets," and a wiki assists Registration Advisors in doing their jobs as efficiently as possible
• improvements to the LRS include automation of some activities, such as contacting individuals with pending accounts (which staff used to do), improving the interface so information is easier to find, enhancing the search function so results can be sorted, and providing options so notes on file can be shared with the registrant or just with staff
• the continued development of more visual tools (videos, webinars, infographics)
• the new call routing system enables calls to be transferred to another Registration Advisor when the assigned Advisor is not available, and is credited by internal key informants with improving the approval time for registrations
• the automated verification of a sample of MCRs that is sent to DPOHs is now mobile-friendly
• a well-developed training plan in place for Registration Advisors (dated February 1, 2017) is considered good preparation for their roles as it includes time for new staff to shadow and be mentored by experienced Registration Advisors
• the assignment of Registration Advisors is considered to create efficiencies in a number of ways, as discussed in Section 4.5.2 under "Assignment of Registration Advisors."

Overall, staff believe that they have the tools, training, and resources to support them in doing their work. Only a few areas of improvement were noted by internal key informants. A few commented that finding the most up-to-date procedures and templates can be difficult and that the internal Procedures Manual is not considered to be a useful reference tool.

4.6.3 OCL collaboration

The evaluation also considered how the RCSD's efficiency might be positively or negatively affected by the collaboration across units within the OCL. Based on internal key informant interviews, the OCL worked in silos in the past, with each directorate or unit working independently with little cross-collaboration. This method of working is considered to be in the process of changing, and working collaboratively is still a work in progress.

In terms of the existing methods of collaboration, key informants mentioned forums that are more information-sharing opportunities than collaborative projects. The all-staff meetings are described as occurring biweekly and being opportunities for each area within the OCL to share information about their activities. In general, the all-staff meetings are considered to be an improvement, although suggestions were made to make the intent or objectives of the meetings — beyond information-sharing — clearer in order to encourage full participation. Eventually these meetings could go beyond information-sharing of individual directorates or units and involve more collaborative mechanisms to discuss and address strategic issues of the OCL.

Specific suggestions for more collaboration involved the policy unit and the RCSD collaborating around the drafting of policies. Having the perspective of the Registration Advisors could be useful to constructing policies, as it ensures a frontline perspective is reflected. In addition, enabling the Registration Advisors to respond to more complex enquiries was also seen as a collaborative project with legal counsel, the policy unit, and the RCSD. This is already under way.

The most-mentioned opportunity for reducing silos mentioned by internal key informants is enabling information-sharing between the Investigations Directorate and the RCSD. The Compliance Advisory Team (CAT) was one mechanism where the RCSD would share issues that could affect compliance of registrants and representatives. That sharing sometimes resulted in the development of materials to promote awareness of an issue and, therefore, compliance. However, the CAT is now meeting on an ad hoc basis.

Almost all internal key informants who raised this issue expressed a desire for the Investigations Directorate and the RCSD to share more information. Whether that should include information at the level of an individual registrant being investigated, or by sharing information on the types of investigations and penalties, was a potential difference of opinion. However, by far, the most common desire was sharing information on individual registrants. By linking the Investigations Directorate and the RCSD information management systems, both directorates would be aware of who is being investigated and for what, whether they have contacted the RCSD and for what reason, and how the RCSD or the policy unit responded. This sharing of information was considered important from both an investigations and client service perspective. These communications can be important to the ongoing investigation and can also help ensure consistent messaging is given. Currently, sufficiently detailed content of telephone conversations are not included in the RCSD information management system, which is considered to be a gap by some key informants who also believe that keeping these statistics would be useful for understanding what issues to target with education or tools. Based on internal communications, the new call routing system, the Enhanced per Agent Service (EPAS), allows the subject matter of the telephone call to be captured based on pre-defined lists that are quite broad (e.g., legislation), but cannot capture further details of the call. The current approach of keeping investigations separate from client services was to ensure that Registration Advisors did not inadvertently disclose information about the investigation, particularly when the matter has been referred to the Royal Canadian Mounted Police.^{Footnote 21} Internal key informants who support information-sharing noted that Registration Advisors have the same level of security clearance as investigators and, with proper training, will know how to respond when contacted by someone who is under investigation.

When looking at the approaches to this issue taken by other registries for best practices, there is not one model, and the philosophy behind sharing information on investigations differs substantially. The Québec Lobbyists' Commissioner uses a centralized information management system so the Advisors and the investigators can both access the same information. They considered this a best practice, as it ensures a consistent message when interacting with someone under investigation. The British Columbia registry manager, who is the person who handles lobbyists' enquiries, also works closely with investigators. The registry manager and investigations unit work together to deal with issues of non-compliance; the registry manager will try to resolve the issue before passing it to investigations and will also review the investigation report before it is sent to the lobbyist. The difference between British Columbia and the other registries is that there is only one staff member, the registry manager, who handles enquiries by lobbyists.

On the other side are the Ontario and City of Toronto lobbyist registries, which do not share information on investigations with staff. They do this to ensure that the knowledge of an investigation does not affect how staff approach a lobbyist who will likely have a long-term relationship with the registry and to protect the neutrality of the registry. The City of Toronto Act also specifies that investigations will be conducted in secret, so that has meant that investigation information is not shared with the registry team. However, information from the registry team is shared with the investigations team to ensure that they have all the information they need on the contacts that the lobbyist has had with the registry. They believe the separation is important to preserve the integrity of the investigation.

Given the evaluation findings, there is clearly internal support for sharing information between the RCSD and the Investigations Directorate, and the Québec Lobbyists' Commissioner provides an example of how this approach can work. The OCL could also consider sharing RCSD information on enquiries and contacts with the Investigations Directorate without sharing with the RCSD the fact that the registrant is under investigation, as per the City of Toronto lobbyist registry approach.

5.0 Conclusions and recommendations

This final section of the report presents conclusions and recommendations based on the findings presented in the previous sections.

The evaluation evidence supports the conclusion that client services and the LRS are accessible.

Assistance from Registration Advisors is experiencing increasing use. The evaluation found that the assistance provided to registrants and representatives by the RCSD receives substantial use, which demonstrates the need to offer email and telephone assistance. Over the two years covered by the evaluation (2016-17 and 2017-18), the client services provided by the RCSD have seen an increase of 27% in the number of total contacts (email and telephone) and a 38% increase when considering only contacts from registered lobbyists and representatives. Both methods of reaching the RCSD to request assistance—telephone and email—are used by clients, but based on administrative data, telephone is most used and interview results indicate that this is because registrants and representatives believe that they get a faster response by telephone.

Outreach services are offered to registrants and representatives. The RCSD offers webinar or in-person outreach sessions to new registrants and representatives as part of the orientation email sent by their Registration Advisor. Over the time period covered by the evaluation, the RCSD provided 87 outreach sessions to registrants and representatives.

The LRS is complex but is considered accessible. Those registrant and representative interviewees who are more frequent users of the LRS reported that the system is accessible, but acknowledged that this ease of use comes from experience.

In general, most of the registrants and representatives surveyed reported that the registration process was clear or very clear (from 69% to 75%, depending on the dimension of the process enquired about) and were satisfied with the ease of completing the steps in the registration process (62%). While a minority view, 14% of registrants and representatives were unsatisfied with the ease of completing all of the steps in the registration process and 11% considered the RUA process to be unclear. The RCSD is moving to a process where an RUA will not have to be separately submitted, which should improve this feature of the LRS. Aspects of registration that are less used, such as procedures related to change of registrant, recertification, and reactivation of an account, were considered to be less clear based on survey responses. This finding could point to areas where the RCSD might want to target education efforts or tools.

Based on interview and survey results, the LRS procedures related to MCRs are generally clear and reasonable, although two thirds (66%) of survey respondents considered the requirements regarding what information should be reported to be clear.

The evaluation found general satisfaction with the tools available to lobbyists.

A wide variety of tools are available on the OCL website, as well as through links in the LRS. These tools cover a range of topics and include many formats (e.g., text, video, interactive, diagrams). Most registrants and representatives surveyed (86%) had used at least one of the available tools and three quarters (75%) responded that they were satisfied with the available tools. When registrants and representatives surveyed were asked what tools need improvement, no tool was identified by more than 25% of respondents. Based on use as well as satisfaction with the individual tools, registration help tools, with the exception of the worksheets, tend to do better than do the interpretation and advice tools.

The RCSD is providing effective services based on all of the available measures of success

The service standards currently tracked are being met. Four service standards are met or largely met. Those that are met include answering 80% of telephone calls received during business hours within 30 seconds; responding to less complicated emails within two business days; and approving registrations or sending them back for corrections within three business days. The service standard that is largely met is responding to more complicated emails within 14 calendar days.

The evaluation found a high level of satisfaction with the assistance from Registration Advisors. Overall, most (86%) survey respondents indicated that they were highly satisfied or satisfied with the support they received from Registration Advisors. This level of satisfaction existed across various dimensions of service (respectfulness, responsiveness, ability to answer questions and/or provide advice needed; and the clarity of the advice or instructions given), with only a small difference based on whether the service was provided by email or telephone. Telephone assistance scored only slightly higher in terms of the level of satisfaction. The assignment of Registration Advisors was important to clients, and internal key informants also noted the benefits of assigning Advisors, including building rapport and ensuring continuity of service.

Clients' level of knowledge of their obligations under the Act and the Code is an indirect indication of success. The majority of registrants or representatives surveyed and interviewed believe that they have a good understanding of their obligations. Interviewees attributed this, in part, to the assistance they received from Registration Advisors and from the use of the available tools and resources.

Clients' level of compliance also provides an indirect measure of success. The level of compliance cannot be directly attributed to the effectiveness of client services, but is an indirect measure. Based on the available statistics, registrants' and representatives' compliance for meeting MCR reporting deadlines is high, as is the accuracy of the reports. The one compliance issue identified was the proportion of newly created accounts that become pending and then are deactivated. Strictly speaking, this may not be a compliance issue as much as registering either prematurely or when not necessary.

The RCSD services are efficient and cost-effective.

Cost per unit service remained steady over the two years. Even with the expanding demand for its services, the cost of RCSD staff per unit of service (enquiries and registration activities) has remained essentially the same, which is an indication of efficiency in service delivery.

The RCSD has taken steps to continuously improve. The evaluation found that staff felt they had the tools, training, and resources to support them in doing their work. The RCSD has developed templates, "cheat sheets" to simplify processes, and a wiki to assist Registration Advisors. The LRS has continued to be improved with automation of some activities that used to require staff time. A new call routing system is credited with improving approval time for registrations.

Areas for improvement

Outreach services. Comparing the total number of new registrations over the last two fiscal years (4,258) to the number of outreach sessions offered (87) during approximately the same time period indicates a potential underutilization of RCSD outreach. In addition, the outreach sessions were found to be too long and cover too many topics. The evaluation also found that the RCSD has ended its process for systematically gathering participant feedback.

The LRS. Detailed suggestions were made to reduce the complexity of the system. One suggestion made by registrants, representatives, and internal key informants was to have an overview page in order to reduce the navigation issues experienced by users, and/or enable Registration Advisors to share their computer screens with users so they can see the correct location in the LRS. More assistance with inputting information was also desired, such as pop-up texts and logic checks, in order to decrease errors. Finally, while there is some interest in the LRS becoming mobile-friendly, just over one quarter (26%) of survey respondents said this was very important to them.

Recommendations: The RCSD should explore ways to improve the navigation of the LRS and reduce user errors, such as through the use of an overview page and more pop-ups and logic checks.

Tools. The evaluation findings indicate that there is room for better promotion of tools and ensuring their visibility/accessibility. Based on survey results, only one tool (Guidance on the Lobbyists' Code of Conduct) was used by a majority (55%) of respondents. A tool often referred to as helpful by registrants, representatives, and internal OCL stakeholders interviewed was the FAQ page, which was also considered "buried" on the website. Also noteworthy is that many of the common questions received by Registration Advisors or mentioned by registrants or representatives (e.g., identifying DPOHs, interpreting the 20% rule) have tools that address them, but either clients are not locating the tools they need or the content is not sufficient to meet their needs. Improving the website should help address some of these issues, and the OCL could use its social media to promote tools and push information to its stakeholders. Other registries use social media to do so.

Other suggestions were to simplify tools — in terms of language used, number of tools, and amount of information — and to offer more visual tools/graphics. Best practices from other registries were a "quick tips" document, flow charts, and video tutorials. Many of these practices are in use or under way by the RCSD, but based on comments related to being overwhelmed by the quantity of tools, it will be important not to continue to add tools without considering the overall toolkit. One method of ensuring that the tools are relevant to stakeholders is to devise tools based on types of common enquiries or compliance issues. It was pointed out that the RCSD does not currently track the type/topic of questions with sufficient detail to use that information to inform tool development. One item that was raised repeatedly in interviews with registrants and representatives, as well as by some survey respondents, was the difficulty encountered in identifying DPOHs.

Service standards. Service standards should be transparent, which means they should be monitored and reported to stakeholders. Not all service standards are reported upon in the OCL annual reports or internally. In addition, the service standards focus on timeliness and do not include other dimensions of service, such as accuracy and access. Service standards should also include operational targets to ensure internal accountability (e.g., the percentage of the activity that should meet the standard). Some service standards would benefit from more clarity (e.g., the definition of a complex enquiry).

Collaboration within the OCL. Internal key informants believe that collaboration within the OCL has improved. Specific suggestions for further improvements included: having the policy unit work with the RCSD on drafting policies in order to include a frontline perspective; and (most often mentioned) enabling the sharing of information between the Investigations Directorate and the RCSD. As to the latter suggestion, there are other registries that engage in sharing information between both areas and one that enables investigators to access client services information, but not vice versa. A key consideration in fully sharing information is providing the proper training for Registration Advisors and ensuring that there is a client service rationale for Registration Advisors having access to information on investigations.

Appendix A – References

References cited

• Office of the Commissioner of Lobbying of Canada [OCL]. (2012, Feb 16). Mandate.
• Office of the Commissioner of Lobbying of Canada [OCL]. (2015a, Mar 31). Annual Report 2014-15 (PDF).
• Office of the Commissioner of Lobbying of Canada [OCL]. (2015c, Nov 30). Full text of the Lobbyists' Code of Conduct.
• Office of the Commissioner of Lobbying of Canada [OCL]. (2016, Mar 31). Annual Report 2015-16.
• Office of the Commissioner of Lobbying of Canada [OCL]. (2017a, Mar 31). Annual Report 2016-17.
• Office of the Commissioner of Lobbying of Canada [OCL]. (2017b, Mar 31). Client Service Standards.
• Office of the Commissioner of Lobbying of Canada [OCL]. (2018). Annual Report 2017-18.
• Treasury Board (2012) Guideline on Service Standards.

Appendix B – Survey methodology

Survey of registrants and representatives Methodology

Survey questionnaire

The survey questionnaire was designed by PRA Inc., the external consultant, with guidance and feedback from representatives of the Office of the Commissioner of Lobbying who had an in-depth understanding of the Lobbyists Registration System (LRS) and the Registration and Client Services Directorate (RCSD).

The survey questionnaire covered topics related to the following:

• the respondent's self-assessed knowledge of their obligations under the Lobbying Act (the Act) and the Lobbyists' Code of Conduct
• clarity of the LRS procedures
• satisfaction with the LRS
• use and satisfaction with assistance from Registration Advisors
• use and satisfaction with online tools and resources
• suggestions for improvement

The questionnaire included closed single response questions, multiple response questions, and open-ended questions. Most questions that asked for a rating (satisfaction, clarity) used a 5-point Likert scale. Two questions used a 10-point Likert scale in order to gather more nuance from respondents. These questions related to the importance of having an assigned Registration Advisor and the importance of being able to enter information on the LRS using a mobile platform.

Due to the skip logic, only an estimate of the number of unique pieces of information per respondent can be provided, but approximately 60 pieces of information from each respondent were collected (e.g., a single response question counts as one piece of information; for a scaled question applied to multiple items, each item is one piece of information). The online survey took approximately 10 minutes to complete.

Potential respondents and census approach

The target population included 2,520 registrants and representatives. Because the target population was manageable in size, a census approach was used.

Of the potential respondents, 2,451 had active email addresses and were sent the survey invitation. In total, 402 respondents completed the survey for a response rate of 16%.

Comparing the survey respondents to the target population on the two characteristics for which information was available, the results show that survey respondents were representative of the type of account, while there were some differences in the registrant type between the survey respondents and the target population. In particular, in-house corporations were under-represented and in-house organizations over-represented.

Margin of error

Although the research used a census approach rather than a random probability sampling methodology, the sample generated appears to be representative of the population. For comparison purposes only, a probability sample of this size (n=402) would have a margin of error of +/- 4.5%, 19 times out of 20.

Analysis

The three open-ended survey questions were coded by themes. All survey data were then analyzed using SPSS, a statistical software package. The survey data were not weighted. Cross-tabulations on account type and registration type were analyzed, but not included in the final evaluation report, as results were not statistically significant.