In Romania, as in many jurisdictions worldwide, the year 2021 was marked by recovery and resilience out of the COVID-19 pandemic, ushered in under several safety and protection measures set by national authorities. The Chamber of Financial Auditors of Romania (CAFR) was authorized and delegated by the Authority for the Public Oversight of the Statutory Audit Activity (ASPAAS) the organization of both the initial professional examinations (pre-practical experience) and the final examination of professional competence after the training period for auditors. Safety of all candidates, staff, and other individuals that support the professional examination process was a high priority. Therefore, CAFR needed to find new approaches to organizing and marking exams from what CAFR would normally do. Prior to 2021, hundreds of candidates would come from all over the country to a central, controlled examination center and sit in-person in Bucharest for pen-and-paper examinations. Afterwards, papers were collected and graded by an Evaluation Commission that would meet in-person. Any appeals or complaints filed after the publication of results were also addressed by a Complaints Commission that also met in-person. The entire process was time-consuming and incurred significant costs and efforts.
In 2021, CAFR determined that the best way forward was through online examinations. The first step was entering into a tripartite agreement between CAFR, ASPAAS, and the Academy of Economic Studies of Bucharest — one of the most reputable economic universities in Romania — to leverage the Academy’s online examination platform, already in use for several years, tested by thousands of students, and at no additional fee for CAFR.
Pre-Exam
Importantly, there was an existing level of trust in this platform among all key stakeholders and CAFR knew that it could meet its needs in terms of questions that could be uploaded and format of the exams – i.e., it could accommodate both multiple choice questions, long-form questions, and a certain time limit could be set.
To avoid any suspected fraud, for each examination, CAFR uploaded into the library platform a significantly larger number of questions than those required for an in-person test and mapped to certain topics. The platform then randomly selected the questions for each candidate, thus also mitigating the human error component.
With increased digitalization of the exam, CAFR was also cognizant of cybersecurity, privacy, and confidentiality risks. Agreements were signed among relevant parties regarding confidentiality. Each candidate was provided an individual account with unique log-in details to the examination platform, and received the access details to a particular Zoom session- used for identity verification and monitoring during the entire exam. Candidates were also allowed to receive technical support if needed, either by using the Zoom session chat or by dialing a specific technical support line.
Still, CAFR had to ensure that there was buy-in among candidates that would be sitting for this new online examination experience. Prior to the exam, CAFR issued a specific set of instructions and guidelines for the examination and a few days before each exam, CAFR organized an online Zoom meeting with the candidates to explain procedures involved, the related monitoring aspects, and to answer any questions and concerns. Alongside this Zoom session, CAFR also organized a practice, simulation exam so that candidates would be directly familiar with the platform before the real examination.
During the Exam
During the entire exam, every candidate had to connect to the Zoom session with the audio and video signal on. Candidates’ audio-visuals and screen sharing feeds were recorded during the test for future reference and for any potential appeals or complaints. There was also a “central physical unit” composed of platform administrators, CAFR and ASPAAS staff, and other supervisors that were monitoring the Zoom sessions. It was very important that CAFR be transparent in its monitoring procedures to maintain a positive reputation among all stakeholders.
Post-Exam
Evaluators assessed the answers and graded the papers online. For multiple choice questions, the examination was fully automated. For long-form questions, the results were not automated, and evaluators assessed the answers and graded accordingly by connecting, with individual log-in details, to the examination platform. The results of the online exams were conveyed to the candidates via the CAFR and ASPAAS websites. Any complaints were also dealt with online by the Complaints Commission.
Challenges
Overall, CAFR has received favorable feedback about the online examinations experience from the past year. Gaining candidates’ trust in the examination platform, especially those who are not particularly tech-savvy was a primary obstacle to overcome. This is why CAFR invested in creating guidelines, practice exams, and provided candidates with an e-mail and phone number that were dedicated to answering any technical questions that might arise during the exam.
Another challenge was for the candidates to have all the technical devices ready to take these online examinations (i.e., up to date software, steady internet connection supplemented by a back-up connection such as Wi-Fi or a smartphone with hotspot capability). In 2021, all candidates took the tests remotely, from their chosen location, from all over Romania. There were candidates who unfortunately withdrew from an examination because of lack of necessary devices or poor internet connection. This will be an important consideration for what CAFR does in the future.
Will Online Exams Remain?
CAFR embraced the digital challenge imposed by the pandemic. CAFR has seen immediate benefits from online examinations, such as decreasing time and cost of setting up physical test centers and less travel expenses incurred by candidates. The entire examination process was quicker, including the post-examination stages such as grading, complaints, etc. Candidates were quite pleased with the overall online examination experience.
Another plus for the online exams is the accessibility to remote locations, enabling wider participation and access to the practical training or the profession. With an internet connection and electronic devices, candidates throughout the country could take the exam, from the comfort of their chosen location.
To attract more people to the audit profession (either as trainees or auditors) and to avoid additional expenses in the post-pandemic period (already troubled by cost increases), CAFR has begun discussions with the ASPAAS regarding the organization of the 2022 initial and final examinations with online and in-person format options. CAFR believes that online examinations are here to stay and will be an important part of the accountancy education ecosystem going forward. Is your PAO ready?
More on PAO Digital Transformation
There are huge opportunities for PAOs to advance their own organization’s success and our global professional community. Our journeys are unique, yet our experiences not uncommon. IFAC’s PAO Digital Readiness Assessment Tool is providing a platform for future synergies and transparency among PAOs as they start exploring technology solutions, new certifications, and training. Patience and tenacity are necessary, but digitalization will unlock new power for PAOs. The IFAC PAO Development & Advisory Group encourages all IFAC members to leverage and complete the free Tool and is available to connect with PAOs on this topic!
Explore IFAC’s regularly updated PAO Digital Transformation Series webpage , which houses helpful resources, articles and videos on Digital Transformation.
Continue the Conversation on Online Examinations
Do you have additional challenges, solutions, or opportunities related to online examinations? Do you have policies, procedures, guidance, et cetera that might be useful to PAOs? Are you willing to assist a PAO on the journey to online examinations? If your answer to any of these questions is “yes,” please contact us at education@ifac.org as we plan to continue to share your knowledge in the Accountancy Education eNews (subscribe today!) and link to your good practices in the IFAC Accountancy Education E-Tool.
