Acronis Votes to Adopt AMTSO’s Testing Protocol Standard

Acronis joined the Anti-Malware Testing Standard Organization (AMTSO) earlier this year to ensure its solutions adhere to the highest standards of security and help shape the evaluation criteria of future technologies. Acronis is happy to take part in establishing these protocols and procedures for the next generation of cyber protection and to help improve security testing practices overall within the industry.

One important step towards this collaboration was to approve and adopt AMTSO’s Testing Protocol Standard, a process that demanded years of development, trials, and consultation before final approval. It was finally adopted at the end of May, marking a major step for the industry. Here’s why it matters.

During the past two years, AMTSO has been developing this first testing Standard, combining the efforts of 20 vendors and testers in the Standards Working Group with regular reviews and feedback from across the membership. According to the AMTSO Standards Working Group, “Over the last few months, AMTSO has been running a public pilot of the Standard, putting several tests from major test labs through the complete process of establishing compliance with the Draft Standard, as well as gathering input from journalists and analysts outside the organization”.

During the recent vote, a wide majority of AMTSO members approved the Standard and have committed to using it. With this approval, the standard was adopted by the AMTSO membership and Board of Directors on May 22, 2018.

“This is big,” said Dennis Batchelder, President of AMTSO. “AMTSO Standard-based tests can remove biases and give enterprises and consumers information and context they need to choose their security providers.”

Let’s take a look what is regulated by this long-awaited testing protocol and how it helps ensure that tests are transparent, fair, and reliable.

The Standard covers a lot of things, we would like to highlight three key points: 

• Equal treatment – All vendors and products being tested (“Test Subjects”, in the formal terminology of the Standard) must be treated fairly and equally throughout testing. This is particularly important when addressing any disagreements between vendors and testers that might arise (called dispute processes) from a test. The Standard requires that any product which was not given the same access to such dispute processes is highlighted in any test reports. There are requirements for testers to disclose any other information which could potentially affect test outcomes, such as the sponsor or commissioner of a test.
• Notification – The Standard requires that all vendors whose products are included in a test are given advance notice that a test is planned and provided with details of how the test will be run. A “Test Plan” with detailed methodology and schedules should be created in advance as well. It ensures that vendors can review and assess test design and provide testers with feedback on any issues which may affect specific products or vendors. To make it easier for testers and vendors, AMTSO has developed a Contact List system to ensure notifications reach the right people. Anyone can join the Contact List, although most notifications will only be sent to security product vendors.
• Right of feedback – The Standard provides multiple opportunities for vendors to share official commentary while their products are being tested. Thus, vendors can publicly share their opinions on the design and implementation of a Test Plan. Testers also have the right to respond to commentary submitted by the vendors to ensure fairness and accuracy.

Vendors taking part in tests also have responsibilities under the AMTSO Standard. To obtain “Participant” status under the Standard, which brings with it additional rights, vendors are expected to respond to both calls for commentary and tester issues in a timely manner. Participants must also disclose any information that could be valuable to the tester in completing their analysis of products.

Alongside the development and implementation of the Standard, AMTSO has committed to providing the framework and support systems to help testers demonstrate their compliance with the Standard in an efficient and practical way.

The ultimate goal of AMTSO, which Acronis as a company deeply shares, is to provide what is best for consumers: clear, fair, and professional evaluation of security products. Better testing means data that are more useful for readers of test reports, which in turn makes us all more secure.