Security-conscious customers face tough choices when evaluating security vendors and their next-generation firewall offerings.
To simplify this process and improve transparency in the cybersecurity market, NetSecOPEN announces SonicWall is one of only four security vendors to be certified in its 2020 NetSecOPEN Test Report.
Tested with 465 combined Public and Private Common Vulnerability and Exposure (CVE) vulnerabilities at the InterOperability Laboratory of the University of New Hampshire, the SonicWall NSa 4650 firewall achieved 100% security effectiveness against all private CVEs used in the test — CVEs unknown to NGFW vendors. Overall, SonicWall rated 99% when factoring in the results of the public CVE test.
“This apples-to-apples comparison provides security buyers with validation of real-world performance and security effectiveness of next-generation firewalls when fully configured for realistic conditions,” said Atul Dhablania, Senior Vice President and Chief Operating Officer, SonicWall, in the official announcement.
Testing firewalls in real-world conditions
The NetSecOPEN open standard is designed to simulate various permutations of real-world test conditions, specifically to address the challenges faced by security professionals when measuring and determining if the tested firewall is performing the way vendors had promised. The value of this service is maximized when test findings help you make clear and conclusive product decisions based on incontrovertible evidence.
SonicWall is among the first to excelled in one of the industry’s most comprehensive, rigorous benchmark tests ever created for NGFW. In summary, the NetSecOPEN Test Report reveals that the SonicWall NSa 4650 NFGW:
- Demonstrated one of the highest security effectiveness ratings in the industry
- Blocked 100% of attacks against all private vulnerabilities used in the test
- Blocked 99% overall all attacks, private and public
- Proved fast performance measured by NetSecOPEN at 3.5 Gbps of threat protection and up to 1.95 Gbps SSL decryption and inspection throughput
- Affirmed its extremely high-performing and scalable enterprise security platform can meet the security and massive data and capacity demands of the largest of data centers
Firewall testing methodologies, metrics
Key performance indications (KPI), such as throughput, latency and other (see below) metrics, are important in determining products’ acceptability. These KPIs were recorded during NetSecOPEN testing using standard recommended firewall configurations and security features typically used in a real-world use case condition.
| KPI | MEANING | INTERPRETATION |
| CPS | TCP Connections Per Second | Measures the average established TCP connections per second in the sustaining period. For “TCP/HTTP(S) Connection Per Second” benchmarking test scenario, the KPI is measured average established and terminated TCP connections per second simultaneously. |
| TPUT | Throughput | Measures the average Layer 2 throughput within the sustaining period as well as average packets per seconds within the same period. The value of throughput is expressed in Kbit/s. |
| TPS | Application Transactions Per Second | Measures the average successfully completed application transactions per second in the sustaining period. |
| TTFB | Time to First Byte | Measure the minimum, maximum and average time to first byte. TTFB is the elapsed time between sending the SYN packet from the client and receiving the first byte of application date from the DUT/SUT. TTFB SHOULD be expressed in millisecond. |
| TTLB | Time to Last Byte | Measures the minimum, maximum and average per URL response time in the sustaining period. The latency is measured at Client and in this case would be the time duration between sending a GET request from Client and the receival of the complete response from the server. |
| CC | Concurrent TCP Connections | Measures the average concurrent open TCP connections in the sustaining period. |
Importance of transparent testing of cybersecurity products
Before making an important business-critical purchase decision that is central to the cyber-defense of an organization, decision-makers likely spent countless days exercising due diligence. This may include conducting extensive vendor research, catching up on analyst opinions and insights, going through various online forums and communities, seeking peer recommendations and, more importantly, finding that one trustworthy third-party review that can help guide your purchase decision.
Unfortunately, locating such reviews can be a bewildering exercise as most third-party testing vendors and their methodologies are not well-defined nor do they follow established open standards and criteria for testing and benchmarking NGFW performance.
Recognizing the fact that customers often rely on third-party reviews to validate vendors’ claims, SonicWall joined NetSecOPEN in December 2018, the first industry organization focused on the creation of open, transparent network security performance testing standards adopted by the Internet Engineering Task Force (IETF), as one of its first founding member.
SonicWall recognizes NetSecOPEN for its reputation as an independent and unbiased product test and validation organization. We endorse its IETF initiative, open standards and benchmarking methodology for network security device performance.
As a contributing member, SonicWall actively works with NetSecOPEN and other members to help define, refine and establish repeatable and consistent testing procedures, parameters, configurations, measurements and KPIs to produce what NetSecOPEN declares as a fair and reasonable comparison across all network security functions. This should give organizations total transparency about cybersecurity vendors and their products’ performance.
2020 NetSecOPEN Performance Report for SonicWall NSa 4650
Comparing firewalls? See why the SonicWall NSa 4650 next-generation firewall earned a perfect score in NetSecOPEN laboratory testing against more than 400 known and unknown vulnerabilities.
