Open Raven Platform Release: Granular Scan Control, Improved (and Expanded) Data Classes, Alongside a Ton of New Rules

This month, team Open Raven continued to expand the power and capabilities of the platform. Our latest release contains enhancements to existing functionality and several new capabilities. These include incremental scanning, scan budgeting, custom tags for Lambdas, improved data class accuracy, and a ton of new rules covering everything from data stores to compliance frameworks, IAM, and ransomware protection. Here are the highlights. 

Data Scanning

A unique aspect of Open Raven’s architecture is that we analyze data where it resides using serverless functions (e.g., AWS Lambda) within a customer’s own account-- no data is ever moved or accessed by the Open Raven platform itself.  This not only is better for safety and privacy, but also delivers on a cost/performance profile that alternative approaches cannot match.  Since the functions run within the customer’s own account, the associated costs for execution must be both transparent and predictable. No matter how inexpensive we’ve made data scanning, in the short history of cloud computing, no one has ever had a “good” surprise on their AWS bill. 

In this spirit, we’ve released a number of things to drive down costs, improve transparency and prevent unhappy surprises.  First, we’ve recently released a number of performance improvements along with incremental scanning so that only new objects are scanned. And we’re not even close to finished; more to follow in June. Second, we tag the functions running Open Raven data scanning so that it’s clear what they’re doing and as of this month you can now request a custom tag for Lambdas as well. Third, we’ve released a new feature that establishes a maximum “budget” for data scanning.

The budget feature works by setting a limit inside the scan UI. It’s a slider that looks like this:

Scans that have a defined max cost will be canceled once the budget is reached using an average of Lambda pricing across AWS regions. If the max budget limit is reached and the scan is canceled, all scan results up to the point at which the scan was canceled will be available in the Data Catalog and elsewhere. Note that the final cost will be within 10% (+/-) the set amount so if you’re extra cautious, set it under the actual threshold you’d like.

Data Classes

Data classes are the key means of classifying customer data at scale into categories such as country-specific passports and phone numbers, API keys, credit card numbers, and more. They are created with scalability and expandability in mind to serve our petabyte-scale classification loads. There is a continuous two-fold development effort in tuning the existing data classes used and extending to new ones.

We are primarily focusing on tuning existing data classes in order to further improve accuracy (e.g., avoid unintended matches). We developed new methodologies to reduce these false positives, and earlier this month, we released 38 upgraded data classes used to identify:

• Country-specific National Identification Numbers (NINs)
• Country-specific Vehicle Identification Numbers (VINs) 
• Country-specific Tax Identification Numbers (TINs) 
• Country-specific Driver License Numbers (DLNs) 
• Country-specific Phone Numbers (PNs)
• Provider-specific Developer Secrets and API Keys

These changes come as improvements based on customer feedback and telemetry which we use to continuously evaluate our classification efficacy.

Policies and Rules 

Last August, we released Magpie, an open-source CSPM available on GitHub. This month, we enhanced the capabilities of the Open Raven Data Security Platform to discover data-related resource configurations and assess their security posture by integrating Magpie discovery capabilities and rules into the platform. 

This integration introduces to the platform more than 260 new rules governing AWS Aurora, DynamoDB, RDS, Redshift, and S3; compliance with CIS AWS Foundations Benchmarks and PCI-DSS; IAM best practices; and includes additional rules for protecting against and recovering from ransomware attacks.

‍

Bugs and enhancements 

• TSV files are now parsed similarly to CSV files instead of as raw text
• Auto-sort arranges all Data Catalog views from highest to lowest counts
• The Data Catalog now correctly shows only the findings from the data classes selected instead of all data classes
• In Maps, the backup icon is now displaying correctly for backed up assets in map view
• The AWS Account Name field in Data Scan Jobs is correctly filled with AWS Account ID numbers
• Fixed the Account/Settings Modal so that when hovering over the OR Avatar it aligns for a user to be able to select something from the menu