Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.
CI/CD / Security / Software Development

JASK Employs Artificial Intelligence to Automate Smarter Security

Aug 14th, 2017 6:00am by
Featued image for: JASK Employs Artificial Intelligence to Automate Smarter Security

Add JASK to the list of startups looking to relieve the burden on overwhelmed security teams through the use of artificial intelligence. The San Francisco-based company unveiled its JASK Trident platform at the recent Black Hat conference in Las Vegas.

JASK provides an end-to-end view of the network to provide “smart alerts” that help security pros triage problems and use their time more effectively. Co-founder Damian Miller calls it “force multiplier of 10 for security teams,” referring to the security staffing difficulties many organizations face.

The company was born of frustration, Miller said, recounting tales that he and co-founder Greg Martin worked with large enterprise clients, include federal government agencies, whose technology couldn’t keep up with rapidly evolving threats.

“We’re focusing on enterprises that are suffering the most from the complete overload of alert data or signal data from outdated technologies. … Often time, with the legacy technology, when you add the network-level data to it, it just becomes a cost driver. It’s exponentially so much more expensive to index these volumes per day when you’re using outdated technologies, versus using the metadata effectively and bringing it up to the cloud,” Miller said.

The company’s software-as-a-service platform focuses primarily on discovery and investigation. The company hopes to provide recommendations for response and automated response actions later.

Network Visibility

Hosted on Amazon Web Services, it employs open source TensorFlow machine learning technology that originated at Google. It also uses the Apache Spark in-memory computing framework deployed on a version of Hadoop curated by Cloudera called Cloudera CDH.

It monitors in a hybrid cloud or on-prem environments and integrates with existing security tools.

A lightweight package is deployed in the customer’s environment that connects with the stream of data in the network. Through its partnership with Gigamon its reach can be extended to the new SSL decryption-type of appliances. If the customer wishes, it can process log data as well.

It’s all securely streamed to the cloud, where the platform processes the data with machine learning and makes the data ready for near-instant search and exploration.

Rather than having to go through 10,000 unique signals, the analyst might receive three smart alerts with a timeline that says, “These are the signals that have progressed around these assets that indicate there’s a high probability that this type of attack is taking place,” he explained.

The analyst can also interact with the UI, say if a vulnerability assessment scanner always generates 5,000 new signals on Monday. It can alert the system of this routine behavior and set it to watch for any variance in that behavior.

It provides data exploration and visualization capabilities through “notebooks,” and security analysts can add in external and internal context to improve time to insight.

JASK allows every customer to stay in single tenancy while continually benefiting from the learning taking place among its customers. Every JASK user contributes to the pool of knowledge about a specific threat, and new clients immediately tap into all the previous training of the AI model.

“The old-school way was to keep adding appliances to add computing capabilities — adding boxes to add more memory, more storage, more compute. The cloud is much more flexible and dynamic. It enables us to provide access for our customers to the data itself rather than locking it into a proprietary database,” Miller said.

They can do this now through REST APIs. In the future, however, it’s looking at doing this with Amazon S3 as well as public cloud object storage.

Security Niche

Miller and Martin founded the company in January 2016. They had worked together at security software vendor ArcSight, which HPE acquired in 2010. Martin also founded ThreatStream, which became Anomali last year.

The company recently landed $12 million in Series A funding, led by Dell Technologies Capital, bringing its total funding to $14.5 million.

It’s the latest entrant in the market segment Gartner calls SOAR (Security Operations, Analytics and Reporting), which includes players including DFLabs, Hexadite and Swimlane.

“Incident response technologies, such as SIEMs, are powerful for showing current security events, but are too tactical to give all the data needed to understand the strategic value of the current state of security. This leaves security teams the manual job of pulling together data from various sources to get a fleeting glimpse of the big picture as it relates to their security state,” states a Gartner report on the market segment.

“With organizations increasingly centralizing their security operations and providing security services within the enterprise, a niche has opened up for technologies that operationalize security data and orchestrate and consolidate security management processes.”

And a new McAfee report indicates automation is taking off in security practice. From responses from more than 700 security professions, it found:

  • 71 percent of advanced security operations centers use a combination of humans and machines to close cybersecurity investigations in one week or less.
  • Of the most advanced organizations, 37 percent closed threat investigations in less than 24 hours.
  • 68 percent say better automation and threat hunting procedures are needed to reach leading capabilities.
  • Successful cybersecurity teams are three times as likely to automate threat investigation and devote 50 percent more time to actual threat hunting.

Feature image via Pixabay.

Group Created with Sketch.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.