Will real-time data processing replace batch processing?
At Confluent's user conference, Kafka co-creator Jay Kreps argued that stream processing would eventually supplant traditional methods of batch processing altogether.
Absolutely: Businesses operate in real-time and are looking to move their IT systems to real-time capabilities.
Eventually: Enterprises will adopt technology slowly, so batch processing will be around for several more years.
No way: Stream processing is a niche, and there will always be cases where batch processing is the only option.
Security / Software Development

GitHub Now Offers Secrets Scanning For Free

Free scanning for secrets? I like this latest GitHub offering!
Dec 19th, 2022 6:00am by
Featued image for: GitHub Now Offers Secrets Scanning For Free

When I started programming, no one would ever put secrets in their code, such as passwords, credentials, keys, and access tokens. It was just asking for trouble. But then along came code-driven automation with secrets. Suddenly we often checked secrets into our code. Then, with Software as a Service (SaaS) and Infrastructure as a Service (IaaS), we’d often insert tokens to invoke other services into our code. The answer to this security problem is, of course, to find and remove them before they land in production. That’s easier said than done. Now, GitHub, with its Secret scanning partner program will let you scan for your secrets in your code for free.

I like this idea! A lot.

Step by Step

To use it, you must go through the following steps:

  1. To get the enrollment process started, email
  2. Then inform GitHub of the secrets you want to scan for and create regular expressions to capture them.
  3. For secret matches found in public repositories, create a secret alert service that accepts webhooks from GitHub that contain the secret scanning message payload.
  4. Implement signature verification in your secret alert service.
  5. Implement secret revocation and user notification in your secret alert service.
  6. Provide feedback for false positives (optional).

This service has already proven its worth. GitHub can scan repositories for 200+ token formats. In 2022 to date, GitHub notified its partners of over 1.7 million potential secrets exposed in public repositories.

Using the Service

The scanning can reveal your leakable secrets. You’re in charge. So, for example, if it’s not possible to notify a partner, say the keys to your self-hosted HashiCorp Vault are exposed, you’ll get the word that you may be in trouble. The secrets alert also makes it easy to track them across all alerts. That way, you can drill deeper into the leak’s source and audit actions taken.

Specifically, once-secret scanning alerts are available on your repository, you can watch them via your repository’s settings under “Code security and analysis” settings. You can see any detected secrets by navigating to the “Security” tab of your repository and selecting “Secret scanning” in the side panel underneath “Vulnerability alerts.” There, you will see a list of any detected secrets, and drill down on any alert to reveal the compromised secret, its location, and suggested remediation action.

That’s great, but wouldn’t it be even nicer to get secret alerts by pushing rather than having to manually look at your setting? Or, better still, get the alerts as the secrets are found of code getting close to delivery?  Why yes, it would be. But, for that, you’ll need to pay extra.

To do this, you must subscribe to GitHub Advanced Security. This service is only available to GitHub Enterprise customers.

Helping Developers

According to Intel Software Engineering Director David Florey, it works well. “If I attempt to push a secret, I immediately know it. GitHub’s secret scanning push protection stops me before a secret is pushed into the code base, saving me tons of time. If instead, I rely solely on external scanning tools to scan the repository after the secret’s already been exposed, I’ll need to revoke the secret and refactor my code quickly. The integration of GitHub’s secret scanning and push protection directly in a developer’s flow saves time and helps educate developers on best practices.”

He’s got an excellent point. To try GitHub Advanced Security in your organization or see a demo, please reach out to your GitHub sales partner.

In the meantime, I highly recommend you try the free service. If you think it’s as useful as I believe you will, in 2023, you’ll want to talk to your CIO about adding to your developer budget.

Group Created with Sketch.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.