Where are you using WebAssembly?
Wasm promises to let developers build once and run anywhere. Are you using it yet?
At work, for production apps
At work, but not for production apps
I don’t use WebAssembly but expect to when the technology matures
I have no plans to use WebAssembly
No plans and I get mad whenever I see the buzzword
Cloud Services / Observability / Software Development

Amazon CloudWatch Gets Feature Flags, User-Based Monitoring

At AWS re:Invent, AWS revealed a number of new features for its CloudWatch application monitoring platform, including feature flags, A/B testing and real user monitoring.
Nov 30th, 2021 10:49am by
Featued image for: Amazon CloudWatch Gets Feature Flags, User-Based Monitoring

Competitively aligning with other application observability platforms, Amazon Web Services has augmented its Amazon CloudWatch monitoring service with feature flags and A/B testing, the company announced at its annual user conference AWS Re:Invent, being held this week in Las Vegas.

In addition, CloudWatch has taken a step beyond synthetic monitoring of web applications, and can now offer performance data (anonymously collected) based on actual usage.

Feature Flags and A/B Testing

The new Amazon CloudWatch Evidently capability provides developers the ability to control what specific features get rolled out to an application’s users, either through feature flags and/or A/B testing.

With feature flags, a developer can test new features of an application by only rolling them out to a select set of users, to test how the feature operates with a small subset of users, rather than roll it out for the entire user base. Specific conditions for when a feature appears can be determined through a set of if-then statements. Then, as the developers grow confident with the feature, it can be rolled out over larger groups of users, monitoring the technical and business metrics along the way.

Picking a feature to test

The A/B testing capabilities also offered by Evidently are similar to feature flags, but more statistically rigorous. With these services, features can be tested through randomized experiments with multiple variations, allowing the developer to test which feature works best. The success of each variation can then be statistically analyzed. Evidently provides some statistical tools to help do this: a built-in statistical engine can provide insight even while the experimentation runs, offering anytime p-value and confidence intervals.

An e-commerce website could be using A/B testing to evaluate different types of  checkout buttons — different sizes, shapes or colors — to see which one brings generates more revenue, an AWS blog post on Evidently posited.

Although another AWS service, AWS AppConfig, part of AWS Systems Manager, also offers features flags, though the Evidently service offers more advanced capabilities, according to AWS. Other companies, such as LaunchDarkly and Optimizely, have long offered feature flag and A/B testing capabilities as well.

The AWS blog post listed above also provides walk-throughs of how both feature flags and A/B testing works on CloudWatch.

Real User Monitoring

In addition to feature flags and A/B testing, CloudWatch now users what AWS calls “Real-User Monitoring” or RUM. To date, CloudWatch has offered synthetic testing,  or the ability to externally watch web applications for possible problems. In contrast, RUM provides a way to watch the performance of various components as they are deployed by actual end-users.

For end-users, a website’s performance can vary dramatically from the aggregated results offered by synthetic monitoring, pointed out AWS Chief Analyst Jeff Barr in a blog post explaining the new service. Factors such as browser type, browser configuration, user location, connectivity can all affect performance in different ways.

Deploying RUM appears to be quite easy: Register an application with AWS, then add a JavaScript snippet to each Web page, which will dispatch telemetry information back to AWS. You can choose what components you want to monitor.

The AWS console shows performance metrics, and how well they stand against industry standards. The data can also be fed into other Amazon CloudWatch monitoring services, such as CloudWatch ServiceLens or X-Ray.

The metrics reported by Amazon CloudWatch RUM

RUM usage costs $1 for every 100,000 events collected.

Group Created with Sketch.
TNS owner Insight Partners is an investor in: Optimizely, LaunchDarkly, Real.
THE NEW STACK UPDATE A newsletter digest of the week’s most important stories & analyses.