Optimizely Updates Local Holdouts in Feature Experimentation

On October 29, 2026, Optimizely announced a change in its Feature Experimentation release notes to how SDKs evaluate local holdouts. A local holdout now excludes a visitor only from the flag rules included in that holdout, and flag evaluation continues with the next rule. Previously, entering a local holdout stopped evaluation of the entire flag. The change affects the JavaScript, Ruby, C#, Swift, Go, Python, and Java SDKs.

How the SDK Evaluates a Flag When a Visitor Enters a Local Holdout

A holdout sets aside a small group of visitors who are excluded from experiments in order to measure the combined impact of an experimentation program. Global holdouts, introduced in Feature Experimentation on March 13, 2026, apply to an entire project: visitors in this group receive the off variation of flags regardless of ongoing A/B tests, targeted deliveries, and multi-armed bandits. Local holdouts, introduced on July 22, 2026, limit exclusion to selected flag rules. This makes it possible to measure the combined impact of a particular set of experiments, such as a series of checkout tests, while other rules in the project continue to receive full traffic.

Before the October 29 change, when a visitor entered a local holdout, the SDK returned the holdout variation and skipped the remaining rules of the flag. The new logic matches how the documentation describes a local holdout: it excludes the visitor from each rule in the set, not from the entire flag.

When a visitor in the holdout reaches a rule included in that holdout, Feature Experimentation skips that rule, records the visitor in the holdout group for that rule, and evaluates the next flag rule. The visitor can still receive a variation from a subsequent rule, including the Then, for everyone else rule. If no rule serves the visitor, the flag is off for that visitor. Optimizely assigns all other visitors to rules as usual.

Optimizely aggregates results across all selected rules and compares the holdout group with visitors who were exposed to those rules. Each entry into a local holdout sends its own impression event in the existing format, regardless of the sendFlagDecisions value. Global holdouts are unchanged.

Configuring a Local Holdout and Its Limitations

A holdout is created under Flags > Holdouts by selecting Create Holdout. In the Scope section, the user selects an environment and the Local holdout type, then adds at least one rule, including A/B tests or other flag rules in the project. Metrics are added at the holdout level and apply to all its rules. Optimizely recommends excluding 1% to 5% of visitors from each rule and displays a warning when the percentage exceeds 5%. The holdout can optionally be restricted to audiences; Everyone is the default, and audience conditions on an individual experiment take precedence over holdout audience conditions. A newly created holdout is a draft and does not affect traffic until it is started.

Editor, Collaborator, and Admin roles can create and edit local holdouts, while results are available to all roles, including Viewer. Documentation notes that feature availability may depend on the subscription plan, so details should be confirmed with a Customer Success Manager. The documentation specifies the following rules for local holdouts:

• The set of included rules is fixed when a holdout starts and cannot be changed, because differences in exposure duration would undermine the statistical validity of the results.
• An entire rule is included in a holdout; individual variations within a rule cannot be selected.
• A running holdout cannot be paused or deleted; it can only be ended. After it ends, all visitors in the holdout are reassigned and receive variations from active rules. The ended holdout cannot be restarted.
• A visitor cannot be in a global and a local holdout at the same time. If conditions overlap, the global holdout takes precedence.
• If a visitor qualifies for several local holdouts for the same rule, the holdout higher in the configured priority order applies. Different local holdouts can exclude the same visitor from different rules of a single flag.

SDK Versions and Availability Dates

Optimizely's release notes date the change to October 29, 2026, while the SDK versions containing it appeared on GitHub over the following days. Public release records were found for the following versions:

• Go SDK 2.7.0 — dated October 30, 2026, in the changelog.
• Java SDK 4.7.0 — dated October 30, 2026, in the changelog; the release was published on November 2.
• C# SDK 4.6.0 — November 2, 2026.
• Swift SDK 6.1.0 — November 2, 2026.
• JavaScript SDK 6.7.0 — November 3, 2026.

Optimizely's release notes also name Python and Ruby among the affected SDKs, but public GitHub release lists for those SDKs did not show versions containing the change at the time of the original review. Their version numbers should therefore be checked in the SDK documentation. Because rule evaluation takes place inside the SDKs, an application needs a version containing this change for local holdouts to work according to the new logic.

About Optimizely Feature Experimentation

Optimizely Feature Experimentation is part of the company's feature management and experimentation solutions. Its product marketing page is titled Feature Management. The company describes it as a tool for safe releases, combining feature flags, staged rollouts, instant feature shutdown, dynamic configuration, and flag lifecycle management, while connecting flags to business metrics. The vendor states that it supports more than 20 SDKs for backend systems, mobile applications, and edge environments, and also mentions a remote MCP server for managing experiments and flags from AI tools. The product page is aimed at product and engineering teams.

Sources

1. Optimizely Docs — 2026 Feature Experimentation Release Notes (July 22 and October 29; updated November 10, 2026)
2. Optimizely Docs — Local Holdouts (updated November 10, 2026)
3. Optimizely Docs — Global Holdouts (updated October 23, 2026)
4. Optimizely GitHub — Go SDK 2.7.0
5. Optimizely GitHub — Java SDK 4.7.0
6. Optimizely GitHub — C# SDK 4.6.0
7. Optimizely GitHub — Swift SDK 6.1.0
8. Optimizely GitHub — JavaScript SDK 6.7.0
9. Optimizely GitHub — Python SDK Releases
10. Optimizely GitHub — Ruby SDK Releases
11. Optimizely — Feature Management