Hi GrowthBook Team, We're using the GrowthBook Flutter SDK for feature flags and experimentation, and we've been investigating an issue where users occasionally receive default or null values instead of their expected experiment variation. Initially, we observed this primarily when SDK initialization failed. To mitigate version shifting in those cases, we implemented our own cache layer that stores the last successfully evaluated value (

lastEvaluatedValue

). On successful evaluations, we update the cache, and when evaluation fails, we fall back to the cached value instead of the default value. This significantly reduced the occurrence of the issue, but it did not eliminate it completely. What we're still seeing is that, in some cases, a feature evaluates to its default value even though: 1. The user is part of an active experiment. 2. The SDK appears to have initialized successfully (no failure logs). 3. Our cache fallback didn't got triggered. To validate evaluations, we rely on the GrowthBook tracking callback for exposure logging. We noticed that in these cases: • No exposure event is fired through the GrowthBook tracking callback. • We also added our own "negative exposure" tracking for null evaluations, but even that does not get trigger here. • This suggests that the SDK is returning a default value without going through the expected experiment evaluation path. (do note we have not define a default fallback feature value at code level). A specific example: • User is assigned to Variant B in an A/B experiment. • Most of the time, the user is correctly evaluated as Variant B and an exposure event is fired. • Occasionally, the same user is evaluated as holdout/default value. • During these occurrences, no exposure event is fired. Our understanding is that if a user is already bucketed into Variant B, they should consistently receive Variant B as long as the experiment configuration and targeting attributes remain unchanged. What is confusing us is the following: • If evaluation had completely failed, our cache layer should have returned the previously evaluated Variant B. • Instead, the SDK is explicitly returning Variant A (the default/holdout value). • Since the SDK returned a value, our fallback cache logic is bypassed. • No exposure event is generated for these evaluations. Given this behavior, we're trying to understand: 1. Under what circumstances can the SDK return the default/holdout value for a user who is otherwise consistently assigned to Variant B? 2. Are there any known Flutter SDK issues related to intermittent evaluation inconsistencies, stale state, cache invalidation, or experiment assignment changes? 3. Is there any scenario where an experiment evaluation could silently fall back to the default value without triggering the tracking callback? We'd appreciate any guidance on what additional diagnostics or logs we should capture to help identify the root cause. Thanks for your help.

Hi @witty-rocket-75751 We’ll investigate this and follow up with you 🙌

hey @powerful-spoon-16837 any findings on it.

Hi, @witty-rocket-75751 Thank you for the detailed write-up. We have been looking into this issue and investigating potential edge cases in the GrowthBook SDK, but since it is difficult to reproduce consistently, we want to make sure we fully understand your setup before drawing any conclusions. On your first question — under what circumstances can the SDK return default/holdout instead of the expected Variant B? The SDK returns the default value without invoking the tracking callback in the following scenarios: • Scenario A — missing or empty hashAttribute. The SDK checks the value of the attribute used for bucketing (e.g.

userId

). If that value is absent from the attributes map, passed as

null

, passed as an empty string

""

, passed as the literal string

"null"

, or if the key name doesn't match the

hashAttribute

in the configuration (e.g.

"userId"

vs

"user_id"

vs

"id"

) — the SDK returns

inExperiment: false

and the tracking callback is never invoked. This also happens if

evalFeature()

is called before

setAttributes()

has received the

userId

— for example, when the user session loads asynchronously after SDK initialization. • Scenario B — targeting conditions not met. If the experiment or feature rule has conditions set (condition, filters, namespace) and the attributes at evaluation time don't satisfy those conditions, the SDK skips the experiment rule and returns the default. The tracking callback does not fire in this case either. • Scenario C — SDK re-initialization with the same userId. If the SDK is re-initialized (e.g. logout → login of the same user), its internal state remembers that this experiment has already been tracked and will not fire the tracking callback again. The variant value itself will be correct, but the missing exposure event may make it look like a default/holdout in your analytics. On your second question — are there any known Flutter SDK issues related to evaluation inconsistencies, stale state, or assignment changes? The SDK uses a deterministic hashing algorithm for bucketing, so for the same

userId

and unchanged experiment configuration, the assigned variant will always be the same. Inconsistencies in evaluation results can occur if: • the

hashAttribute

value changed between calls (Scenario A above); • the experiment configuration changed server-side (updated weights, coverage, or targeting); • or the user's attributes changed and no longer satisfy the experiment conditions. If none of these apply, the most likely cause is an inconsistent state of the attributes map at the time

evalFeature()

is called. On your third question — is there a scenario where evaluation silently returns the default without triggering the tracking callback? Yes, and this is documented SDK behavior. The tracking callback is only invoked when

inExperiment = true

— that is, when the user has successfully been assigned to the experiment. In all other cases, the callback is not called and the SDK does not log an error. To help us investigate further, we have a few additional diagnostic questions: 1. What does

GBFeatureResult.source

return in the problematic cases? 2. Would it be possible to log the full

GBExperimentResult

during a problematic evaluation — specifically the

hashAttribute

and

hashValue

fields? 3. Has the experiment been paused at any point from the GrowthBook dashboard? 4. What

hashAttribute

is specified in the experiment configuration on the server side (not in the code)? Thank you for your patience and for providing such a thorough description of the issue. Looking forward to hearing from you!

hey sorry for the delayed reply: We don't use remote evaluation and we didn't observe users shifting from variant A to B or vice versa (as per gb eval) most of the time the issue is regarding A or B assigned user gets to see the default value. We do observe some issues regarding SDK init failures for certain % of people but in that case gb eval returns null values and we handle null cases from our side. we've also checked the issue we reported not all users are logging out loggin in etc.. it happens in fresh sessions as well. Regarding 1,2 and 4, during unsuccessful evaluation (only in cases of null or undefined values) we log otherwise we don't log if we get a value from sdk itself. so we don't have logs regarding if source or hash attribute is changing or not will add a log from our side but this might take some time to get data as app release is required on our end with the log. and for 3 No we havn’t paused experiment at any point of time. we noticed it when the experiment was initially rolled out at lest say 20% and as we kept on increasing the rollout the numbers increased. the number is around ~2% users. of the overall experiment eligible base.

Hi @witty-rocket-75751 Thanks for the extra detail — it helped us narrow things down a lot. We've been investigating at the SDK source level and comparing the Flutter SDK against the canonical

growthbook-js

reference implementation. A quick summary of where we are. The core behaviour. When an experiment rule is skipped for any reason, the SDK does not return an error or

null

— it falls through to the feature's

defaultValue

and does not fire the tracking callback. This is by design and matches the reference. The important consequence for you: because a real (non-null) value is returned, your null-based fallback is never triggered, and no exposure event is logged — which is exactly the symptom you're describing. We reproduced several distinct, independent paths that produce "non-null default + no exposure", and confirmed each one locally with tests: 1.

hashAttribute

value is empty / the literal string

"null"

at eval time. If

attributes[<hashAttribute>]

is missing, empty, or stringifies to

"null"

when

evalFeature

runs, the user is silently excluded → default value, no exposure. This is intermittent by nature (e.g. an async session load that hasn't populated the id yet on a cold start). 2. Cache-vs-network timing on startup. The SDK loads features from cache first, before (or instead of, on a failed refresh) the network update completes. If the loaded payload is stale/partial, the feature exists but its experiment rule doesn't yet → it falls through to the default. Note this is not the "init failure → null" case you already handle: here the cache load "succeeds", so no init-failure callback fires, and the value is a non-null default. This is also intermittent and happens on fresh sessions. 3. Exposure de-duplication across instances. The tracking callback is de-duped per (hashAttribute + value + experiment + variation). If the SDK is re-initialized during the app's lifetime, the value stays correct but the exposure may not re-fire — so in analytics it can look like the user wasn't in the experiment. 4. Phase / seed re-randomization on rollout changes. This is GrowthBook dashboard behaviour rather than the SDK: when you change an experiment's traffic % (coverage), GrowthBook can start a new phase and generate a new seed. Since the seed feeds the hash, a new seed re-buckets every user — so someone previously in Variant B can land in a different variation or fall outside the (still partial) coverage and get the default, with no exposure. This would scale exactly with each rollout step. One caveat: a seed change would normally also cause some users to shift between variations (A↔️B), which you said you don't see — so we'd like to confirm whether your rollout steps actually changed the seed. We also checked a hashing/rounding edge case, but measured it across 200k synthetic ids and it accounts for <0.05% (and 0% for a standard 50/50 split), so it does not explain your ~2%. To pin down which path is yours, could you share (no real user data needed): 1. The experiment/feature config from your GrowthBook API response — specifically

coverage

,

hashVersion

,

hashAttribute

, `weights`/`ranges`, the experiment `seed`/`phase`, and any `condition`/`filters`/`namespace`. (Or your SDK client key via DM and we'll pull it ourselves — read-only.) 2. A few setup details: ◦ Is`backgroundSync`enabled? What`ttlSeconds`? ◦ Do you call`evalFeature`immediately after`initialize()`, or wait for it to complete? ◦ Do you ever re-initialize the SDK during a session (e.g. on login)? ◦ When you increased the rollout, did each step start a*new phase / re-randomize traffic*(i.e. did the experiment`seed`change)? ◦ Are you using*Sticky Bucketing*(a`StickyBucketService`)? It preserves a user's assigned variation across seed/phase changes. 3. ⚠️ Please do not send real user IDs or any personal data. For the failing evaluations, we only need the shape of the

hashAttribute

value: its runtime type (e.g.

String

,

Null

,

int

), and whether it can ever be empty /

null

/ the literal string

"null"

. A redacted/synthetic example that preserves the format is perfect. One thing that will narrow this down quickly: if

coverage

is

1

and the rule has no `condition`/`filters`/`namespace`, then returning the default can only mean the experiment rule was skipped entirely — not "a different variation was chosen". In that case the cause is almost certainly the

hashAttribute

value being empty/`null`/`"null"` at eval time, or a stale/partial cached payload — so the single most useful data point is the runtime type and shape of

attributes["<hashAttribute>"]

in the failing cases. Could you also confirm the

coverage

value on the rule? With the config + value shape we can reproduce your exact case locally and confirm the root cause — we don't need to wait on your app release for that; the logging on your side would just be the final confirmation. Thanks again for your patience here.

Thanks for the quick reply @powerful-spoon-16837 here's the things you asked for: 1. The experiment configuration:

"app-ia-v12.2": {
  "defaultValue": "app_v11",
  "rules": [
    {
      "condition": {
        "id": "some user id"
      },
      "force": "app_v11"
    },
    {
      "condition": {
        "app_package_name": {
          "$in": [
            "some app packages"
          ]
        }
      },
      "force": "app_v12"
    },
    {
      "condition": {
        "id": {
          "$in": [
            "some ids..."
          ]
        }
      },
      "force": "app_v12"
    },
    {
      "coverage": 0.8,
      "hashAttribute": "id",
      "seed": "783654eb-2e1c-4ee9-bd2b-83cf64d03ad7",
      "hashVersion": 1,
      "variations": [
        "app_v11_control_group",
        "app_v12"
      ],
      "weights": [
        0.5,
        0.5
      ],
      "key": "app-ia-v12.2",
      "phase": "0",
      "meta": [
        {
          "key": "0"
        },
        {
          "key": "1"
        }
      ]
    }
  ]
}

there are some saved group and targeting attribute filters for our internal testing present. 2. our init setup is client driven no remote eval or background sync, we wait init to get completed have completer based logic added for that.if init is in progress we wait till it gets completed before actually evaluating the requested feature value.

final GrowthBookSDK gb = await GBSDKBuilderApp(
                hostURL: ourHostUrl,
                apiKey: ourGbApiKey,
                attributes: _attributes,
                gbFeatures: _defaultValues,
                growthBookTrackingCallBack: (gbTrackData) {
                  // send event to our analytics manager
                },
                onInitializationFailure: _errorCallBack)
            .initialize()

• We do kinda re-init the sdk not acutally creating a new instance of it our instance is singleton but we clear the attributes and then once we get new user's attribute during login we call setAttributes. a callout regarding this not all users in that <2% are login/logout cases a lot of them faced this in simple new sessions where user was already logged in. • No we don't re-randomize or start a new phase when we increase the rollout % and we don't use sticky bucketing either. •

hashAttribute

is unique ids corresponding to users. it is a string datatype. ideally if the user is logged in as mentioned above we set this user id via

setAttributes

ideally that hash attribute shouldn’t be null but as mentioned we do configure our

id

via setAttribute and not actually during init but just after init once we get user's id. we do refresh the gb instance but lmk if this can cause artefact.

Future<void> refreshGbValues() async {
    if (_instance != null) {
      await _instance!.refresh();
    }
  }

coverage is 0.8 as of now. lmk if you need any more info on any specific point.

Thanks for the config and setup details. Based on your experiment config (coverage 0.8, hashVersion 1, 50/50, with bucket boundaries landing exactly on a 4-decimal grid), we've ruled out bucketing/rounding and seed/phase issues. We've reproduced this locally against your exact config at the SDK level (

GBSDKBuilderApp → initialize() → setAttributes → feature()

): on a single SDK instance, calling

feature('app-ia-v12.2')

before

setAttributes(id)

returns the non-null default

app_v11

with no exposure, and the same instance returns the proper variation with an exposure once

id

is set. So this isn't a hypothesis about the algorithm - it's about the timing of

id

in your app. In short: when

id

is missing, empty, or the literal string

"null"

at the exact moment

evalFeature

runs, the SDK skips the experiment rule and returns the non-null

defaultValue

(

app_v11

) with no exposure - which matches your symptom exactly. To confirm, could you clarify (1 and 2 are the key ones): 1. What does your completer wait on -

initialize()

only, or also

setAttributes(id)

? Since`id`is set via`setAttributes()`after`initialize()`, is there a window where init has completed but`setAttributes(id)`hasn't run yet? 2. *How and when is

id

obtained?*Synchronously at startup, or fetched asynchronously? Can`evalFeature('app-ia-v12.2')`ever be called before`setAttributes(id)`? 3. Why isn't

id

passed directly into

attributes

of

GBSDKBuilderApp(...)

? If it's available before init, that removes the window entirely. 4. *Can

attributes['id']

ever be

""

or the literal string `"null"`*(e.g. from serializing a null upstream)? The SDK treats both as "missing". 5. (secondary / rule-out) How often is

refresh()

called during an active session, and could an

evalFeature

run around the same time? We just want to exclude a stale/partial payload being swapped in mid-session. A quick way to verify on your side - without a release and without sending any user data - is to log only the timing and shape of

id

(never the value itself) at three points: when

initialize()

completes, when

setAttributes(id)

runs, and inside every

evalFeature

. If you ever see an

evalFeature('app-ia-v12.2')

firing with a non-valid

id

before

setAttributes(id)

, that confirms the root cause. Please don't send real user IDs or any personal data - only the runtime type and shape (

null

/

empty

/

"null"

/

valid

) are needed. Diagnostic snippet (no PII, no release blockers):

T// Requires: import 'package:growthbook_sdk_flutter/growthbook_sdk_flutter.dart';
import 'dart:developer';

/// Reduces a raw id to a non-PII "shape" label so we can log STATE without
/// ever logging the actual identifier.
String idShapeOf(dynamic id) {
  if (id == null) return 'null';
  if (id is String && id.isEmpty) return 'empty';
  if (id.toString() == 'null') return '"null"';   // e.g. serialized null
  return 'valid';
}

/// One shared stopwatch started at app launch, so all timestamps are comparable.
final Stopwatch _launch = Stopwatch()..start();

// 1) Right after initialize() completes:
void onInitDone(Map<String, dynamic> attributes) {
  log('[GB] init done @${_launch.elapsedMilliseconds}ms '
      'id.shape=${idShapeOf(attributes['id'])}');
}

// 2) Right when setAttributes(id) runs:
void onSetAttributes(Map<String, dynamic> attributes) {
  log('[GB] setAttributes @${_launch.elapsedMilliseconds}ms '
      'id.shape=${idShapeOf(attributes['id'])}');
}

// 3) Wrap EVERY feature read in this so each eval is timestamped + shaped:
GBFeatureResult evalTraced(GrowthBookSDK gb, String key) {
  final id = gb.context.attributes?['id'];
  final result = gb.feature(key);

  final shape = idShapeOf(id);
  log('[GB] eval($key) @${_launch.elapsedMilliseconds}ms '
      'id.runtimeType=${id.runtimeType} id.shape=$shape '
      'source=${result.source} value=${result.value}');

  // The smoking gun for the race:
  if (key == 'app-ia-v12.2' &&
      shape != 'valid' &&
      result.source == GBFeatureSource.defaultValue) {
    log('[GB] ⚠️ HIT: app-ia-v12.2 evaluated with non-valid id '
        '→ defaultValue WITHOUT exposure');
  }

  return result;
}

Sure @powerful-spoon-16837 i'll check and get back to you in some time for the same. i also had another query regarding growthbook projects. Currently our org doesn’t have projects linked to their respective clients and we were facing scalability issue. as all our features are fetched by all the clients we wanted to configure projects for their respective clients. can you let me know if i configure a project and update my features and experiments to respective feature will there be any impact on ongoing experiments like users shuffling due to change in salt or hash etc..? or we can migrate without worrying about the same.

On the Projects question: assigning a feature/experiment to a Project does not affect bucketing - users will not re-shuffle just from the project move. A Project is an organizational/access grouping and is not an input to the assignment hash. Assignment is a pure function of the experiment

seed

, the

hashAttribute

value,

hashVersion

, and the `coverage`/`weights` ranges - none of which change when you move a feature between projects. (On "salt": the experiment

seed

is the bucketing salt and only changes on a *new phase with re-randomize*; the org-level "secure attribute salt" is for hashing private targeting attributes and is unrelated to assignment.) So you can migrate without re-shuffling, as long as it's a reassignment of the existing features/experiments - keep the same keys,

seed

, and phase; don't recreate them as new objects and don't start a new phase. What does move users: a new phase with re-randomize, changing variation weights/split, decreasing coverage in the same phase (some in-experiment users fall out to the default), or changing the hashAttribute/hashVersion. (Increasing coverage with the same split is safe - it only adds users.) ⚠️ The real thing to plan is SDK Connection scoping, not bucketing: once features are filtered by project, each client's SDK Connection must include the project(s) whose features it needs - otherwise that client stops receiving those features and

evalFeature

returns the default with no exposure. Suggested order: (1) scope each client's SDK Connection to the right project(s), (2) reassign features/experiments to projects (same keys/seed/phase), (3) verify on one client key/staging that the payload contains exactly the expected features before rolling out. How to verify the seed didn't change, yourself: take an existing experiment (not a new one — new experiments get a fresh random seed), note its

seed

in the API payload, move it to the new project, and re-fetch — the

seed

,

hashAttribute

,

hashVersion

,

weights

, and

coverage

should be byte-identical, and no new phase should be added.

i tried mocking the issue by creating some delay as per that the issue should be resolved but we still need to push it to prod and cross-check on actual users.