https://www.growthbook.io/ logo
Join Slack
Powered by
I tried a simple experiment which shows no value. ...
# announcements
r
I tried a simple experiment which shows no value. query result as per dashboard is
Copy code
{
  "dimensions": [
    {
      "dimension": "All",
      "variations": [
        {
          "variation": null,
          "stats": {
            "mean": 1,
            "count": 45409,
            "stddev": 0
          }
        },
        {
          "variation": null,
          "stats": {
            "mean": 1,
            "count": 4,
            "stddev": 0
          }
        }
      ]
    }
  ]
}
but according to bigquery is am I missing something? why variation is null on dashboard?
f
If you run the same query on your BigQuery, you get values, but not in GrowthBook?
can you share the query?
r
Yeah the image is the result of bugquery
f
Is there any warning on the results page in GrowthBook about unknown variation IDs?
r
no warning for some reason it doesn’t interpret the variations it gets from bigquery properly
basically I don’t know why “variation” is null on growthbook but has values (control / enabled) on bigquery results. I think I am missing some trivial setting
BQ result in json
Copy code
[
  {
    "variation": "enabled",
    "dimension": "All",
    "users": "46267"
  },
  {
    "variation": "control",
    "dimension": "All",
    "users": "5"
  }
]
the other query is same. just users is renamed to count as my metric is trivial
Copy code
{
  "dimensions": [
    {
      "dimension": "All",
      "variations": [
        {
          "variation": null,
          "stats": {
            "mean": 1,
            "count": 46266,
            "stddev": 0
          }
        },
        {
          "variation": null,
          "stats": {
            "mean": 1,
            "count": 4,
            "stddev": 0
          }
        }
      ]
    }
  ]
}
f
On the experiment results page, can you click the 3 dots next to the Update Data button and click "View Queries". You should see the raw SQL and the rows returned. Do those rows look identical to what you get from BigQuery directly?
r
Already did and pasted the result. Variants are null in growthbook but not in bugquery results
here is the query:
Copy code
-- Number of users in experiment
WITH __rawExperiment as (
    SELECT
      DATETIME(server_time) as timestamp,
      user_id,
      user_id as anonymous_id,
      experiment_id,
      variation_id

    FROM
      events
),
__experiment as (
  -- Viewed Experiment
  SELECT
    e.user_id as user_id,
    e.variation_id as variation,
    e.timestamp as actual_start,
    DATETIME_ADD(e.timestamp, INTERVAL 3 DAY) as conversion_end,
    DATETIME_SUB(e.timestamp, INTERVAL 30 MINUTE) as session_start
  FROM
    __rawExperiment e
  WHERE
    e.experiment_id = 'consolidated_apm_settings_panel_frontend'
    AND e.timestamp >= DATETIME "2022-01-18 04:11:00"
),
__distinctUsers as (
  -- One row per user/dimension/variation
  SELECT
    e.user_id,
    e.variation,
    'All' as dimension
  FROM
    __experiment e
  GROUP BY
    variation,
    dimension,
    e.user_id
) -- Count of distinct users in experiment per variation/dimension
SELECT
  variation,
  dimension,
  COUNT(*) as users
FROM
  __distinctUsers
GROUP BY
  variation,
  dimension
click to minimize
ok built the image from master and it told me more info:
Copy code
Warning: Expected 2 variation ids (0, 1), but database returned a different set (control, enabled).
looks like it expects numbers for variants
so 
"growthbook/growthbook:latest"
seems old. the one from master let me change ids
f
Hmm. The 
latest
tag for docker should point to the most recent commit
For variation ids, by default we expect numbers, but you can change that. On the "Info" tab of the experiment, if you click the Edit button, you can set variation ids
r
yeah that’s the thing. I wasn’t seeing any of these options until I changed to locally build master
f
You might need to force docker to pull down the latest image.
Copy code
docker-compose pull growthbook
docker-compose stop growthbook
docker-compose up -d growthbook
✅ 1
154 Views
Open in Slack
PreviousNext
Powered by