Hello! I noticed there are missing rows for the pe...
# announcementso
orange-magician-36994
03/13/2023, 10:00 PMHello! I noticed there are missing rows for the percent change p values when I export the results from an experiment dashboard (see screenshot). I'm wondering if the missing values are a bug, or just not reported because the numbers are so small (which seems odd compared to some of the other pvalues listed (e.g., 1.08407061105709E-10). The issue only occurs when I export the results to a csv file- they appear just fine on the dashboard.
@wide-jordan-14758 @ambitious-apartment-58735
h
helpful-application-7107
03/13/2023, 11:26 PMHmm, I can't seem to recreate this bug myself. Could you share a quick screen shot of the p-value column from the results table?
helpful-application-7107
03/13/2023, 11:26 PMThanks for reporting it, btw.
o
orange-magician-36994
03/13/2023, 11:33 PMOf course! The p-values for the missing rows still appear (as p < .001). I was worried maybe "p < .001" was defaulting for missing values, but the csv contains p-values for metric #5
h
helpful-application-7107
03/13/2023, 11:34 PMYou know, it could be typescript actually rendering these values as 0 when we check if the pValue exists at all in our snapshot.
helpful-application-7107
03/13/2023, 11:35 PMI think your hunch is right.
o
orange-magician-36994
03/13/2023, 11:37 PMOh boy. Glad I caught it then!
orange-magician-36994
03/13/2023, 11:38 PMIs there a reason the p-values wouldn't be calculating?
h
helpful-application-7107
03/13/2023, 11:38 PMOh!
helpful-application-7107
03/13/2023, 11:38 PMno i mean the pvalues are fine in the UI
🙌 1
helpful-application-7107
03/13/2023, 11:39 PMit's just that when we process the data from your snapshot for the CSV, we are inappropriately treating 0 as missing (and TypeScript thinks some really tiny number, even smaller than 1e-10, is actually 0).
helpful-application-7107
03/13/2023, 11:39 PMI can land a fix, but there's a decent change that the values end up being actually 0 in your CSV, becauase we've lost the precision to render some tiny number (e.g. 1e-39 or something)
o
orange-magician-36994
03/13/2023, 11:40 PMThat's good to know. I don't think it'll affect us if the numbers are truly that small.
h
helpful-application-7107
03/13/2023, 11:41 PMIt won't. You can see from the UI that the confidence intervals are very far from 0 in those rows, so the p-values will be super tiny.
🙏 1
helpful-application-7107
03/13/2023, 11:44 PMWe probably won't land a fix until later in the week, but just know the results in the UI are unaffected by this bug and All of your missing pvalues here are very close to 0 (< 1e-10, at least)
o
orange-magician-36994
03/13/2023, 11:46 PMNot a problem thank you so much for clarifying!
27 Views