I tried creating a new Phase of a test with 0% traffic allocation, however, users were still making it into my test buckets. Is this an expected behavior? Seems like a bug to me

Do you have sticky bucketing enabled?

Looks like it was enabled:

"disableStickyBucketing": false

would that only have applied to users that had already loaded the page in the prior phase?

It keeps users enrolled in the variant they first saw, and I haven’t read that code, but it seems that it checks sticky state without looking at the precent rollout

I grabbed the distinct user ids of all events with “experiment_started” after I started my second phase, totaling 389 users. I could only find 4 of those ids in my log table from the first phase. Wouldn’t that suggest that 385 users were new when exposed to the phase 2 configuration with the 0% allocation?