It looks like, in order to remove a data source, you have to delete it. To do that, you have to delete all its metrics.
But if you still want to preserve the historical experiments, it seems like you'd be stuck? Is it possible to remove the data source without breaking those?
Or could it be an option to archive a data source instead?